Commit Graph

863 Commits

Author SHA1 Message Date
Patrick Robertson
2c87474186 Change path for scripts 2025-02-11 18:37:03 +00:00
Patrick Robertson
e65a99078f Slight tweaks of toctrees + reordering 2025-02-11 18:28:21 +00:00
Patrick Robertson
62154ddfef Further tweaks and fixes 2025-02-11 14:37:29 +00:00
Patrick Robertson
29901da601 Merge branch 'load_modules' into docs_update 2025-02-11 14:10:56 +00:00
Patrick Robertson
895c843f04 Add a cheat sheet for configs and better folder structure for core modules 2025-02-11 14:06:53 +00:00
Patrick Robertson
2f51d3917a Further addition to docs: creating modules, configurations, installation 2025-02-11 13:49:30 +00:00
Erin Clark
aa5ac18d6a Merge pull request #189 from bellingcat/add_module_tests
- Add module tests
- Fix some storage related bugs 
- Separate the modules setup() method and base module config_setup()
2025-02-11 13:11:41 +00:00
Patrick Robertson
7d87b858d6 Merge branch 'load_modules' into docs_update 2025-02-11 13:09:38 +00:00
erinhmclark
c8cd7ea63c Merge branch 'load_modules' into add_module_tests
# Conflicts:
#	src/auto_archiver/modules/telethon_extractor/telethon_extractor.py
2025-02-11 13:08:08 +00:00
msramalho
977618b4ce doc: adds note about telethon vs telegram extractors 2025-02-11 13:04:59 +00:00
msramalho
d90d3cec28 fix telethon_extractor setup 2025-02-11 13:03:18 +00:00
msramalho
977f06c37a renames api_db property for clarity 2025-02-11 12:56:33 +00:00
msramalho
5c59029221 updates api_db for new API endpoint 2025-02-11 12:53:58 +00:00
msramalho
4eeb39477c improves gsheetdb feedback on retrieve sheet failure 2025-02-11 12:53:46 +00:00
msramalho
6fdd5f0e66 fix cases of single : vs :: in entrypoint 2025-02-11 12:53:12 +00:00
msramalho
e6594ad3dc merge result into cached results for context preservation 2025-02-11 12:52:42 +00:00
msramalho
7309cd32e7 fix: context to be updated on Metadata.merge 2025-02-11 12:51:17 +00:00
erinhmclark
d1d6cde008 Set mock timestamp without z format 2025-02-11 12:27:48 +00:00
erinhmclark
5e2e93382f Test fixes for 3.10 compliance. 2025-02-11 12:17:42 +00:00
erinhmclark
f97ec6a9e0 Fixed S3 module import 2025-02-11 11:58:28 +00:00
erinhmclark
89d9140d15 Fixed setup/ config_setup reference 2025-02-11 11:47:11 +00:00
erinhmclark
1792e02d1d skip authenticated tests in test_gdrive_storage.py 2025-02-11 11:34:36 +00:00
erinhmclark
18666ff027 skip authenticated tests in test_gsheet_feeder.py 2025-02-11 11:28:24 +00:00
erinhmclark
a69ac3e509 Fix file hash reference in S3 tests 2025-02-11 09:46:22 +00:00
Patrick Robertson
ed81dcdaf0 Remove dangling 'b = ' from config.py 2025-02-10 23:07:03 +00:00
Patrick Robertson
e7273bc741 Fix link 2025-02-10 23:04:41 +00:00
Patrick Robertson
dbc564e18b Add sphinx_book_theme theme to poetry 2025-02-10 22:58:52 +00:00
Patrick Robertson
2650cd8fb2 Use a script to auto-generate documentation for the core modules from the manifest file 2025-02-10 22:51:04 +00:00
erinhmclark
8d894066f2 Merge branch 'load_modules' into add_module_tests
# Conflicts:
#	src/auto_archiver/modules/gsheet_feeder/gsheet_feeder.py
#	src/auto_archiver/utils/misc.py
2025-02-10 19:00:05 +00:00
erinhmclark
3dae2337a1 remove cdn_url check before storage. 2025-02-10 18:56:46 +00:00
erinhmclark
e97ccf8a73 Separate setup() and module_setup(). 2025-02-10 18:07:47 +00:00
erinhmclark
2c3d1f591f Separate setup() and module_setup(). 2025-02-10 17:25:15 +00:00
msramalho
12f14cccc9 fixes gsheet feeder<->db connection via context. 2025-02-10 16:58:35 +00:00
msramalho
ab6cf52533 fixes bad hash initialization 2025-02-10 16:45:28 +00:00
Patrick Robertson
824728739a Start fleshing out the docs more - rearrange, separate out modules section, move files over to md (from rst) 2025-02-10 16:24:16 +00:00
erinhmclark
c4bb667cec Merge branch 'load_modules' into add_module_tests
# Conflicts:
#	src/auto_archiver/modules/s3_storage/s3_storage.py
#	src/auto_archiver/utils/gsheet.py
#	src/auto_archiver/utils/misc.py
2025-02-10 16:17:08 +00:00
erinhmclark
f311621e58 Small fixes.
Add timestamp helper method.
2025-02-10 15:57:42 +00:00
msramalho
15abf686b1 decouples s3_storage from hash_enricher 2025-02-10 15:48:54 +00:00
msramalho
8fb3dc754b fixing telethon extractor to use default entrypoint 2025-02-10 14:59:51 +00:00
msramalho
7c848046e8 adds better info about wrong/missing modules 2025-02-10 14:59:32 +00:00
Patrick Robertson
f3f6b92817 Implementation test cleanup 2025-02-10 12:43:21 +00:00
Patrick Robertson
74207d7821 Implementation tests for auto-archiver 2025-02-10 13:27:11 +01:00
Patrick Robertson
e9dd321dcd Fix setting cli_feeder as default feeder on clean install 2025-02-10 13:06:24 +01:00
Patrick Robertson
1fad37fd93 Remove blank file 2025-02-07 23:08:30 +01:00
Patrick Robertson
63aba6ad39 Fix sphinx-autoapi imports 2025-02-07 21:54:49 +01:00
erinhmclark
950624dd4b Fix S3 storage to media in whisper_enricher.py. 2025-02-07 20:26:00 +00:00
erinhmclark
2920cf685f Small fixes to whisper_enricher.py. 2025-02-07 12:35:40 +00:00
erinhmclark
e9ad1e1b85 Pass media to storage cdn_call 2025-02-06 22:01:55 +00:00
erinhmclark
266c7a14e6 Context related fixes, some more tests. 2025-02-06 16:53:00 +00:00
erinhmclark
67504a683e Merge branch 'load_modules' into add_module_tests 2025-02-06 10:13:37 +00:00