bellingcat/auto-archiver
1
0
Fork 0
mirror of https://github.com/bellingcat/auto-archiver.git synced 2026-06-08 03:18:28 +03:00
Code Issues Packages Projects Releases Wiki Activity
855 Commits 19 Branches 124 Tags
e507fc81d2ed8c175a79891308e7a78bd397d213
Commit Graph

855 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
msramalho
e507fc81d2 improves mimetype guessing, previously file.sub.something would not have an extension 2025-02-11 15:02:49 +00:00
msramalho
5478ed3860 bsky fix media fetching 2025-02-11 15:02:00 +00:00
msramalho
47d1dc9d47 typing warnings fixed 2025-02-11 15:01:37 +00:00
Erin Clark
aa5ac18d6a Merge pull request #189 from bellingcat/add_module_tests 
- Add module tests
- Fix some storage related bugs 
- Separate the modules setup() method and base module config_setup()
 2025-02-11 13:11:41 +00:00
erinhmclark
c8cd7ea63c Merge branch 'load_modules' into add_module_tests 
# Conflicts:
#	src/auto_archiver/modules/telethon_extractor/telethon_extractor.py
 2025-02-11 13:08:08 +00:00
msramalho
977618b4ce doc: adds note about telethon vs telegram extractors 2025-02-11 13:04:59 +00:00
msramalho
d90d3cec28 fix telethon_extractor setup 2025-02-11 13:03:18 +00:00
msramalho
977f06c37a renames api_db property for clarity 2025-02-11 12:56:33 +00:00
msramalho
5c59029221 updates api_db for new API endpoint 2025-02-11 12:53:58 +00:00
msramalho
4eeb39477c improves gsheetdb feedback on retrieve sheet failure 2025-02-11 12:53:46 +00:00
msramalho
6fdd5f0e66 fix cases of single : vs :: in entrypoint 2025-02-11 12:53:12 +00:00
msramalho
e6594ad3dc merge result into cached results for context preservation 2025-02-11 12:52:42 +00:00
msramalho
7309cd32e7 fix: context to be updated on Metadata.merge 2025-02-11 12:51:17 +00:00
erinhmclark
d1d6cde008 Set mock timestamp without z format 2025-02-11 12:27:48 +00:00
erinhmclark
5e2e93382f Test fixes for 3.10 compliance. 2025-02-11 12:17:42 +00:00
erinhmclark
f97ec6a9e0 Fixed S3 module import 2025-02-11 11:58:28 +00:00
erinhmclark
89d9140d15 Fixed setup/ config_setup reference 2025-02-11 11:47:11 +00:00
erinhmclark
1792e02d1d skip authenticated tests in test_gdrive_storage.py 2025-02-11 11:34:36 +00:00
erinhmclark
18666ff027 skip authenticated tests in test_gsheet_feeder.py 2025-02-11 11:28:24 +00:00
erinhmclark
a69ac3e509 Fix file hash reference in S3 tests 2025-02-11 09:46:22 +00:00
Patrick Robertson
ed81dcdaf0 Remove dangling 'b = ' from config.py 2025-02-10 23:07:03 +00:00
erinhmclark
8d894066f2 Merge branch 'load_modules' into add_module_tests 
# Conflicts:
#	src/auto_archiver/modules/gsheet_feeder/gsheet_feeder.py
#	src/auto_archiver/utils/misc.py
 2025-02-10 19:00:05 +00:00
erinhmclark
3dae2337a1 remove cdn_url check before storage. 2025-02-10 18:56:46 +00:00
erinhmclark
e97ccf8a73 Separate setup() and module_setup(). 2025-02-10 18:07:47 +00:00
erinhmclark
2c3d1f591f Separate setup() and module_setup(). 2025-02-10 17:25:15 +00:00
msramalho
12f14cccc9 fixes gsheet feeder<->db connection via context. 2025-02-10 16:58:35 +00:00
msramalho
ab6cf52533 fixes bad hash initialization 2025-02-10 16:45:28 +00:00
erinhmclark
c4bb667cec Merge branch 'load_modules' into add_module_tests 
# Conflicts:
#	src/auto_archiver/modules/s3_storage/s3_storage.py
#	src/auto_archiver/utils/gsheet.py
#	src/auto_archiver/utils/misc.py
 2025-02-10 16:17:08 +00:00
erinhmclark
f311621e58 Small fixes. 
Add timestamp helper method.
 2025-02-10 15:57:42 +00:00
msramalho
15abf686b1 decouples s3_storage from hash_enricher 2025-02-10 15:48:54 +00:00
msramalho
8fb3dc754b fixing telethon extractor to use default entrypoint 2025-02-10 14:59:51 +00:00
msramalho
7c848046e8 adds better info about wrong/missing modules 2025-02-10 14:59:32 +00:00
Patrick Robertson
f3f6b92817 Implementation test cleanup 2025-02-10 12:43:21 +00:00
Patrick Robertson
74207d7821 Implementation tests for auto-archiver 2025-02-10 13:27:11 +01:00
Patrick Robertson
e9dd321dcd Fix setting cli_feeder as default feeder on clean install 2025-02-10 13:06:24 +01:00
Patrick Robertson
1fad37fd93 Remove blank file 2025-02-07 23:08:30 +01:00
Patrick Robertson
63aba6ad39 Fix sphinx-autoapi imports 2025-02-07 21:54:49 +01:00
erinhmclark
950624dd4b Fix S3 storage to media in whisper_enricher.py. 2025-02-07 20:26:00 +00:00
erinhmclark
2920cf685f Small fixes to whisper_enricher.py. 2025-02-07 12:35:40 +00:00
erinhmclark
e9ad1e1b85 Pass media to storage cdn_call 2025-02-06 22:01:55 +00:00
erinhmclark
266c7a14e6 Context related fixes, some more tests. 2025-02-06 16:53:00 +00:00
erinhmclark
67504a683e Merge branch 'load_modules' into add_module_tests 2025-02-06 10:13:37 +00:00
erinhmclark
5b0bad832f Updated test, test metadata 2025-02-06 10:11:56 +00:00
Patrick Robertson
a506f2a88f Clarify that an extractor's method can also return False if no valid data was found 2025-02-06 10:20:05 +01:00
Patrick Robertson
6ab8fd2ee4 Tidy up setting modules as Orchestrator attributes on startup. 
Don't override the values in config['steps'] – the config should be left as is
 2025-02-06 10:20:05 +01:00
erinhmclark
52542812dc Merge tests from version with context. 2025-02-05 16:42:58 +00:00
Patrick Robertson
48abb5e66b Remove dangling screenshot_enricher file. Moved to modules/screenshot_enricher 2025-02-04 18:16:03 +01:00
Patrick Robertson
91ca325fd5 Update yt-dlp to latest version + remove code no longer needed from bluesky dropin 2025-02-04 17:46:46 +01:00
Patrick Robertson
0633e17998 Close the facebook 'login' window if it's there - to allow for proper screenshots 2025-02-04 14:18:46 +01:00
Patrick Robertson
034197a81f Fix typos in csv feeder docs (in manifest) 2025-02-04 13:40:07 +01:00