msramalho
3cf51dd874
adds tracker remove feature and tests
2025-06-11 11:56:42 +01:00
msramalho
ef0e909a72
extractor to auto detect best quality
2025-06-10 16:29:35 +01:00
msramalho
18cc05a2fe
allows auth_for_site to receive do.main directly
2025-06-08 14:16:12 +01:00
msramalho
e2e6490b49
minimal changes
2025-06-07 18:15:21 +01:00
erinhmclark
68992025b0
Update version comparison.
2025-03-28 14:29:44 +00:00
Patrick Robertson
25f1f5dc93
Merge pull request #279 from bellingcat/telethon_tweaks
...
Fix calling extractor.cleanup (fixes telethon issue) + tidy up telethon extractor session file naming
2025-03-28 14:13:26 +04:00
Patrick Robertson
a448e2532c
Code tweak for clarity
2025-03-27 15:20:52 +04:00
Patrick Robertson
95ea9fb231
Telethon unit tests + tidyup
2025-03-26 22:53:27 +04:00
Patrick Robertson
17d2d14680
Fix running 'cleanup' method on extractors that fail to start
2025-03-26 22:52:52 +04:00
Patrick Robertson
76e90dd23a
Small code tidy ups
2025-03-26 15:34:33 +04:00
Patrick Robertson
5b131996c6
Add return type for auth_for_site
2025-03-21 11:55:12 +04:00
Patrick Robertson
e6c5705f70
Merge pull request #261 from bellingcat/wacz_separate_profile
...
Wacz minor adjustments
2025-03-20 15:51:56 +00:00
Patrick Robertson
5e5e1c43a1
When loading modules, check they have been added to the right 'step' in the config
...
Fixes an issue seen on discord where a user accidentally set up metadata_enricher under 'extractors'
2025-03-20 18:09:26 +04:00
Patrick Robertson
f22af5e123
Tweak WACZ enricher docs + add comment on WACZ_ENABLE_DOCKER
2025-03-20 16:48:30 +04:00
Patrick Robertson
244341d22c
Skip check for 'docker' bin dependency if already running in docker
2025-03-19 18:08:04 +04:00
Patrick Robertson
7badf89c28
Create the 'secrets' folder if it doesn't exist on first run
...
Easier setup for users
2025-03-17 09:40:46 +00:00
Patrick Robertson
d59530c8e7
Fix if logic bug
2025-03-17 09:40:27 +00:00
Patrick Robertson
0ec5451f66
Nicer error log when no URLs provided for CLI feeder - don't need the stacktrace
2025-03-17 09:34:33 +00:00
Patrick Robertson
b8da7607e8
Merge branch 'main' into opentimestamps
2025-03-14 12:36:03 +00:00
erinhmclark
8673bc5979
Fix unused imports and include rule.
2025-03-13 13:55:31 +00:00
erinhmclark
e76551ba22
Add documentation, pre-commit hook, more make commands and
2025-03-13 13:21:32 +00:00
erinhmclark
6e52a534e7
More fixes from Bugbear suggestions
2025-03-12 16:07:05 +00:00
Patrick Robertson
1423c10363
Finish off timestamping module
2025-03-12 10:24:57 +00:00
erinhmclark
8ca7698fa0
Move Makefile and fix import error with unused import.
2025-03-11 19:58:02 +00:00
erinhmclark
81aa343f21
Merge main.
2025-03-11 10:45:07 +00:00
erinhmclark
441f341139
Merge branch 'main' into linting_etc
...
# Conflicts:
# src/auto_archiver/core/consts.py
# src/auto_archiver/core/orchestrator.py
# src/auto_archiver/core/storage.py
# src/auto_archiver/modules/local_storage/local_storage.py
# src/auto_archiver/modules/s3_storage/s3_storage.py
# tests/storages/test_S3_storage.py
# tests/storages/test_local_storage.py
# tests/storages/test_storage_base.py
2025-03-11 10:39:47 +00:00
erinhmclark
e7fa88f1c7
Implementing ruff suggestions.
2025-03-10 21:45:30 +00:00
erinhmclark
ca44a40b88
Ruff fix on src.
2025-03-10 19:03:45 +00:00
erinhmclark
85abe1837a
Ruff format with defaults.
2025-03-10 18:44:54 +00:00
Patrick Robertson
a9c3477289
Improve docs on the path_generator and filename_generator config options
2025-03-10 16:43:14 +00:00
Patrick Robertson
770f4c8a3d
Refactoring of storage code:
...
1. Fix some bugs in local_storage
2. Refactor unit tests to not set Media.key explicitly (unless it's well-known beforehand, which it isn't)
3. Limit length of URL for 'url' type path_generator
4. Throw an error if 'save_to' of local storage is too long
5. A few other tidyups
2025-03-10 16:39:48 +00:00
Patrick Robertson
e89a8da3b4
Unit tests for storage types + fix storage too long issues for local storage
2025-03-10 11:30:15 +00:00
Patrick Robertson
3fac353407
Merge pull request #217 from bellingcat/settings_page
...
Settings page user interface
2025-03-07 16:10:50 +00:00
Erin Clark
85a75755e2
Merge pull request #236 from bellingcat/cleanup_fixes
...
Cleanup fixes
2025-03-07 15:37:05 +00:00
Patrick Robertson
333201acec
Merge branch 'main' into settings_page
2025-03-07 15:17:42 +00:00
Patrick Robertson
027985024b
Merge pull request #234 from bellingcat/update_suggestions
...
Auto Updates
2025-03-07 15:12:03 +00:00
erinhmclark
bdd35408ce
Fix ref before assignment in orchestrator.py
2025-03-07 11:23:51 +00:00
Patrick Robertson
478f0b2171
Tidy-ups to auto-updating code
2025-03-07 09:59:18 +00:00
Patrick Robertson
e6a578e60e
Check for auto-archiver updates and present warning if there's a newer version available
2025-03-04 16:51:17 +00:00
Patrick Robertson
0b5a0fcb32
Better error logs if users have XXXX_archiver modules enabled in config
2025-03-03 19:57:09 +00:00
Patrick Robertson
1fe023cd70
Throw a nicer error if a user has an orchestration.yaml file in the old format (feeder: / archivers: / formatter: )
2025-03-03 19:51:55 +00:00
Patrick Robertson
dea0a49600
Download correct gecko-driver for the platform + fix setting executable path when running in Docker
...
Fixes #232
2025-03-03 15:41:44 +00:00
Patrick Robertson
bb961b131c
Turn cli_feeder *back* into a module, it's better like this for settings etc, documentation etc.
2025-02-26 15:41:33 +00:00
erinhmclark
8124bb831d
Merge branch 'main' into small_issues
...
# Conflicts:
# src/auto_archiver/core/base_module.py
# src/auto_archiver/utils/misc.py
2025-02-26 13:19:49 +00:00
erinhmclark
1df5129268
Small typos.
2025-02-25 14:08:38 +00:00
Patrick Robertson
ca1ed418aa
Throw an error for invalid __manifest__ syntax + fix: allow default values of False/None
2025-02-24 21:46:24 +00:00
Patrick Robertson
091a19e25c
Further docs improvements/tidy ups
2025-02-21 16:52:30 +00:00
Patrick Robertson
9661e90a05
Allow disabling logging in auto_archiver with logging: enabled: false
2025-02-20 15:45:32 +00:00
Patrick Robertson
0bec71d203
Finish how to on authentication
2025-02-20 15:33:50 +00:00
Patrick Robertson
eda359a1ef
Fix json loader - it should go in 'validators' not 'utils'
...
Fixes #214
2025-02-20 13:10:39 +00:00
