erinhmclark
c7c24fbaf2
Update style_guide.md to clarify pre-commit setup, add Docker commands to Makefile and merge ruff actions.
2025-03-13 20:26:29 +00:00
erinhmclark
0bef78b0b4
Remove autouse property of mock_sleep.
2025-03-13 15:23:35 +00:00
erinhmclark
16012df30b
Revert exception check in test.
2025-03-13 13:57:04 +00:00
erinhmclark
8673bc5979
Fix unused imports and include rule.
2025-03-13 13:55:31 +00:00
erinhmclark
e76551ba22
Add documentation, pre-commit hook, more make commands and
2025-03-13 13:21:32 +00:00
erinhmclark
6e52a534e7
More fixes from Bugbear suggestions
2025-03-12 16:07:05 +00:00
erinhmclark
753c3c6214
Linting tests
2025-03-12 14:27:45 +00:00
erinhmclark
79f576be1d
Run fix on tests.
2025-03-12 10:38:16 +00:00
erinhmclark
94aeee8313
Move Makefile to the root of the project and add commands for tests, linting and running docker.
2025-03-12 10:37:30 +00:00
erinhmclark
abc90b19d5
Update pyproject.toml
2025-03-12 10:35:56 +00:00
erinhmclark
8ca7698fa0
Move Makefile and fix import error with unused import.
2025-03-11 19:58:02 +00:00
erinhmclark
28c5396b74
Move ruff to dev dependencies.
2025-03-11 17:25:24 +00:00
erinhmclark
81aa343f21
Merge main.
2025-03-11 10:45:07 +00:00
erinhmclark
441f341139
Merge branch 'main' into linting_etc
...
# Conflicts:
# src/auto_archiver/core/consts.py
# src/auto_archiver/core/orchestrator.py
# src/auto_archiver/core/storage.py
# src/auto_archiver/modules/local_storage/local_storage.py
# src/auto_archiver/modules/s3_storage/s3_storage.py
# tests/storages/test_S3_storage.py
# tests/storages/test_local_storage.py
# tests/storages/test_storage_base.py
2025-03-11 10:39:47 +00:00
Patrick Robertson
e2442b2f6b
Merge pull request #243 from bellingcat/fix-long-path-names
...
Unit tests for storage types + fix storage too long issues for local storage
2025-03-11 10:05:09 +00:00
erinhmclark
e7fa88f1c7
Implementing ruff suggestions.
2025-03-10 21:45:30 +00:00
erinhmclark
ca44a40b88
Ruff fix on src.
2025-03-10 19:03:45 +00:00
erinhmclark
85abe1837a
Ruff format with defaults.
2025-03-10 18:44:54 +00:00
Miguel Sozinho Ramalho
3fcec57492
minor string fix
2025-03-10 17:17:59 +00:00
Patrick Robertson
2b91dc9514
Fix up unit tests
2025-03-10 16:51:16 +00:00
Patrick Robertson
a9c3477289
Improve docs on the path_generator and filename_generator config options
2025-03-10 16:43:14 +00:00
Patrick Robertson
770f4c8a3d
Refactoring of storage code:
...
1. Fix some bugs in local_storage
2. Refactor unit tests to not set Media.key explicitly (unless it's well-known beforehand, which it isn't)
3. Limit length of URL for 'url' type path_generator
4. Throw an error if 'save_to' of local storage is too long
5. A few other tidyups
2025-03-10 16:39:48 +00:00
erinhmclark
cbb0414e5f
Switch to ruff
2025-03-10 16:05:23 +00:00
erinhmclark
f4f2424eb5
Add black and flake8
2025-03-10 13:15:11 +00:00
Miguel Sozinho Ramalho
58bd38e292
Adds new extractor for tiktok via unofficial API ( #237 )
...
* minor update to defaults in api_db
* readme typo
* adds and tests new tikwm tiktok downloader
* addresses PR comments
2025-03-10 11:56:45 +00:00
Patrick Robertson
e89a8da3b4
Unit tests for storage types + fix storage too long issues for local storage
2025-03-10 11:30:15 +00:00
Erin Clark
ce46a8a7ac
Merge pull request #240 from bellingcat/update_release
...
Update project version to: 0.13.5.
Update the release process docs and the latest version in pyproject.toml
2025-03-07 18:21:17 +00:00
erinhmclark
7e10040bbd
Update the release description to tag on release
2025-03-07 18:04:51 +00:00
erinhmclark
b386ae6287
Add poetry.lock and pyproject.toml paths to trigger tests.
2025-03-07 18:01:08 +00:00
erinhmclark
1a2d9de819
Update the release process docs and the latest version in pyproject.toml
2025-03-07 17:33:32 +00:00
Erin Clark
4c21795d5f
Merge pull request #226 from bellingcat/merge_modules
...
Merge modules with multi-functionality:
- gsheet_feeder and gsheet_db are now one module, gsheet_feeder_db
- atlos_feeder, atlos_db and atlos_storage are now one module, atlos_feeder_db_storage.
This pull request also add documentation and updates references.
2025-03-07 16:47:30 +00:00
Patrick Robertson
e519ba2433
Add 'reject all' cookie button
2025-03-07 16:40:34 +00:00
Patrick Robertson
a8fcd0b9a0
Further info in how to for the new config format
2025-03-07 16:37:58 +00:00
Patrick Robertson
09e09e9ab9
Document module renames in 'upgrading from 0.12' how to
2025-03-07 16:28:17 +00:00
Patrick Robertson
be513e95aa
Merge branch 'main' into merge_modules
2025-03-07 16:19:51 +00:00
Patrick Robertson
3fac353407
Merge pull request #217 from bellingcat/settings_page
...
Settings page user interface
2025-03-07 16:10:50 +00:00
Erin Clark
928c6f88a9
Merge pull request #239 from bellingcat/fix_docker_registry_ref
...
Fix docker registry reference
2025-03-07 15:57:29 +00:00
erinhmclark
8fcec692b7
Add comments to highlight different steps of atlos_feeder_db_storage.py
2025-03-07 15:42:20 +00:00
erinhmclark
65109e377f
Remove raising exception in atlos_feeder_db_storage.py
2025-03-07 15:39:15 +00:00
Erin Clark
85a75755e2
Merge pull request #236 from bellingcat/cleanup_fixes
...
Cleanup fixes
2025-03-07 15:37:05 +00:00
erinhmclark
4949e9bcd2
Update docker-publish.yaml to use docker registry.
2025-03-07 15:34:56 +00:00
erinhmclark
3877b538be
Update docker-publish.yaml to use docker registry.
2025-03-07 15:31:25 +00:00
Erin Clark
2e0e989793
Merge pull request #238 from bellingcat/cache_docker_to_registry
...
Add cache-from and cache-to to docker-publish.yaml, using Dockerhub Registry as the cache.
2025-03-07 15:23:50 +00:00
Patrick Robertson
333201acec
Merge branch 'main' into settings_page
2025-03-07 15:17:42 +00:00
erinhmclark
87ab98c270
Update docker/build-push version
2025-03-07 15:14:20 +00:00
Patrick Robertson
027985024b
Merge pull request #234 from bellingcat/update_suggestions
...
Auto Updates
2025-03-07 15:12:03 +00:00
erinhmclark
7bbf0da0d1
Add cache-from and cache-to to docker-publish.yaml.
2025-03-07 15:09:10 +00:00
Patrick Robertson
48b29d43f7
Merge pull request #233 from bellingcat/docker-webdriver-aarch64
...
Docker webdriver aarch64
2025-03-07 15:04:45 +00:00
Erin Clark
8ae3d9c031
Merge pull request #235 from bellingcat/instagram_extractor_bugfix
...
Instagram extractor bugfix:
- Fix typo from config changes
- Add warning message to documentation to alert to it not being maintained.
2025-03-07 15:02:05 +00:00
Patrick Robertson
158e6be0b1
Don't force enable a module under all types - allows modules to be set as only feeder/only extractor etc.
2025-03-07 14:59:11 +00:00
