bellingcat/auto-archiver
1
0
Fork 0
mirror of https://github.com/bellingcat/auto-archiver.git synced 2026-06-08 03:18:28 +03:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • a9c3477289 Improve docs on the path_generator and filename_generator config options Patrick Robertson 2025-03-10 16:43:14 +00:00
  • 770f4c8a3d Refactoring of storage code: Patrick Robertson 2025-03-10 16:39:31 +00:00
  • cbb0414e5f Switch to ruff erinhmclark 2025-03-10 16:05:23 +00:00
  • f4f2424eb5 Add black and flake8 erinhmclark 2025-03-10 13:15:11 +00:00
  • 58bd38e292 Adds new extractor for tiktok via unofficial API (#237) Miguel Sozinho Ramalho 2025-03-10 17:56:45 +06:00
  • e89a8da3b4 Unit tests for storage types + fix storage too long issues for local storage Patrick Robertson 2025-03-10 11:30:15 +00:00
  • ce46a8a7ac Merge pull request #240 from bellingcat/update_release v0.13.5 Erin Clark 2025-03-07 18:21:17 +00:00
  • 7e10040bbd Update the release description to tag on release erinhmclark 2025-03-07 18:04:51 +00:00
  • b386ae6287 Add poetry.lock and pyproject.toml paths to trigger tests. erinhmclark 2025-03-07 18:01:08 +00:00
  • 1a2d9de819 Update the release process docs and the latest version in pyproject.toml erinhmclark 2025-03-07 17:33:32 +00:00
  • 76bb1496c8 Merge branch 'main' into feat/yt-dlp-pots erinhmclark 2025-03-07 16:54:01 +00:00
  • 4c21795d5f Merge pull request #226 from bellingcat/merge_modules Erin Clark 2025-03-07 16:47:30 +00:00
  • e519ba2433 Add 'reject all' cookie button Patrick Robertson 2025-03-07 16:40:34 +00:00
  • a8fcd0b9a0 Further info in how to for the new config format Patrick Robertson 2025-03-07 16:37:58 +00:00
  • 09e09e9ab9 Document module renames in 'upgrading from 0.12' how to Patrick Robertson 2025-03-07 16:28:17 +00:00
  • be513e95aa Merge branch 'main' into merge_modules Patrick Robertson 2025-03-07 16:19:51 +00:00
  • 3fac353407 Merge pull request #217 from bellingcat/settings_page Patrick Robertson 2025-03-07 16:10:50 +00:00
  • 928c6f88a9 Merge pull request #239 from bellingcat/fix_docker_registry_ref Erin Clark 2025-03-07 15:57:29 +00:00
  • 8fcec692b7 Add comments to highlight different steps of atlos_feeder_db_storage.py erinhmclark 2025-03-07 15:42:20 +00:00
  • 65109e377f Remove raising exception in atlos_feeder_db_storage.py erinhmclark 2025-03-07 15:39:15 +00:00
  • 85a75755e2 Merge pull request #236 from bellingcat/cleanup_fixes Erin Clark 2025-03-07 15:37:05 +00:00
  • 4949e9bcd2 Update docker-publish.yaml to use docker registry. erinhmclark 2025-03-07 15:34:56 +00:00
  • 3877b538be Update docker-publish.yaml to use docker registry. erinhmclark 2025-03-07 15:31:25 +00:00
  • 2e0e989793 Merge pull request #238 from bellingcat/cache_docker_to_registry Erin Clark 2025-03-07 15:23:50 +00:00
  • 333201acec Merge branch 'main' into settings_page Patrick Robertson 2025-03-07 15:17:42 +00:00
  • 87ab98c270 Update docker/build-push version erinhmclark 2025-03-07 15:14:20 +00:00
  • 027985024b Merge pull request #234 from bellingcat/update_suggestions Patrick Robertson 2025-03-07 15:12:03 +00:00
  • 7bbf0da0d1 Add cache-from and cache-to to docker-publish.yaml. erinhmclark 2025-03-07 15:09:10 +00:00
  • 48b29d43f7 Merge pull request #233 from bellingcat/docker-webdriver-aarch64 Patrick Robertson 2025-03-07 15:04:45 +00:00
  • 8ae3d9c031 Merge pull request #235 from bellingcat/instagram_extractor_bugfix Erin Clark 2025-03-07 15:02:05 +00:00
  • 158e6be0b1 Don't force enable a module under all types - allows modules to be set as only feeder/only extractor etc. Patrick Robertson 2025-03-07 14:59:11 +00:00
  • 4df03255a4 Fix typo in __manifest__.py erinhmclark 2025-03-07 14:56:35 +00:00
  • 503ba3d1c1 Add note on auto updates to readme Patrick Robertson 2025-03-07 14:46:50 +00:00
  • 40e5fe7a7e Update __manifest__.py for merged Atlos module. erinhmclark 2025-03-07 13:46:09 +00:00
  • f6f397700e Fix typo + use hidden password fields for secret info Patrick Robertson 2025-03-07 12:41:09 +00:00
  • 89d2a8bb54 Update the __manifest__.py of the Instagram Extractor. erinhmclark 2025-03-07 12:34:19 +00:00
  • e72b3e14ba Change default height of screenshots to attempt to capture more information Patrick Robertson 2025-03-07 12:08:29 +00:00
  • dba44b1ac1 Use WebDriverWait when waiting for elements in screenshot enricher Patrick Robertson 2025-03-07 12:07:54 +00:00
  • e756f1504f Remove geckodriver .tar file Patrick Robertson 2025-03-07 11:52:14 +00:00
  • 2c5e138263 Add a note on disabling the auto-update for yt-dlp Patrick Robertson 2025-03-07 11:44:24 +00:00
  • fb56aac15e Catch edge case to ensure iterator is reached in instagram_tbot_extractor.py erinhmclark 2025-03-07 11:24:25 +00:00
  • bdd35408ce Fix ref before assignment in orchestrator.py erinhmclark 2025-03-07 11:23:51 +00:00
  • 478f0b2171 Tidy-ups to auto-updating code Patrick Robertson 2025-03-07 09:59:18 +00:00
  • 32329c6b2c Update Google Sheet how to docs. erinhmclark 2025-03-07 00:11:43 +00:00
  • fa1e65f54c Fix instagram_extractor.py typo, add warning to docs, and add basic regex test. erinhmclark 2025-03-06 16:25:38 +00:00
  • b9c2f98f46 Update Atlos tests erinhmclark 2025-03-05 21:24:38 +00:00
  • 0f911543cd Atlos refactor erinhmclark 2025-03-05 13:49:11 +00:00
  • 6cb7afefdc Initial Atlos merge erinhmclark 2025-03-05 10:24:54 +00:00
  • 358884c5d1 Fix unit tests for yt-dlp update Patrick Robertson 2025-03-04 17:04:23 +00:00
  • be09aa927d Make 'STARTED' command INFO not warning Patrick Robertson 2025-03-04 16:51:09 +00:00
  • e6a578e60e Check for auto-archiver updates and present warning if there's a newer version available Patrick Robertson 2025-03-04 16:44:15 +00:00
  • 0eb112431b Auto-update yt-dlp based on generic_extractor.ytdlp_update_interval (default=5 days) Patrick Robertson 2025-03-04 16:43:46 +00:00
  • 22932645aa Merge remote-tracking branch 'origin/merge_modules' into merge_modules erinhmclark 2025-03-04 14:07:12 +00:00
  • d1c8d4ba0e Initial merge of Atlos Feeder and DB erinhmclark 2025-02-27 11:18:10 +00:00
  • d775e4612e Update gsheet_feeder references in tests. erinhmclark 2025-02-25 21:38:41 +00:00
  • 077b56c150 Merge GSheet Feeder and Database. erinhmclark 2025-02-25 21:32:32 +00:00
  • 7e4b44883b Add temp options for testing erinhmclark 2025-03-04 14:03:39 +00:00
  • f54d6519a8 Fix sorting of steps in the outputted file Patrick Robertson 2025-03-04 11:51:26 +00:00
  • 07ee773a54 Better drag & drop + keep comments in file Patrick Robertson 2025-03-04 10:54:16 +00:00
  • 77b517cfc1 Merge remote-tracking branch 'origin/feat/yt-dlp-pots' into feat/yt-dlp-pots erinhmclark 2025-03-03 22:02:14 +00:00
  • 2c1753e14b Added Bgutils PO token provider. erinhmclark 2025-02-25 10:49:57 +00:00
  • dd07b0b830 Allow flexible extractor_args in generic_extractor.py. erinhmclark 2025-02-24 11:40:44 +00:00
  • 0eae2bee6a Add yt-dlp-get-pot and yt-dlp-getpot-wpc requirements. erinhmclark 2025-02-24 11:40:39 +00:00
  • a705a78632 Fix instagram_extractor.py typo in config value. erinhmclark 2025-03-03 21:06:09 +00:00
  • dcaf7639be Rename 'upgrading...' page to 'upgrading from...' because it's also valid for 0.13+ versions Patrick Robertson 2025-03-03 19:58:47 +00:00
  • 0b5a0fcb32 Better error logs if users have XXXX_archiver modules enabled in config Patrick Robertson 2025-03-03 19:57:09 +00:00
  • 1fe023cd70 Throw a nicer error if a user has an orchestration.yaml file in the old format (feeder: / archivers: / formatter: ) Patrick Robertson 2025-03-03 19:51:55 +00:00
  • a47e18ef9a Bump gecko driver to 0.36.0 Patrick Robertson 2025-03-03 16:00:11 +00:00
  • 0dfab2d1bc Add some code to attempt to click the cookies banners on various websites Patrick Robertson 2025-03-03 15:55:04 +00:00
  • dea0a49600 Download correct gecko-driver for the platform + fix setting executable path when running in Docker Patrick Robertson 2025-03-03 15:41:44 +00:00
  • 011ded2bde Merge pull request #225 from bellingcat/small_issues Erin Clark 2025-03-03 13:06:27 +00:00
  • a88a37d0a5 Hook in to RTD theme to set react theme Patrick Robertson 2025-03-03 11:56:23 +00:00
  • a0869bb3b2 Fixed up timestamp verifying - waiting on issue with rfc-client to be fixed Patrick Robertson 2025-03-03 10:28:30 +00:00
  • 9845804277 Fix up TODO plus add comments on integration into RTD page Patrick Robertson 2025-03-03 09:18:19 +00:00
  • cc14e5cb9f Remove extra html/head tag from page - now it's embedded in RTD Patrick Robertson 2025-03-03 09:06:40 +00:00
  • 6ba79049d9 Capitalize help text Patrick Robertson 2025-02-27 22:16:33 +00:00
  • 7620a671d1 Overwrite settings_base file Patrick Robertson 2025-02-27 22:02:44 +00:00
  • 54a2a19dd7 Also build auto-archiver Patrick Robertson 2025-02-27 21:42:21 +00:00
  • 3eb4ab41b8 Also generate the schema on each run Patrick Robertson 2025-02-27 21:37:37 +00:00
  • 65a9885d86 A few more manifest types Patrick Robertson 2025-02-27 21:33:04 +00:00
  • 4ee1e75aa2 Fix readthedocs config file Patrick Robertson 2025-02-27 21:24:34 +00:00
  • 1141c00e9a Remove unused files, set up for RTD Patrick Robertson 2025-02-27 21:23:38 +00:00
  • 15da907e81 Add a bit of typescripting Patrick Robertson 2025-02-27 15:58:30 +00:00
  • 2ec44f4170 Documentation on building the settings page Patrick Robertson 2025-02-27 15:42:37 +00:00
  • 1e92c03b1d Tweaks to settings page + more declarations in manifests Patrick Robertson 2025-02-27 15:21:11 +00:00
  • efe9fdf915 Tidy ups to config editor page Patrick Robertson 2025-02-27 13:02:50 +00:00
  • 4280791f07 Fix mocking in test_wayback_enricher.py. erinhmclark 2025-02-27 11:25:58 +00:00
  • f58f110436 Check at least 1 URL provided for new cli_feeder module rewrite Patrick Robertson 2025-02-26 17:59:13 +00:00
  • 70d89c71ce Fully-working settings page editor Patrick Robertson 2025-02-26 17:02:49 +00:00
  • bb961b131c Turn cli_feeder *back* into a module, it's better like this for settings etc, documentation etc. Patrick Robertson 2025-02-26 15:39:51 +00:00
  • e467fc90c2 Merge branch 'main' into settings_page Patrick Robertson 2025-02-26 15:32:07 +00:00
  • 8124bb831d Merge branch 'main' into small_issues erinhmclark 2025-02-26 13:19:49 +00:00
  • b2e654aef9 Remove context manager from test_pdq_hash_enricher.py erinhmclark 2025-02-26 12:57:33 +00:00
  • 9157846930 Add docstrings to explain date formats. erinhmclark 2025-02-26 10:01:52 +00:00
  • 600f43e790 Set up structure for react Patrick Robertson 2025-02-26 09:34:44 +00:00
  • afc117a229 Get downloading certs working Patrick Robertson 2025-02-26 09:33:56 +00:00
  • 696aafb52d Update gsheet_feeder references in tests. erinhmclark 2025-02-25 21:38:41 +00:00
  • 75380b0716 Merge GSheet Feeder and Database. erinhmclark 2025-02-25 21:32:32 +00:00
  • 35b5ab2eb1 Update poetry.lock erinhmclark 2025-02-25 20:17:48 +00:00
  • 83a08dd215 Update date parsing to use dateutil.parser in misc.py erinhmclark 2025-02-25 20:17:31 +00:00