Patrick Robertson
76e90dd23a
Small code tidy ups
2025-03-26 15:34:33 +04:00
Patrick Robertson
0073a08525
Update manifest dependencies to remove tsp_client et al.
2025-03-26 14:57:55 +04:00
Patrick Robertson
d6be1ff84f
Merge branch 'main' into timestamping_rewrite
2025-03-26 14:37:51 +04:00
Patrick Robertson
beb7f3893d
Add comments/notes to WACZ enricher about browser profiles
2025-03-24 17:39:47 +04:00
Patrick Robertson
3c4625d708
Further ruff tweaks
2025-03-24 16:39:59 +04:00
Patrick Robertson
31fa7380f5
Fix up unit tests + issue when working with self-signed certs
2025-03-24 16:00:40 +04:00
Patrick Robertson
396ec03bae
Tidy up unit tests further + make more non-download
2025-03-24 15:26:22 +04:00
Patrick Robertson
dfde6f1995
Merge main into timestamping_enricher
2025-03-24 15:09:29 +04:00
Patrick Robertson
c980500978
Actually restart AA after updating yt-dlp.
...
A simple 'importlib.reload()' doesn't take into account all imports
2025-03-24 14:33:59 +04:00
Patrick Robertson
aacb874b56
removeprefix for www. is required here
2025-03-21 12:23:45 +04:00
Patrick Robertson
4b5a8c0199
Add warning *inside* instagram_extractor that it's not actively maintained
2025-03-21 12:09:58 +04:00
Patrick Robertson
14c56f4916
Provide better logs for screenshot enricher when auth is/isn't supported (cookies only)
2025-03-21 12:05:47 +04:00
Patrick Robertson
5b131996c6
Add return type for auth_for_site
2025-03-21 11:55:12 +04:00
Patrick Robertson
168dfb6254
Unit tests for url utils
2025-03-21 11:53:47 +04:00
Patrick Robertson
42e16aebd6
Merge pull request #255 from bellingcat/autogenerate_services_account
...
Script to auto-generate a service account
2025-03-20 18:00:45 +00:00
Patrick Robertson
e6c5705f70
Merge pull request #261 from bellingcat/wacz_separate_profile
...
Wacz minor adjustments
2025-03-20 15:51:56 +00:00
Erin Clark
613ba0c05d
Merge pull request #262 from bellingcat/generic_extractor_args
...
Add flexible extractor_args to generic_extractor.py
This allows users to pass any of the options listed [here](https://github.com/yt-dlp/yt-dlp/blob/master/README.md#extractor-arguments ) to yt-dlp extractor_args.
example usage:
```
generic_extractor:
facebook_cookie:
...
extractor_args:
youtube:
player_client: web,tv
generic:
is_live: true
```
2025-03-20 15:38:20 +00:00
Patrick Robertson
0a5ba3385e
Fix small bug in twitter dropin
...
- previously the 'content' was being set to a json dump of the tweet, it should be set to full_text
2025-03-20 18:55:22 +04:00
Patrick Robertson
034857075d
Merge branch 'main' into wrong_steps
2025-03-20 18:44:19 +04:00
Patrick Robertson
5e5e1c43a1
When loading modules, check they have been added to the right 'step' in the config
...
Fixes an issue seen on discord where a user accidentally set up metadata_enricher under 'extractors'
2025-03-20 18:09:26 +04:00
Patrick Robertson
f22af5e123
Tweak WACZ enricher docs + add comment on WACZ_ENABLE_DOCKER
2025-03-20 16:48:30 +04:00
erinhmclark
2921061fde
Add flexible extractor_args to generic_extractor.py
2025-03-19 19:19:28 +00:00
Patrick Robertson
e531906d73
Create an independent profile file for each wacz_extractor_enricher instance
2025-03-19 18:08:24 +04:00
Patrick Robertson
244341d22c
Skip check for 'docker' bin dependency if already running in docker
2025-03-19 18:08:04 +04:00
Patrick Robertson
488675056b
Download generate_google_services.sh script from GH - it's not packaged with the app
2025-03-19 15:52:39 +04:00
erinhmclark
a577228465
Update generic_extractor.py for general/ youtube extraction.
2025-03-18 21:10:06 +00:00
Miguel Sozinho Ramalho
f6863b8eb2
Update src/auto_archiver/modules/gsheet_feeder_db/__manifest__.py
2025-03-18 14:10:47 +00:00
Patrick Robertson
d03ecdb037
Standardise parse dates to get_datetime_from_str
2025-03-18 10:22:58 +00:00
Patrick Robertson
89e387030d
Tests for suitable URLs for tikwm
2025-03-18 10:04:03 +00:00
Patrick Robertson
8ec053ed1b
Refactor the dropin 'is_suitable' method + fix tikwm implementation
...
Makes it easier to maintain/understand.
2025-03-18 09:14:14 +00:00
Patrick Robertson
29db537fab
Docs on using the script to auto-generate service accounts
2025-03-17 18:11:18 +00:00
Patrick Robertson
3d4056ef70
Merge pull request #223 from bellingcat/facebook_extractor
...
Create facebook dropin - working for images + text.
2025-03-17 12:45:05 +00:00
Patrick Robertson
0765640bff
Fix up tiktok dropin for slightly modified generic_extractor format
2025-03-17 10:31:22 +00:00
Patrick Robertson
06b1f4c0ca
Fix lingering merge conflict issues
2025-03-17 10:12:55 +00:00
Patrick Robertson
59b910ec30
Merge main
2025-03-17 10:05:11 +00:00
Patrick Robertson
7e360240bf
Copy ytdlp code into AA project - seems like ytdlp won't be merged anytime soon
2025-03-17 09:57:05 +00:00
Patrick Robertson
7badf89c28
Create the 'secrets' folder if it doesn't exist on first run
...
Easier setup for users
2025-03-17 09:40:46 +00:00
Patrick Robertson
d59530c8e7
Fix if logic bug
2025-03-17 09:40:27 +00:00
Patrick Robertson
0ec5451f66
Nicer error log when no URLs provided for CLI feeder - don't need the stacktrace
2025-03-17 09:34:33 +00:00
Patrick Robertson
99e9ac2465
Fix 'Syntax Error' warning in python3.12+
2025-03-17 09:29:51 +00:00
Patrick Robertson
42162c5e3f
Various docs improvements based on Friday Office Hours discussion
2025-03-17 09:23:43 +00:00
Patrick Robertson
17463de937
Merge pull request #247 from bellingcat/opentimestamps
...
Opentimestamps Module
2025-03-14 13:41:46 +00:00
Patrick Robertson
a8e5585e6c
github format
2025-03-14 12:52:01 +00:00
Patrick Robertson
19715c8ec2
Merge branch 'main' into webdriver-cookies
2025-03-14 12:44:48 +00:00
Patrick Robertson
17ae75fb95
Ruff fixes
2025-03-14 12:38:12 +00:00
Patrick Robertson
b8da7607e8
Merge branch 'main' into opentimestamps
2025-03-14 12:36:03 +00:00
erinhmclark
72f48f0147
Fix merge conflicts.
2025-03-14 12:11:24 +00:00
erinhmclark
846474a4e2
Merge branch 'main' into linting_etc
2025-03-14 10:50:13 +00:00
Patrick Robertson
f504d2e304
Merge branch 'main' into webdriver-cookies
2025-03-14 09:37:12 +00:00
msramalho
4d67dce4c8
minor log fix
2025-03-13 19:24:05 +00:00
