Erin Clark
613ba0c05d
Merge pull request #262 from bellingcat/generic_extractor_args
...
Add flexible extractor_args to generic_extractor.py
This allows users to pass any of the options listed [here](https://github.com/yt-dlp/yt-dlp/blob/master/README.md#extractor-arguments ) to yt-dlp extractor_args.
example usage:
```
generic_extractor:
facebook_cookie:
...
extractor_args:
youtube:
player_client: web,tv
generic:
is_live: true
```
2025-03-20 15:38:20 +00:00
Patrick Robertson
b997bbea2b
Merge pull request #263 from bellingcat/wrong_steps
...
When loading modules, check they have been added to the right 'step' in the config
2025-03-20 15:31:38 +00:00
erinhmclark
54f53886ef
Update tests for default config values
2025-03-20 14:57:26 +00:00
Patrick Robertson
0a5ba3385e
Fix small bug in twitter dropin
...
- previously the 'content' was being set to a json dump of the tweet, it should be set to full_text
2025-03-20 18:55:22 +04:00
Patrick Robertson
034857075d
Merge branch 'main' into wrong_steps
2025-03-20 18:44:19 +04:00
Patrick Robertson
6700250891
Add a test for checking module type on setup
2025-03-20 18:18:53 +04:00
Patrick Robertson
5e5e1c43a1
When loading modules, check they have been added to the right 'step' in the config
...
Fixes an issue seen on discord where a user accidentally set up metadata_enricher under 'extractors'
2025-03-20 18:09:26 +04:00
erinhmclark
2921061fde
Add flexible extractor_args to generic_extractor.py
2025-03-19 19:19:28 +00:00
Erin Clark
90932a7bc8
Merge pull request #259 from bellingcat/fix_youtube_generic
...
Small fix for generic_extractor.py for general/ youtube extraction.
2025-03-19 11:52:56 +00:00
erinhmclark
a577228465
Update generic_extractor.py for general/ youtube extraction.
2025-03-18 21:10:06 +00:00
Patrick Robertson
7d972ee9b8
Merge pull request #258 from bellingcat/version_bump
...
Version bump
2025-03-18 12:18:09 +00:00
Patrick Robertson
b64826dc16
Merge pull request #257 from bellingcat/standardise_parsedates
...
Standardise parse dates to get_datetime_from_str
2025-03-18 12:17:51 +00:00
Patrick Robertson
23e74803ee
Version bump
2025-03-18 10:52:23 +00:00
Patrick Robertson
d03ecdb037
Standardise parse dates to get_datetime_from_str
2025-03-18 10:22:58 +00:00
Patrick Robertson
a5ebbf4726
Merge pull request #256 from bellingcat/dropin_cleanup
...
Refactor the dropin 'is_suitable' method + fix for tikwm
2025-03-18 10:08:24 +00:00
Patrick Robertson
89e387030d
Tests for suitable URLs for tikwm
2025-03-18 10:04:03 +00:00
Patrick Robertson
8ec053ed1b
Refactor the dropin 'is_suitable' method + fix tikwm implementation
...
Makes it easier to maintain/understand.
2025-03-18 09:14:14 +00:00
Patrick Robertson
3ea02c115e
Merge pull request #254 from bellingcat/rtd_docs
...
Add info on building RTD versions + automated building of tagged versions
2025-03-17 13:01:20 +00:00
Patrick Robertson
ab03e48708
Add info on building RTD versions + automated building of tagged versions
2025-03-17 12:52:04 +00:00
Patrick Robertson
3d4056ef70
Merge pull request #223 from bellingcat/facebook_extractor
...
Create facebook dropin - working for images + text.
2025-03-17 12:45:05 +00:00
Patrick Robertson
51041bf91e
Merge pull request #253 from bellingcat/settings_page
...
Update material version, minify code
2025-03-17 11:59:37 +00:00
Patrick Robertson
f56cd6891b
Finish incomplete sentence
2025-03-17 10:33:50 +00:00
Patrick Robertson
0765640bff
Fix up tiktok dropin for slightly modified generic_extractor format
2025-03-17 10:31:22 +00:00
Patrick Robertson
06b1f4c0ca
Fix lingering merge conflict issues
2025-03-17 10:12:55 +00:00
Patrick Robertson
59b910ec30
Merge main
2025-03-17 10:05:11 +00:00
Patrick Robertson
7e360240bf
Copy ytdlp code into AA project - seems like ytdlp won't be merged anytime soon
2025-03-17 09:57:05 +00:00
Patrick Robertson
9e03d745d8
Add '-it' to the list of docker flags, so that docker gives a colour log output
2025-03-17 09:45:12 +00:00
Patrick Robertson
7badf89c28
Create the 'secrets' folder if it doesn't exist on first run
...
Easier setup for users
2025-03-17 09:40:46 +00:00
Patrick Robertson
d59530c8e7
Fix if logic bug
2025-03-17 09:40:27 +00:00
Patrick Robertson
0ec5451f66
Nicer error log when no URLs provided for CLI feeder - don't need the stacktrace
2025-03-17 09:34:33 +00:00
Patrick Robertson
99e9ac2465
Fix 'Syntax Error' warning in python3.12+
2025-03-17 09:29:51 +00:00
Patrick Robertson
42162c5e3f
Various docs improvements based on Friday Office Hours discussion
2025-03-17 09:23:43 +00:00
Patrick Robertson
3afe519176
Fix link to module types in config editor
2025-03-17 09:17:17 +00:00
Patrick Robertson
f13349bacf
Fix incorrect path in cp
2025-03-16 10:33:52 +00:00
Patrick Robertson
92c79ed994
Remove schema.json file from git - is auto-generated on release
2025-03-16 10:27:08 +00:00
Patrick Robertson
2643b8e717
Update material version, minify code
2025-03-16 10:22:54 +00:00
Patrick Robertson
b2238427a0
Merge pull request #251 from bellingcat/ruff-check
...
Fix pre-commit for ruff check
2025-03-14 14:22:03 +00:00
Patrick Robertson
282380d8cc
Add note on skipping pre-commit hook
2025-03-14 14:20:32 +00:00
Patrick Robertson
6920585f6d
Version bump to 0.13.6
2025-03-14 13:42:58 +00:00
Patrick Robertson
17463de937
Merge pull request #247 from bellingcat/opentimestamps
...
Opentimestamps Module
2025-03-14 13:41:46 +00:00
Patrick Robertson
29cc1d317f
Fix pre-commit for ruff check
2025-03-14 13:40:57 +00:00
Patrick Robertson
733aef0b08
Merge pull request #246 from bellingcat/webdriver-cookies
...
Better checking of cookies to add to webdriver + generic extractor tweaks
2025-03-14 13:35:58 +00:00
erinhmclark
562d06916e
Revert pre commit
2025-03-14 13:08:57 +00:00
Patrick Robertson
b21467c922
Fix ruff checks
2025-03-14 12:59:37 +00:00
Patrick Robertson
a8e5585e6c
github format
2025-03-14 12:52:01 +00:00
Patrick Robertson
abaeec0cc6
Add ruff check
2025-03-14 12:48:06 +00:00
Patrick Robertson
19715c8ec2
Merge branch 'main' into webdriver-cookies
2025-03-14 12:44:48 +00:00
Patrick Robertson
17ae75fb95
Ruff fixes
2025-03-14 12:38:12 +00:00
Patrick Robertson
b8da7607e8
Merge branch 'main' into opentimestamps
2025-03-14 12:36:03 +00:00
Erin Clark
a01a873f37
Merge pull request #244 from bellingcat/linting_etc
...
Add Linting and Formatting with Ruff.
- Add pre-commit hook for formatting (not lint fixing, this should be done manually)
- Move Makefile to project root and add commands for linting, test, docs, Docker
- Add GH Action for ruff to check linting and formatting
- Make suggested changes from current ruff configs
2025-03-14 12:27:04 +00:00
