msramalho
48c1ab3c1f
doc improvements
2025-06-07 19:14:16 +01:00
msramalho
b2ee42ee95
adds the first antibot dropin: VKontakte
2025-06-07 19:10:01 +01:00
msramalho
07ff5baf07
adds Dropin flexible integration for antibot
2025-06-07 19:09:37 +01:00
msramalho
d202d79e0f
lint
2025-06-07 19:06:14 +01:00
msramalho
e2e6490b49
minimal changes
2025-06-07 18:15:21 +01:00
msramalho
952487da30
adds missing bin dependency
2025-06-07 18:14:42 +01:00
msramalho
c7a84bc97a
generalizes ydl info to filename method for reusing
2025-06-07 18:14:08 +01:00
msramalho
8a897cf601
minimal changes: standard naming
2025-06-04 15:06:08 +01:00
Miguel Sozinho Ramalho
14c8af5cc8
Merge pull request #310 from djhmateer/waczscreenshot bug fix
...
counter_screenshots to counter_warc_files in wacz_extractor so don't …
2025-06-04 15:01:12 +01:00
msramalho
264ba82ea0
finish removing screenshot_enricher references
2025-06-04 14:31:07 +01:00
msramalho
5f68c151a0
removes webdriver utils used by screenshot enricher
2025-06-04 14:17:19 +01:00
msramalho
bc8cf2fb29
minor TODO
2025-06-04 14:10:19 +01:00
msramalho
e6f3826a3a
dropping screenshot enricher
2025-06-04 12:08:59 +01:00
msramalho
e5a78a5d06
antibot can be used out of the box
2025-06-04 12:01:42 +01:00
msramalho
258fb4faaf
visual HTML preview improvements
2025-06-04 12:00:40 +01:00
msramalho
22408e2a98
adds test for antibot
2025-06-04 11:59:59 +01:00
msramalho
378b1a6d22
expand S3 objects content type for better preview results in non-latin languages
2025-06-04 11:53:41 +01:00
msramalho
d130c1b3fa
WIP attempt at ytdlp impersonation
2025-06-04 11:53:18 +01:00
msramalho
cbd189c97d
general cleanup
2025-06-04 11:53:01 +01:00
msramalho
d2e8f1a512
introduces antibot step with seleniumbase
2025-06-04 11:20:46 +01:00
Dave Mateer
c772082f0e
counter_screenshots to counter_warc_files in wacz_extractor so don't get error about add mulitple items with same id.
2025-06-03 12:34:41 +01:00
msramalho
ee68f3efee
Merge remote-tracking branch 'origin/main' into feat/seleniumbase
2025-06-03 11:05:16 +01:00
Miguel Sozinho Ramalho
6735fa890b
v1.0.1 dependency updates, generic extractor improvements ( #307 )
...
* wacz: allow exceptional cases where more than one resource image is available
* improves generic extractor edge-cases and yt-dlp updates
* REMOVES vk_extractor until further notice
* bumps browsertrix in docker image
* npm version bump on scripts/settings
* poetry updates
* Changed log level on gsheet_feeder_db started from warning to info (#301 )
* closes 305 and further fixes finding local downloads from uncommon ytdlp extractors
* use ffmpeg -bitexact to reduce duplicate content storing
* formatting
* adds yt-dlp curl-cffi
* version bump
* linting
---------
Co-authored-by: Dave Mateer <davemateer@gmail.com >
2025-06-02 20:57:12 +01:00
Dave Mateer
48be13fb2a
catch for if self.comments are true but no actual comments in video ( #303 )
...
* catch for if self.comments are true but no actual comments in video
* simplifies check code
---------
Co-authored-by: Miguel Sozinho Ramalho <19508417+msramalho@users.noreply.github.com >
2025-06-02 13:02:19 +01:00
msramalho
5cf640af8a
experiments with seleniumbase
2025-04-28 11:08:00 +01:00
erinhmclark
144adaad5b
Only return success for instagram_tbot_extractor.py with content.
2025-03-31 14:14:36 +01:00
Erin Clark
c7c7eb00a1
Merge pull request #286 from bellingcat/version_comparison
...
Small code fixes and GH Actions cache
2025-03-31 12:40:42 +01:00
erinhmclark
7e4ba62918
Small code change
2025-03-31 12:05:39 +01:00
erinhmclark
8940580638
Add poetry cache clear, and small code change
2025-03-31 11:41:26 +01:00
erinhmclark
a590647279
Small code tidy to trigger tests.
2025-03-31 11:23:49 +01:00
Erin Clark
8685b6bf13
Merge pull request #285 from bellingcat/fix-ubuntu-22
...
[WIP] Change order of poetry install - in case this fixes core tests
2025-03-28 15:38:03 +00:00
Patrick Robertson
fd540bd03a
Code change to trigger tests
2025-03-28 18:29:59 +04:00
erinhmclark
68992025b0
Update version comparison.
2025-03-28 14:29:44 +00:00
Erin Clark
96efdcbba1
Merge pull request #281 from bellingcat/add_inst_api_script
...
Add InstagrAPI server script to be used with the Instagram API Extractor.
2025-03-28 13:58:37 +00:00
erinhmclark
5e7c57650b
Update "default" to "auto" for clarity, update docs
2025-03-28 13:16:16 +00:00
erinhmclark
b5dc1854a2
Merge branch 'main' into feat/yt-dlp-pots
2025-03-28 10:42:24 +00:00
Patrick Robertson
25f1f5dc93
Merge pull request #279 from bellingcat/telethon_tweaks
...
Fix calling extractor.cleanup (fixes telethon issue) + tidy up telethon extractor session file naming
2025-03-28 14:13:26 +04:00
erinhmclark
00b29db390
Update documentaion for instagrapi api
2025-03-28 00:41:35 +00:00
Patrick Robertson
a448e2532c
Code tweak for clarity
2025-03-27 15:20:52 +04:00
Patrick Robertson
b7949a489f
Simplify telethon unit tests for CI (don't use TestExtractorBase - it causes loading issues)
2025-03-26 23:51:21 +04:00
Patrick Robertson
95ea9fb231
Telethon unit tests + tidyup
2025-03-26 22:53:27 +04:00
Patrick Robertson
17d2d14680
Fix running 'cleanup' method on extractors that fail to start
2025-03-26 22:52:52 +04:00
erinhmclark
2cad5edea8
Fix default config
2025-03-26 17:33:00 +00:00
Patrick Robertson
580de88366
Set the new session filename *before* copying
...
Fixes a potential bug whereby if the copy fails for some reason, the 'cleanup' command would remove the original session file
2025-03-26 21:32:23 +04:00
erinhmclark
093ce34a6a
Ruff format.
2025-03-26 17:02:20 +00:00
erinhmclark
7872d9356c
Merge branch 'main' into feat/yt-dlp-pots
2025-03-26 17:00:38 +00:00
erinhmclark
ae523eb06f
Udpate PO token generation script method
2025-03-26 16:45:29 +00:00
erinhmclark
d87c0dc3a9
Implement update for pot plugin.
2025-03-26 16:02:29 +00:00
Patrick Robertson
20bc80b9ef
Slightly more consistent/tidier naming for the session files
...
Don't add/remove .session from name, keep the file name without .session at all times
2025-03-26 15:57:11 +04:00
Patrick Robertson
76e90dd23a
Small code tidy ups
2025-03-26 15:34:33 +04:00
