Miguel Sozinho Ramalho
6735fa890b
v1.0.1 dependency updates, generic extractor improvements ( #307 )
...
* wacz: allow exceptional cases where more than one resource image is available
* improves generic extractor edge-cases and yt-dlp updates
* REMOVES vk_extractor until further notice
* bumps browsertrix in docker image
* npm version bump on scripts/settings
* poetry updates
* Changed log level on gsheet_feeder_db started from warning to info (#301 )
* closes 305 and further fixes finding local downloads from uncommon ytdlp extractors
* use ffmpeg -bitexact to reduce duplicate content storing
* formatting
* adds yt-dlp curl-cffi
* version bump
* linting
---------
Co-authored-by: Dave Mateer <davemateer@gmail.com >
2025-06-02 20:57:12 +01:00
erinhmclark
a0d955fe84
lock
2025-03-28 13:39:58 +00:00
erinhmclark
bc35116975
Update poetry.lock
2025-03-28 10:37:13 +00:00
erinhmclark
f54b5c5f18
Update poetry.lock
2025-03-26 18:05:04 +00:00
Patrick Robertson
5bb0cbf3ff
Lock poetry file
2025-03-26 15:43:03 +04:00
Patrick Robertson
cb3ae055d6
Also remove certvalidator from poetry/project
2025-03-26 15:11:25 +04:00
Patrick Robertson
4af23e13d1
Bump rfc3161-client to 1.0.1
2025-03-26 14:50:12 +04:00
Patrick Robertson
d6be1ff84f
Merge branch 'main' into timestamping_rewrite
2025-03-26 14:37:51 +04:00
Patrick Robertson
260e76dd3d
Update dependencies
2025-03-24 17:48:25 +04:00
Patrick Robertson
dfde6f1995
Merge main into timestamping_enricher
2025-03-24 15:09:29 +04:00
Patrick Robertson
0f9c6a9a5c
Update yt-dlp to latest
2025-03-24 14:49:18 +04:00
Patrick Robertson
b8da7607e8
Merge branch 'main' into opentimestamps
2025-03-14 12:36:03 +00:00
erinhmclark
e76551ba22
Add documentation, pre-commit hook, more make commands and
2025-03-13 13:21:32 +00:00
Patrick Robertson
1423c10363
Finish off timestamping module
2025-03-12 10:24:57 +00:00
erinhmclark
28c5396b74
Move ruff to dev dependencies.
2025-03-11 17:25:24 +00:00
erinhmclark
cbb0414e5f
Switch to ruff
2025-03-10 16:05:23 +00:00
erinhmclark
f4f2424eb5
Add black and flake8
2025-03-10 13:15:11 +00:00
erinhmclark
1a2d9de819
Update the release process docs and the latest version in pyproject.toml
2025-03-07 17:33:32 +00:00
erinhmclark
6cb7afefdc
Initial Atlos merge
2025-03-05 10:24:54 +00:00
erinhmclark
35b5ab2eb1
Update poetry.lock
2025-02-25 20:17:48 +00:00
Patrick Robertson
6987a4827e
Set poetry packages - remove tsp_client and update cryptography
2025-02-25 11:57:20 +00:00
erinhmclark
f0fd9bf445
Updates tests to use pytest-mock.
2025-02-18 23:32:03 +00:00
erinhmclark
ce5a200d1f
Added tests, updated instagram_tbot_extractor.py raise failure.
2025-02-18 12:59:10 +00:00
erinhmclark
8ed3ef2f33
Merge branch 'main' into tests/add_module_tests
2025-02-14 12:47:40 +00:00
Miguel Sozinho Ramalho
5614af3f63
removes fixed oscrypto dependency, it blocked pypi publishing ( #195 )
...
* disables tests on ubuntu-latest
* drops fixed oscrypto version for git commit
* version bump
2025-02-14 10:51:56 +00:00
erinhmclark
319c1e8f92
Add more tests.
2025-02-14 09:48:37 +00:00
erinhmclark
3fce593aad
Merge branch 'main' into tests/add_module_tests
2025-02-12 19:33:29 +00:00
erinhmclark
d9d936c2ca
Thumbnail enricher fix seconds to minutes.
2025-02-12 12:22:27 +00:00
Patrick Robertson
1d69053dd5
Upgrade certifi
2025-02-11 18:41:37 +00:00
Patrick Robertson
40a95f7348
Add likify deps
2025-02-11 18:40:44 +00:00
Patrick Robertson
dbc564e18b
Add sphinx_book_theme theme to poetry
2025-02-10 22:58:52 +00:00
Patrick Robertson
91ca325fd5
Update yt-dlp to latest version + remove code no longer needed from bluesky dropin
2025-02-04 17:46:46 +01:00
Patrick Robertson
b7d9145f6c
Further tidyups + refactoring for new structure
...
* Add implementation tests for orchestrator + logging tests
* Standardise method/class vars for extractors to see if they are suitable
* Fix bugs with removing default loguru logger (allows further customisation)
* Fix bug loading required fields from file
*
2025-01-30 13:21:10 +01:00
Patrick Robertson
18ff36ce15
Add ruamel to dependencies (replaces pyyaml)
2025-01-29 19:37:41 +01:00
erinhmclark
e1a9373336
Refactoring for new config setup
2025-01-27 19:03:02 +00:00
Patrick Robertson
f1e9ab6751
Merge branch 'main' into load_modules
2025-01-24 15:23:15 +01:00
Patrick Robertson
e8138eac1c
Add ubuntu-latest to the matrix of test runners ( #181 )
...
* Don't clutter logs with info about generic dropin
* Add ubuntu-latest to unit tests
This is currently failing due to an issue with oscrypto and newer openssl https://github.com/wbond/oscrypto/issues/78#issuecomment-1756317472
* fix oscrypto version for ubuntu 24 compatibility (boto3 too see #180 )
---------
Co-authored-by: msramalho <19508417+msramalho@users.noreply.github.com >
2025-01-24 14:03:55 +00:00
Patrick Robertson
54995ad6ab
Further tweaks based on __manifest__.py files
...
Loading configs now works
2025-01-22 13:11:43 +01:00
Patrick Robertson
6388983815
Merge branch 'main' into youtubedlp-rewrite
2025-01-21 16:43:14 +01:00
Patrick Robertson
394bcd8d47
Further refactoring of youtubedl_archiver->base_archiver
...
* Keep twitter_api_archiver
* Remove unit tests for obsolete archivers
* Guess filename of media using the 'Content-Type' header
* Add mechanism to run 'expensive' tests last (see conftest.py) and also flag expensive tests to fail straight off (pytest.mark.incremental)
2025-01-17 11:56:08 +01:00
erinhmclark
d3eec5d90f
Basic docs structure for RTD
2025-01-15 21:45:29 +00:00
Patrick Robertson
3ff7a9444d
Update yt-dlp to latest version (2025.1.12) to add bsky support
2025-01-15 17:58:07 +01:00
Patrick Robertson
74cf1f5f23
Merge branch 'main' into youtubedlp-rewrite
2025-01-15 17:47:23 +01:00
Patrick Robertson
c3dd19f309
Sniff filetype of downloaded media and add extension
...
Also download in chunks - fixes 2 x TODOs
2025-01-15 17:46:47 +01:00
Patrick Robertson
74a4a24a23
Remove toml - unused
...
(pytest etc. use tomli, which is instlled)
2025-01-14 18:13:27 +01:00
Patrick Robertson
20726c1116
Remove tiktok-downloader - getting info is broken
...
TODO: switch to using youtube-dlp
2025-01-14 17:40:45 +01:00
Patrick Robertson
2eb2ab9ac9
Merge branch 'main' into remove_dependencies
2025-01-14 17:39:20 +01:00
Patrick Robertson
080f474d49
Remove minify_html package - HTML file is no longer minified
...
Savings were 5K (~15KB vs ~20KB) for the generated .html file, but minify_html is currently not compatible with python3.13+
2025-01-14 11:36:10 +01:00
Patrick Robertson
6329b72ee8
Remove argparse from dependency list
...
Argparse is installed by default on python>=3.2, we only support python3.10+
Ref: https://pypi.org/project/argparse/#description
2025-01-14 10:50:47 +01:00
Patrick Robertson
ab2eb3c7f5
Add dev dependencies to poetry
2025-01-13 20:42:08 +01:00
