bellingcat/cisticola
1
0
Fork 0
mirror of https://github.com/bellingcat/cisticola.git synced 2026-06-08 03:18:34 +03:00
Code Issues Packages Projects Releases Wiki Activity
41 Commits 14 Branches 4 Tags
965bf1e2dc5013ee5865be4885973be872cedb97
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE
Tristan Lee 965bf1e2dc added youtube scraper, moved from official youtube-dl repo to using yt-dlp because download speed for youtube videos is much better
2022-03-11 17:19:52 -06:00
cisticola
added youtube scraper, moved from official youtube-dl repo to using yt-dlp because download speed for youtube videos is much better
2022-03-11 17:19:52 -06:00
docs
fixed warnings from sphinx-build, made build path consistent with gitignore (removed sphinx build directory from version control)
2022-02-21 16:13:16 -06:00
tests
added youtube scraper, moved from official youtube-dl repo to using yt-dlp because download speed for youtube videos is much better
2022-03-11 17:19:52 -06:00
.gitignore
added capability of running scraper without archiving media, and implemented prototype Telethon scraper for Telegram
2022-03-09 12:12:01 -06:00
Pipfile
added youtube scraper, moved from official youtube-dl repo to using yt-dlp because download speed for youtube videos is much better
2022-03-11 17:19:52 -06:00
Pipfile.lock
implemented method to reset database, to enable the 'contoller' fixture scope to be shared across the whole package, which will enable the transformer tests to be run without re-running the scrapers
2022-03-10 10:20:49 -06:00
pytest.ini
added capability of running scraper without archiving media, and implemented prototype Telethon scraper for Telegram
2022-03-09 12:12:01 -06:00
README.md
Add documentation generation with Sphinx
2022-02-21 17:52:38 +01:00
russian_telegram_ingest.py
renamed 'archive_media' and 'media' to avoid name collision, changed scope of test fixture controller to 'function' so that db is fresh for each executed test
2022-03-09 13:19:35 -06:00
test.py
Implement transformer for TwitterScraper that handles media; implement image OCR and EXIF extraction
2022-03-10 15:34:24 +01:00

README.md

Cisticola

Cisticola, the bird

Description
Coordinates scrapers and interfaces with database
open-source-researchpythonscraperservice
Readme 1 MiB
Languages
Python 99.8%
Shell 0.2%