11 Commits

Author	SHA1	Message	Date
Tristan Lee	b7871b060d	added capability to scrape Gab group posts	2022-03-30 09:11:07 -05:00
Logan Williams	571b019137	Fix tests for Twitter transformer	2022-03-22 11:33:27 +01:00
Tristan Lee	e287fd03d9	merged scraper into main and fixed minor merge conflict	2022-03-15 09:12:12 -05:00
Tristan Lee	750f0cc887	added scraper for Instagram	2022-03-14 10:28:10 -05:00
Logan Williams	fd4b617743	Add TwitterTransformer test	2022-03-14 13:39:10 +01:00
Tristan Lee	965bf1e2dc	added youtube scraper, moved from official youtube-dl repo to using yt-dlp because download speed for youtube videos is much better	2022-03-11 17:19:52 -06:00
Tristan Lee	821c39004b	incorporated vkontakte scraper	2022-03-10 22:32:39 -06:00
Tristan Lee	5783206ad8	implemented method to reset database, to enable the 'contoller' fixture scope to be shared across the whole package, which will enable the transformer tests to be run without re-running the scrapers	2022-03-10 10:20:49 -06:00
Tristan Lee	6cf3b8842d	renamed 'archive_media' and 'media' to avoid name collision, changed scope of test fixture controller to 'function' so that db is fresh for each executed test	2022-03-09 13:19:35 -06:00
Tristan Lee	739e1d8484	added capability of running scraper without archiving media, and implemented prototype Telethon scraper for Telegram	2022-03-09 12:12:01 -06:00
Tristan Lee	cd5f68e9e5	added basic unit tests	2022-03-04 12:36:09 -06:00