Commit Graph

25 Commits

Author SHA1 Message Date
JustAnotherArchivist
c5a5dcb92c snscrape is now on PyPI 2018-10-09 17:26:03 +02:00
JustAnotherArchivist
cfb1c9a2aa Version 0.1.3 v0.1.3 2018-10-01 03:26:22 +02:00
JustAnotherArchivist
d0d3c8b2a6 Better log output for temporary failures (fixes #2) 2018-10-01 03:24:29 +02:00
JustAnotherArchivist
4d0350e541 Disable "quality filter" on Twitter (fixes #3) 2018-10-01 02:51:33 +02:00
JustAnotherArchivist
d17aa15bcb Version 0.1.2 v0.1.2 2018-09-11 12:44:07 +02:00
JustAnotherArchivist
d1ef280d6e Fix snscrape.modules not getting installed 2018-09-11 12:43:10 +02:00
JustAnotherArchivist
2823272e0b Version 0.1.1 v0.1.1 2018-09-11 12:30:35 +02:00
JustAnotherArchivist
540f557002 Fix typo in setup.py preventing installation 2018-09-11 12:30:21 +02:00
JustAnotherArchivist
5fc60fe978 Version 0.1 v0.1 2018-09-10 22:15:11 +02:00
JustAnotherArchivist
cf36e8be97 Add README, LICENSE, and metadata 2018-09-10 22:15:03 +02:00
JustAnotherArchivist
0350ab0692 Fix Facebook scraper returning strings instead of Items 2018-09-10 19:38:43 +02:00
JustAnotherArchivist
6b6ae3d33b Rename from socialmediascraper to snscrape 2018-08-21 22:54:14 +02:00
JustAnotherArchivist
9fb3ac6013 Add support for Google+ user profiles 2018-08-21 18:58:43 +02:00
JustAnotherArchivist
897f5bebe6 Add support for POST requests 2018-08-21 18:58:09 +02:00
JustAnotherArchivist
e28a2cdb4b Fix Instagram again
- __a=1 is no longer supported, so we need to extract the JSON from the HTML page instead.
- There is now a X-Instagram-GIS header that needs to be set correctly.
2018-08-21 18:55:40 +02:00
JustAnotherArchivist
5a084af85c Fix Instagram
Instagram dropped the max_id parameter, so it is no longer possible to iterate over the posts so easily. Switch to GraphQL instead, which is what's used in the browser as well.
2018-08-21 18:50:00 +02:00
JustAnotherArchivist
14831d4137 Add support for Facebook user profiles 2018-08-21 18:48:34 +02:00
JustAnotherArchivist
6d54655a7f Add support for Instagram user profiles 2018-08-21 18:47:44 +02:00
JustAnotherArchivist
3ab69a1a0f Merge Twitter user and hashtag into one, and add support for generic Twitter search scrapes 2018-08-21 18:46:34 +02:00
JustAnotherArchivist
d03c82d413 Support nested inheritance from socialmediascraper.base.Scraper 2018-08-21 18:44:15 +02:00
JustAnotherArchivist
02473876d7 Add milliseconds to the log timestamps 2018-08-21 18:43:44 +02:00
JustAnotherArchivist
e3190ee541 Add support for Twitter hashtags 2018-08-21 18:40:42 +02:00
JustAnotherArchivist
606b81e066 Use a session for proper cookie handling, and add exponential backoff in case of errors 2018-08-21 18:39:39 +02:00
JustAnotherArchivist
d085018a5f Split up into modules 2018-08-21 18:35:40 +02:00
JustAnotherArchivist
1ae006b268 Initial commit 2018-08-21 18:28:16 +02:00