Commit Graph

7 Commits

Author SHA1 Message Date
erinhmclark
e97ccf8a73 Separate setup() and module_setup(). 2025-02-10 18:07:47 +00:00
erinhmclark
2c3d1f591f Separate setup() and module_setup(). 2025-02-10 17:25:15 +00:00
msramalho
15abf686b1 decouples s3_storage from hash_enricher 2025-02-10 15:48:54 +00:00
Patrick Robertson
c574b694ed Set up screenshot enricher to use authentication/cookies 2025-02-03 17:25:59 +01:00
Patrick Robertson
7a2be5a0da Add cookie extraction to 'authentication' options, get generic_extractor working using this info 2025-02-03 16:03:07 +01:00
Patrick Robertson
c25d5cae84 Remove ArchivingContext completely
Context for a specific url/item is now passed around via the metadata (metadata.set_context('key', 'val') and metadata.get_context('key', default='something')
The only other thing that was passed around in ArchivingContext was the storage info, which is already accessible now via self.config
2025-01-30 17:50:54 +01:00
Patrick Robertson
d6b4b7a932 Further cleanup
* Removes (partly) the ArchivingOrchestrator
* Removes the cli_feeder module, and makes it the 'default', allowing you to pass URLs directly on the command line, without having to use the cumbersome --cli_feeder.urls. Just do auto-archiver https://my.url.com
* More unit tests
* Improved error handling
2025-01-30 16:44:40 +01:00