Commit Graph

10 Commits

Author SHA1 Message Date
erinhmclark
85abe1837a Ruff format with defaults. 2025-03-10 18:44:54 +00:00
Patrick Robertson
78e6418249 Unit tests for csv feeder + fix some bugs 2025-02-04 13:37:26 +01:00
Patrick Robertson
c25d5cae84 Remove ArchivingContext completely
Context for a specific url/item is now passed around via the metadata (metadata.set_context('key', 'val') and metadata.get_context('key', default='something')
The only other thing that was passed around in ArchivingContext was the storage info, which is already accessible now via self.config
2025-01-30 17:50:54 +01:00
Patrick Robertson
d6b4b7a932 Further cleanup
* Removes (partly) the ArchivingOrchestrator
* Removes the cli_feeder module, and makes it the 'default', allowing you to pass URLs directly on the command line, without having to use the cumbersome --cli_feeder.urls. Just do auto-archiver https://my.url.com
* More unit tests
* Improved error handling
2025-01-30 16:44:40 +01:00
erinhmclark
e1a9373336 Refactoring for new config setup 2025-01-27 19:03:02 +00:00
erinhmclark
024fe58377 fix config parsing in manifests, remove module level configs 2025-01-24 13:33:12 +00:00
erinhmclark
0453d95f56 fix config parsing in manifests 2025-01-24 13:24:54 +00:00
erinhmclark
aa7ca93a43 Update manifests and modules 2025-01-24 12:58:16 +00:00
erinhmclark
1274a1b231 More manifests, base modules and rename from archiver to extractor. 2025-01-23 16:40:48 +00:00
erinhmclark
79684f8348 Set up feeder manifests (not merged by source yet) 2025-01-23 09:16:42 +00:00