msramalho
afd9090a4c
concludes logging standardization refactor
2025-06-26 17:20:04 +01:00
msramalho
ce4d7ac649
WIP refactor logging
2025-06-21 15:54:51 +01:00
erinhmclark
e7fa88f1c7
Implementing ruff suggestions.
2025-03-10 21:45:30 +00:00
erinhmclark
85abe1837a
Ruff format with defaults.
2025-03-10 18:44:54 +00:00
Patrick Robertson
034197a81f
Fix typos in csv feeder docs (in manifest)
2025-02-04 13:40:07 +01:00
Patrick Robertson
78e6418249
Unit tests for csv feeder + fix some bugs
2025-02-04 13:37:26 +01:00
Patrick Robertson
b301f60ea3
Fix using validators set in __manifest__.py
...
E.g. you can use the validator 'is_file' to check if a config is a valid file
2025-02-04 13:37:26 +01:00
Patrick Robertson
c25d5cae84
Remove ArchivingContext completely
...
Context for a specific url/item is now passed around via the metadata (metadata.set_context('key', 'val') and metadata.get_context('key', default='something')
The only other thing that was passed around in ArchivingContext was the storage info, which is already accessible now via self.config
2025-01-30 17:50:54 +01:00
Patrick Robertson
d6b4b7a932
Further cleanup
...
* Removes (partly) the ArchivingOrchestrator
* Removes the cli_feeder module, and makes it the 'default', allowing you to pass URLs directly on the command line, without having to use the cumbersome --cli_feeder.urls. Just do auto-archiver https://my.url.com
* More unit tests
* Improved error handling
2025-01-30 16:44:40 +01:00
Patrick Robertson
00a7018f36
Fix up dependency checking (use 'dependencies' instead of 'external_dependencies' -> simpler/easier to remember
2025-01-29 19:25:22 +01:00
erinhmclark
e1a9373336
Refactoring for new config setup
2025-01-27 19:03:02 +00:00
Patrick Robertson
7fd95866a1
Further fixes/changes to loading 'types' for config + manifest edits
2025-01-27 11:48:04 +01:00
erinhmclark
dd402b456f
Fix and add types to manifest
2025-01-24 18:50:11 +00:00
erinhmclark
024fe58377
fix config parsing in manifests, remove module level configs
2025-01-24 13:33:12 +00:00
erinhmclark
0453d95f56
fix config parsing in manifests
2025-01-24 13:24:54 +00:00
erinhmclark
aa7ca93a43
Update manifests and modules
2025-01-24 12:58:16 +00:00
erinhmclark
1274a1b231
More manifests, base modules and rename from archiver to extractor.
2025-01-23 16:40:48 +00:00
erinhmclark
79684f8348
Set up feeder manifests (not merged by source yet)
2025-01-23 09:16:42 +00:00
