Commit Graph

14 Commits

Author SHA1 Message Date
erinhmclark
e76551ba22 Add documentation, pre-commit hook, more make commands and 2025-03-13 13:21:32 +00:00
erinhmclark
8ca7698fa0 Move Makefile and fix import error with unused import. 2025-03-11 19:58:02 +00:00
erinhmclark
81aa343f21 Merge main. 2025-03-11 10:45:07 +00:00
Patrick Robertson
a9c3477289 Improve docs on the path_generator and filename_generator config options 2025-03-10 16:43:14 +00:00
Patrick Robertson
770f4c8a3d Refactoring of storage code:
1. Fix some bugs in local_storage
2. Refactor unit tests to not set Media.key explicitly (unless it's well-known beforehand, which it isn't)
3. Limit length of URL for 'url' type path_generator
4. Throw an error if 'save_to' of local storage is too long
5. A few other tidyups
2025-03-10 16:39:48 +00:00
Patrick Robertson
e89a8da3b4 Unit tests for storage types + fix storage too long issues for local storage 2025-03-10 11:30:15 +00:00
Patrick Robertson
a9802dd004 Remove the global _LAZY_LOADED_MODULES and allow each instance of ArchivingOrchestrator to load its own modules 2025-02-19 12:25:35 +00:00
Patrick Robertson
a0c4a82825 Improved docstrings for base modules 2025-02-12 11:32:13 +00:00
msramalho
12f14cccc9 fixes gsheet feeder<->db connection via context. 2025-02-10 16:58:35 +00:00
Patrick Robertson
9a8c94b641 Fix getting/setting folder context for metadata 2025-02-03 16:02:17 +01:00
Patrick Robertson
c25d5cae84 Remove ArchivingContext completely
Context for a specific url/item is now passed around via the metadata (metadata.set_context('key', 'val') and metadata.get_context('key', default='something')
The only other thing that was passed around in ArchivingContext was the storage info, which is already accessible now via self.config
2025-01-30 17:50:54 +01:00
Patrick Robertson
953011f368 Don't make modules 'dataclasses' 2025-01-30 16:44:40 +01:00
erinhmclark
cddae65a90 Update modules for new core structure. 2025-01-30 08:42:23 +00:00
erinhmclark
e1a9373336 Refactoring for new config setup 2025-01-27 19:03:02 +00:00