Commit Graph

12 Commits

Author SHA1 Message Date
erinhmclark
ca44a40b88 Ruff fix on src. 2025-03-10 19:03:45 +00:00
erinhmclark
85abe1837a Ruff format with defaults. 2025-03-10 18:44:54 +00:00
Patrick Robertson
460a71649c Merge pull request #190 from bellingcat/docs_update
Docs improvement
2025-02-12 12:38:04 +01:00
Patrick Robertson
a0c4a82825 Improved docstrings for base modules 2025-02-12 11:32:13 +00:00
msramalho
e507fc81d2 improves mimetype guessing, previously file.sub.something would not have an extension 2025-02-11 15:02:49 +00:00
Patrick Robertson
63aba6ad39 Fix sphinx-autoapi imports 2025-02-07 21:54:49 +01:00
Patrick Robertson
a506f2a88f Clarify that an extractor's method can also return False if no valid data was found 2025-02-06 10:20:05 +01:00
Patrick Robertson
c25d5cae84 Remove ArchivingContext completely
Context for a specific url/item is now passed around via the metadata (metadata.set_context('key', 'val') and metadata.get_context('key', default='something')
The only other thing that was passed around in ArchivingContext was the storage info, which is already accessible now via self.config
2025-01-30 17:50:54 +01:00
Patrick Robertson
d6b4b7a932 Further cleanup
* Removes (partly) the ArchivingOrchestrator
* Removes the cli_feeder module, and makes it the 'default', allowing you to pass URLs directly on the command line, without having to use the cumbersome --cli_feeder.urls. Just do auto-archiver https://my.url.com
* More unit tests
* Improved error handling
2025-01-30 16:44:40 +01:00
Patrick Robertson
fade68c6f4 Fix up unit tests - dataclass + subclasses not having @dataclass was breaking it 2025-01-30 13:45:24 +01:00
Patrick Robertson
b7d9145f6c Further tidyups + refactoring for new structure
* Add implementation tests for orchestrator + logging tests
* Standardise method/class vars for extractors to see if they are suitable
* Fix bugs with removing default loguru logger (allows further customisation)
* Fix bug loading required fields from file
*
2025-01-30 13:21:10 +01:00
erinhmclark
e1a9373336 Refactoring for new config setup 2025-01-27 19:03:02 +00:00