Use a script to auto-generate documentation for the core modules from the manifest file

docs/source/core_modules.md

@@ -0,0 +1,26 @@
+# Module Documentation
+
+These pages describe the core modules that come with `auto-archiver` and provide the basic functionality for archiving websites on the internet. There are five core module types:
+
+1. Feeders - these 'feed' information (the URLs) from various sources to the `auto-archiver` for processing
+2. Extractors - these 'extract' the page data for a given URL that is fed in by a feeder
+3. Enrichers - these 'enrich' the data extracted in the previous step with additional information
+4. Storage - these 'store' the data in a persistent location (on disk, Google Drive etc.)
+5. Databases - these 'store' the status of the entire archiving process in a log file or database.
+
+
+```{include} modules/autogen/module_list.md
+```
+
+
+```{toctree}
+:maxdepth: 1
+:caption: Core Modules
+:hidden:
+
+modules/feeder
+modules/extractor
+modules/enricher
+modules/storage
+modules/database
+```