547 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
msramalho	60a1f3a27a	minor fixes	2023-07-31 16:08:48 +01:00
msramalho	31c07a02e1	Bump version to v0.6.2 for release v0.6.2	2023-07-28 13:10:14 +01:00
msramalho	bd231488ff	parameter fix	2023-07-28 13:10:06 +01:00
msramalho	fb197f1064	excluding telegram embeds	2023-07-28 12:57:15 +01:00
msramalho	ec1a78e973	Bump version to v0.6.1 for release v0.6.1	2023-07-28 12:51:37 +01:00
msramalho	139bdec051	excludes files from perceptual hash	2023-07-28 12:51:24 +01:00
msramalho	f15a70f859	missing hash_enricher import	2023-07-28 12:51:04 +01:00
msramalho	419eaef449	fixes unsued tmp_dir	2023-07-28 12:50:52 +01:00
msramalho	1695954c98	new metadata enricher	2023-07-28 12:46:30 +01:00
msramalho	aa71c85a98	improving ignored content from waczs	2023-07-28 12:19:14 +01:00
msramalho	7a5c9c65bd	detects duplicates before storing, eg: wacz getting media already fetched by another archiver	2023-07-28 10:51:48 +01:00
msramalho	fc93ebaba0	cleanup	2023-07-28 10:49:39 +01:00
msramalho	1b44a302cd	removing some reverse search engines	2023-07-28 10:49:20 +01:00
msramalho	1368f7aebc	feat: making grayscale a toggle	2023-07-28 10:49:03 +01:00
msramalho	e3a0003a47	adding WACZ screenshots	2023-07-27 21:36:25 +01:00
msramalho	59551b3b20	minor improvements: finding best twitter image quality	2023-07-27 21:36:15 +01:00
msramalho	f086d89111	new escape message	2023-07-27 20:14:59 +01:00
msramalho	3dd3775cbd	removes rearchiving logic	2023-07-27 20:14:50 +01:00
msramalho	1e66a2c905	Bump version to v0.6.0 for release v0.6.0	2023-07-27 15:42:29 +01:00
msramalho	e8f44b652e	minor improvements	2023-07-27 15:42:23 +01:00
msramalho	dd034da844	feat: WACZ enricher can now be probed for media, and used as an archiver OR enricher	2023-07-27 15:42:10 +01:00
msramalho	65e3c99483	Bump version to v0.5.28 for release v0.5.28	2023-07-26 16:13:14 +01:00
msramalho	888ad8f004	fix: twitter hack videos extension detection	2023-07-26 16:12:56 +01:00
msramalho	086a9e6c84	fix: remove unnecessary log	2023-07-11 12:17:15 +01:00
msramalho	4d80ee6f02	Bump version to v0.5.27 for release v0.5.27	2023-07-11 12:16:06 +01:00
msramalho	92569ae6be	fix: telegram archiver was outdated for images	2023-07-11 12:15:56 +01:00
msramalho	abaf86c776	Bump version to v0.5.26 for release v0.5.26	2023-07-02 18:42:59 +02:00
msramalho	8005a1955a	fixes #82 twitter api walls	2023-07-02 18:42:43 +02:00
msramalho	b7889a182d	readme update	2023-06-26 18:18:46 +01:00
msramalho	04f827f183	Bump version to v0.5.25 for release v0.5.25	2023-06-26 18:15:45 +01:00
msramalho	485901da3c	security update	2023-06-26 18:15:19 +01:00
msramalho	a2c6cdc111	Bump version to v0.5.24 for release v0.5.24	2023-06-26 17:58:47 +01:00
Miguel Sozinho Ramalho	8bb7883eeb	Merge pull request #81 from emieldatalytica/add_perceptual_hash	2023-06-26 17:34:27 +01:00
msramalho	a0971fc601	final code review changes	2023-06-26 17:32:19 +01:00
msramalho	0cba2c25c6	get all media method	2023-06-26 17:28:19 +01:00
msramalho	7c0b05b276	new column	2023-06-26 17:27:57 +01:00
msramalho	3bbfdf6eba	fix: excluding screenshots	2023-06-26 17:27:49 +01:00
msramalho	a7a6bda1c2	improve missing col behaviour to error log	2023-06-26 17:27:37 +01:00
msramalho	d80145002d	formatter to accommodate properties of inner media	2023-06-26 17:06:50 +01:00
msramalho	b4f86d0e8d	refactor to hash all images and save hex string	2023-06-26 17:06:30 +01:00
msramalho	6cf3e109ed	refactor discovery of inner media elements	2023-06-26 17:05:25 +01:00
msramalho	d4f983e575	adds missing lib numpy	2023-06-26 16:55:19 +01:00
msramalho	88b07d777b	cleanup example file	2023-06-26 16:55:05 +01:00
Emiel de Heij	222e6ddb28	add perceptual hashing with pdq	2023-06-26 15:42:44 +02:00
Emiel de Heij	3e340b2580	change to old status	2023-06-26 15:37:47 +02:00
Emiel de Heij	9fc09c724b	add module for perceptual hashing with pdq	2023-06-26 15:25:55 +02:00
Emiel de Heij	f6e5a14d75	add dependencies	2023-06-26 15:24:55 +02:00
Miguel Sozinho Ramalho	0e9c765b96	Merge pull request #80 from brrttwrks/update_orchestration_example	2023-06-26 13:25:52 +01:00
Eric Nicholas Barrett	87f553661b	add csb_db config to exapmle.orchestration.yaml ... Added an example config section to the example.orchestration.yaml file to clarify how to store info about what's been archived and also stores the archive result	2023-06-21 20:54:14 +04:00
Logan Williams	cc66ee3fd4	bump to patch 23 v0.5.23	2023-06-06 12:24:43 -06:00