salvacybersec 62a347f476 feat(10-07): add Replit and CodeSandbox scraping sources ...

- ReplitSource scrapes /search HTML extracting /@user/repl anchors
- CodeSandboxSource scrapes /search HTML extracting /s/slug anchors
- Both use golang.org/x/net/html parser, 10 req/min rate, RespectsRobots=true
- 10 httptest-backed tests covering extraction, ctx cancel, rate/name assertions

2026-04-06 01:16:39 +03:00

..

sources

feat(10-07): add Replit and CodeSandbox scraping sources

2026-04-06 01:16:39 +03:00

dedup_test.go

test(09-03): add failing test for cross-source Dedup

2026-04-06 00:42:45 +03:00

dedup.go

feat(09-03): implement stable cross-source finding Dedup

2026-04-06 00:43:07 +03:00

engine_test.go

feat(09-01): add Engine with parallel fanout and ExampleSource

2026-04-06 00:42:51 +03:00

engine.go

feat(09-01): add Engine with parallel fanout and ExampleSource

2026-04-06 00:42:51 +03:00

example.go

feat(09-01): add Engine with parallel fanout and ExampleSource

2026-04-06 00:42:51 +03:00

integration_test.go

test(09-06): add recon pipeline integration test

2026-04-06 00:51:08 +03:00

limiter_test.go

feat(09-02): add LimiterRegistry with per-source rate limiters and jitter

2026-04-06 00:41:33 +03:00

limiter.go

feat(09-02): add LimiterRegistry with per-source rate limiters and jitter

2026-04-06 00:41:33 +03:00

robots_test.go

test(09-04): add failing tests for RobotsCache

2026-04-06 00:42:03 +03:00

robots.go

feat(09-04): implement RobotsCache with 1h per-host TTL

2026-04-06 00:42:33 +03:00

source.go

feat(09-01): add ReconSource interface and Config

2026-04-06 00:40:46 +03:00

stealth_test.go

test(09-03): add failing test for stealth UA pool

2026-04-06 00:41:55 +03:00

stealth.go

feat(09-03): implement stealth UA pool and StealthHeaders

2026-04-06 00:42:22 +03:00