salvacybersec/keyhunter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(09-04): complete robots.txt cache plan

Browse Source 
Adds SUMMARY, marks RECON-INFRA-07 complete, updates phase 9 roadmap.
This commit is contained in:
salvacybersec
2026-04-06 00:43:49 +03:00
parent 851b2432b8
commit 4dbc38dcc5
4 changed files with 122 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.planning/REQUIREMENTS.md
View File

@@ -206,7 +206,7 @@ Requirements for initial release. Each maps to roadmap phases.
- [x] **RECON-INFRA-05**: Per-source rate limiter with configurable limits
- [ ] **RECON-INFRA-06**: Stealth mode (--stealth) with UA rotation and increased delays
- [ ] **RECON-INFRA-07**: robots.txt respect (--respect-robots, default on)
- [x] **RECON-INFRA-07**: robots.txt respect (--respect-robots, default on)
- [ ] **RECON-INFRA-08**: Recon full command — parallel sweep across all sources with deduplication
### Dork Engine