From 1d5d12740c2db43bcc83042bc7383315225972e9 Mon Sep 17 00:00:00 2001 From: salvacybersec Date: Mon, 6 Apr 2026 00:42:15 +0300 Subject: [PATCH] docs(09-02): complete LimiterRegistry plan --- .planning/REQUIREMENTS.md | 2 +- .../09-osint-infrastructure/09-02-SUMMARY.md | 97 +++++++++++++++++++ 2 files changed, 98 insertions(+), 1 deletion(-) create mode 100644 .planning/phases/09-osint-infrastructure/09-02-SUMMARY.md diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index d99f3cc..fa71367 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -204,7 +204,7 @@ Requirements for initial release. Each maps to roadmap phases. ### OSINT/Recon — Infrastructure -- [ ] **RECON-INFRA-05**: Per-source rate limiter with configurable limits +- [x] **RECON-INFRA-05**: Per-source rate limiter with configurable limits - [ ] **RECON-INFRA-06**: Stealth mode (--stealth) with UA rotation and increased delays - [ ] **RECON-INFRA-07**: robots.txt respect (--respect-robots, default on) - [ ] **RECON-INFRA-08**: Recon full command — parallel sweep across all sources with deduplication diff --git a/.planning/phases/09-osint-infrastructure/09-02-SUMMARY.md b/.planning/phases/09-osint-infrastructure/09-02-SUMMARY.md new file mode 100644 index 0000000..5a88127 --- /dev/null +++ b/.planning/phases/09-osint-infrastructure/09-02-SUMMARY.md @@ -0,0 +1,97 @@ +--- +phase: 09-osint-infrastructure +plan: 02 +subsystem: recon +tags: [recon, rate-limiting, stealth, infrastructure] +requires: + - golang.org/x/time/rate +provides: + - pkg/recon.LimiterRegistry + - pkg/recon.NewLimiterRegistry + - LimiterRegistry.For + - LimiterRegistry.Wait +affects: [] +tech-stack: + added: [] + patterns: + - per-source rate.Limiter map guarded by sync.Mutex + - idempotent registry lookup (first registration wins) + - optional stealth jitter (100ms-1s) gated by flag +key-files: + created: + - pkg/recon/limiter.go + - pkg/recon/limiter_test.go + modified: [] +decisions: + - Jitter range fixed at 100ms + rand[0,900)ms per plan spec (max ~1000ms) + - First For() call wins; later rate/burst arguments are ignored to preserve token state + - Added TestJitterDisabled as extra guard (Rule 2: correctness) +metrics: + completed: 2026-04-05 + duration: ~5 min + tasks: 1/1 +requirements: [RECON-INFRA-05] +--- + +# Phase 09 Plan 02: LimiterRegistry Summary + +Per-source rate limiting foundation using `golang.org/x/time/rate` with optional stealth jitter, keyed by source name via a mutex-guarded registry. + +## What Was Built + +- **`pkg/recon/limiter.go`** — `LimiterRegistry` type with: + - `NewLimiterRegistry()` constructor (empty map) + - `For(name, r, burst) *rate.Limiter` — idempotent lookup; creates on first call, returns the same pointer on subsequent calls + - `Wait(ctx, name, r, burst, stealth) error` — acquires limiter (via `For`), blocks on `limiter.Wait(ctx)`, then sleeps a random 100ms–1000ms jitter if `stealth==true`, respecting ctx cancellation during the sleep +- **`pkg/recon/limiter_test.go`** — 5 tests: + - `TestLimiterPerSourceIsolation` — different names → distinct `*rate.Limiter` + - `TestLimiterIdempotent` — same name → same pointer + - `TestWaitRespectsContext` — pre-cancelled ctx returns error + - `TestJitterRange` — stealth=true elapsed ∈ [90ms, 1200ms] + - `TestJitterDisabled` — stealth=false elapsed < 50ms + +## Satisfies + +- **RECON-INFRA-05** — "each source owns its own limiter — no centralization". Verified via `TestLimiterPerSourceIsolation`. +- Partial foundation for **RECON-INFRA-06** (stealth jitter) — fully wired into the engine in plan 09-03. + +## Verification + +``` +go test -run 'TestLimiter|TestWait|TestJitter' -count=1 -v \ + ./pkg/recon/limiter.go ./pkg/recon/limiter_test.go ./pkg/recon/source.go +``` + +Result: 5/5 PASS. + +Tests were executed via explicit file list because sibling plans in the same wave (09-03, 09-04) have landed test files (`engine_test.go`, `robots_test.go`) whose implementation files are still in flight. This is expected parallel-wave behavior and does not affect the correctness of this plan's code. Once all wave-1 plans merge, `go test ./pkg/recon/...` will run the full suite. + +`go vet ./pkg/recon/limiter.go ./pkg/recon/source.go` is clean. + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 2 — Missing coverage] Added `TestJitterDisabled`** +- **Found during:** Task 1 +- **Issue:** Plan tested jitter upper/lower bounds with stealth on, but did not assert the negative case — a regression that applied jitter even when stealth=false would go undetected. +- **Fix:** Added `TestJitterDisabled` asserting elapsed < 50ms when stealth=false. +- **Files modified:** `pkg/recon/limiter_test.go` +- **Commit:** 590fc33 + +No architectural deviations. No blockers. + +## Commits + +- `590fc33` — feat(09-02): add LimiterRegistry with per-source rate limiters and jitter + +## Self-Check: PASSED + +- FOUND: pkg/recon/limiter.go +- FOUND: pkg/recon/limiter_test.go +- FOUND commit: 590fc33 +- All plan tasks (1/1) complete +- All plan success criteria met: + - LimiterRegistry exported with `For` and `Wait` ✓ + - Each source receives its own `*rate.Limiter` ✓ + - Stealth jitter range 100ms–1s enforced ✓