docs(09-02): complete LimiterRegistry plan
This commit is contained in:
@@ -204,7 +204,7 @@ Requirements for initial release. Each maps to roadmap phases.
|
|||||||
|
|
||||||
### OSINT/Recon — Infrastructure
|
### OSINT/Recon — Infrastructure
|
||||||
|
|
||||||
- [ ] **RECON-INFRA-05**: Per-source rate limiter with configurable limits
|
- [x] **RECON-INFRA-05**: Per-source rate limiter with configurable limits
|
||||||
- [ ] **RECON-INFRA-06**: Stealth mode (--stealth) with UA rotation and increased delays
|
- [ ] **RECON-INFRA-06**: Stealth mode (--stealth) with UA rotation and increased delays
|
||||||
- [ ] **RECON-INFRA-07**: robots.txt respect (--respect-robots, default on)
|
- [ ] **RECON-INFRA-07**: robots.txt respect (--respect-robots, default on)
|
||||||
- [ ] **RECON-INFRA-08**: Recon full command — parallel sweep across all sources with deduplication
|
- [ ] **RECON-INFRA-08**: Recon full command — parallel sweep across all sources with deduplication
|
||||||
|
|||||||
97
.planning/phases/09-osint-infrastructure/09-02-SUMMARY.md
Normal file
97
.planning/phases/09-osint-infrastructure/09-02-SUMMARY.md
Normal file
@@ -0,0 +1,97 @@
|
|||||||
|
---
|
||||||
|
phase: 09-osint-infrastructure
|
||||||
|
plan: 02
|
||||||
|
subsystem: recon
|
||||||
|
tags: [recon, rate-limiting, stealth, infrastructure]
|
||||||
|
requires:
|
||||||
|
- golang.org/x/time/rate
|
||||||
|
provides:
|
||||||
|
- pkg/recon.LimiterRegistry
|
||||||
|
- pkg/recon.NewLimiterRegistry
|
||||||
|
- LimiterRegistry.For
|
||||||
|
- LimiterRegistry.Wait
|
||||||
|
affects: []
|
||||||
|
tech-stack:
|
||||||
|
added: []
|
||||||
|
patterns:
|
||||||
|
- per-source rate.Limiter map guarded by sync.Mutex
|
||||||
|
- idempotent registry lookup (first registration wins)
|
||||||
|
- optional stealth jitter (100ms-1s) gated by flag
|
||||||
|
key-files:
|
||||||
|
created:
|
||||||
|
- pkg/recon/limiter.go
|
||||||
|
- pkg/recon/limiter_test.go
|
||||||
|
modified: []
|
||||||
|
decisions:
|
||||||
|
- Jitter range fixed at 100ms + rand[0,900)ms per plan spec (max ~1000ms)
|
||||||
|
- First For() call wins; later rate/burst arguments are ignored to preserve token state
|
||||||
|
- Added TestJitterDisabled as extra guard (Rule 2: correctness)
|
||||||
|
metrics:
|
||||||
|
completed: 2026-04-05
|
||||||
|
duration: ~5 min
|
||||||
|
tasks: 1/1
|
||||||
|
requirements: [RECON-INFRA-05]
|
||||||
|
---
|
||||||
|
|
||||||
|
# Phase 09 Plan 02: LimiterRegistry Summary
|
||||||
|
|
||||||
|
Per-source rate limiting foundation using `golang.org/x/time/rate` with optional stealth jitter, keyed by source name via a mutex-guarded registry.
|
||||||
|
|
||||||
|
## What Was Built
|
||||||
|
|
||||||
|
- **`pkg/recon/limiter.go`** — `LimiterRegistry` type with:
|
||||||
|
- `NewLimiterRegistry()` constructor (empty map)
|
||||||
|
- `For(name, r, burst) *rate.Limiter` — idempotent lookup; creates on first call, returns the same pointer on subsequent calls
|
||||||
|
- `Wait(ctx, name, r, burst, stealth) error` — acquires limiter (via `For`), blocks on `limiter.Wait(ctx)`, then sleeps a random 100ms–1000ms jitter if `stealth==true`, respecting ctx cancellation during the sleep
|
||||||
|
- **`pkg/recon/limiter_test.go`** — 5 tests:
|
||||||
|
- `TestLimiterPerSourceIsolation` — different names → distinct `*rate.Limiter`
|
||||||
|
- `TestLimiterIdempotent` — same name → same pointer
|
||||||
|
- `TestWaitRespectsContext` — pre-cancelled ctx returns error
|
||||||
|
- `TestJitterRange` — stealth=true elapsed ∈ [90ms, 1200ms]
|
||||||
|
- `TestJitterDisabled` — stealth=false elapsed < 50ms
|
||||||
|
|
||||||
|
## Satisfies
|
||||||
|
|
||||||
|
- **RECON-INFRA-05** — "each source owns its own limiter — no centralization". Verified via `TestLimiterPerSourceIsolation`.
|
||||||
|
- Partial foundation for **RECON-INFRA-06** (stealth jitter) — fully wired into the engine in plan 09-03.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
```
|
||||||
|
go test -run 'TestLimiter|TestWait|TestJitter' -count=1 -v \
|
||||||
|
./pkg/recon/limiter.go ./pkg/recon/limiter_test.go ./pkg/recon/source.go
|
||||||
|
```
|
||||||
|
|
||||||
|
Result: 5/5 PASS.
|
||||||
|
|
||||||
|
Tests were executed via explicit file list because sibling plans in the same wave (09-03, 09-04) have landed test files (`engine_test.go`, `robots_test.go`) whose implementation files are still in flight. This is expected parallel-wave behavior and does not affect the correctness of this plan's code. Once all wave-1 plans merge, `go test ./pkg/recon/...` will run the full suite.
|
||||||
|
|
||||||
|
`go vet ./pkg/recon/limiter.go ./pkg/recon/source.go` is clean.
|
||||||
|
|
||||||
|
## Deviations from Plan
|
||||||
|
|
||||||
|
### Auto-fixed Issues
|
||||||
|
|
||||||
|
**1. [Rule 2 — Missing coverage] Added `TestJitterDisabled`**
|
||||||
|
- **Found during:** Task 1
|
||||||
|
- **Issue:** Plan tested jitter upper/lower bounds with stealth on, but did not assert the negative case — a regression that applied jitter even when stealth=false would go undetected.
|
||||||
|
- **Fix:** Added `TestJitterDisabled` asserting elapsed < 50ms when stealth=false.
|
||||||
|
- **Files modified:** `pkg/recon/limiter_test.go`
|
||||||
|
- **Commit:** 590fc33
|
||||||
|
|
||||||
|
No architectural deviations. No blockers.
|
||||||
|
|
||||||
|
## Commits
|
||||||
|
|
||||||
|
- `590fc33` — feat(09-02): add LimiterRegistry with per-source rate limiters and jitter
|
||||||
|
|
||||||
|
## Self-Check: PASSED
|
||||||
|
|
||||||
|
- FOUND: pkg/recon/limiter.go
|
||||||
|
- FOUND: pkg/recon/limiter_test.go
|
||||||
|
- FOUND commit: 590fc33
|
||||||
|
- All plan tasks (1/1) complete
|
||||||
|
- All plan success criteria met:
|
||||||
|
- LimiterRegistry exported with `For` and `Wait` ✓
|
||||||
|
- Each source receives its own `*rate.Limiter` ✓
|
||||||
|
- Stealth jitter range 100ms–1s enforced ✓
|
||||||
Reference in New Issue
Block a user