144 lines
7.9 KiB
Markdown
144 lines
7.9 KiB
Markdown
---
|
|
phase: 05-verification-engine
|
|
plan: 05
|
|
subsystem: cmd
|
|
tags: [cli, verification, scan, output, consent]
|
|
one-liner: "Wire --verify into scan pipeline with consent gate, configurable timeout/workers, and VERIFY column rendering in output table."
|
|
requires:
|
|
- verify.EnsureConsent (Plan 05-02)
|
|
- verify.HTTPVerifier + VerifyAll (Plan 05-03)
|
|
- storage.Finding verify_* columns (Plan 05-01)
|
|
- engine.Finding verification fields (Plan 05-01)
|
|
provides:
|
|
- Working `keyhunter scan --verify` command end-to-end
|
|
- `--verify-timeout` and `--verify-workers` user-facing flags
|
|
- VERIFY column + metadata line in table output (backward compatible)
|
|
affects:
|
|
- cmd/scan.go RunE now collects findings, verifies, then persists
|
|
- pkg/output/table.go now renders verify column conditionally
|
|
tech-stack:
|
|
added: []
|
|
patterns:
|
|
- "Collect-then-verify-then-persist batch pattern (verification is not per-finding streaming)"
|
|
- "Provider+masked-key tuple for back-assigning Result to Finding index"
|
|
- "Conditional table columns driven by anyVerified flag for backward compat"
|
|
key-files:
|
|
created:
|
|
- cmd/scan_test.go
|
|
- pkg/output/table_test.go
|
|
- .planning/phases/05-verification-engine/05-05-SUMMARY.md
|
|
modified:
|
|
- cmd/scan.go
|
|
- pkg/output/table.go
|
|
decisions:
|
|
- "Verification runs in batch after scan completes, not streamed per-finding — matches plan requirement and keeps consent prompt off the hot path."
|
|
- "Result→Finding back-assignment via provider+KeyMasked tuple (VerifyAll preserves neither order nor identity)."
|
|
- "Scoped scan_test.go to flag-registration only (per plan escape hatch) — full RunE integration test would require stdin injection refactor that's out of scope."
|
|
- "Table column conditionally added based on anyVerified to preserve pre-Phase-5 output format exactly when --verify is not used."
|
|
- "Metadata keys rendered in sorted order for deterministic test assertions and stable display."
|
|
metrics:
|
|
duration: "~12min"
|
|
completed: "2026-04-05T12:54:59Z"
|
|
tasks: 2
|
|
commits: 4
|
|
requirements: [VRFY-01, VRFY-04, VRFY-05]
|
|
---
|
|
|
|
# Phase 5 Plan 05: Scan --verify Integration Summary
|
|
|
|
## Objective
|
|
|
|
Wire Plans 05-02/03/04 together into the scan command: gate `--verify` behind consent, run the HTTPVerifier over collected findings, persist verify results, and render a new VERIFY column in the output table. Add `--verify-timeout` and `--verify-workers` flags.
|
|
|
|
## What Was Built
|
|
|
|
### Task 1: Scan command wiring (cmd/scan.go)
|
|
|
|
- Added `flagVerifyTimeout time.Duration` (default 10s) and `flagVerifyWorkers int` (default 10) package-level vars.
|
|
- Registered `--verify-timeout` and `--verify-workers` flags in `init()`.
|
|
- Imported `github.com/salvacybersec/keyhunter/pkg/verify`.
|
|
- Refactored `RunE` scan loop:
|
|
1. **Collect** — drain `eng.Scan` channel into `[]engine.Finding` without persisting.
|
|
2. **Verify** — when `--verify && len(findings) > 0`, call `verify.EnsureConsent(db, os.Stdin, os.Stderr)`. If declined, print notice to stderr and skip. If granted, create `verify.NewHTTPVerifier(flagVerifyTimeout)`, call `VerifyAll(ctx, findings, reg, flagVerifyWorkers)`, and back-assign each `Result` to its `Finding` via a `provider|masked` lookup index (populating `Verified`, `VerifyStatus`, `VerifyHTTPCode`, `VerifyMetadata`, `VerifyError`).
|
|
3. **Persist** — loop over enriched findings once, build `storage.Finding` with verify_* fields, call `db.SaveFinding`.
|
|
- Output rendering unchanged at this layer (Task 2 handles the column).
|
|
|
|
### Task 2: Output table verify column (pkg/output/table.go)
|
|
|
|
- Added five lipgloss styles: `styleVerifyLive` (green), `styleVerifyDead` (red), `styleVerifyRate` (yellow), `styleVerifyErr` (red), `styleVerifyUnk` (gray).
|
|
- `PrintFindings` now scans findings once to compute `anyVerified`.
|
|
- When `anyVerified` is true, the header gains a `VERIFY` column and each row prints `verifySymbol(f)` (✓ live / ✗ dead / ⚠ rate / ! err / ? unk). When false, the layout matches the exact pre-Phase-5 output (backward compatible — `TestPrintFindings_NoVerification_Unchanged` enforces this).
|
|
- When `len(f.VerifyMetadata) > 0`, an indented secondary line ` ↳ key: value, key: value` is rendered with keys sorted alphabetically for deterministic output.
|
|
- New `verifySymbol(f engine.Finding) string` helper maps `VerifyStatus` → colored glyph.
|
|
|
|
## Tests
|
|
|
|
### cmd/scan_test.go (new)
|
|
|
|
- `TestScan_VerifyFlags_Registered/verify-timeout_flag_exists_with_10s_default`
|
|
- `TestScan_VerifyFlags_Registered/verify-workers_flag_exists_with_default_10`
|
|
- `TestScan_VerifyFlags_Registered/verify_flag_still_present`
|
|
|
|
All three pass. Per plan escape hatch, behavioral integration tests (declined/granted consent end-to-end) were deferred because the current `scanCmd.RunE` reads `os.Stdin` directly without injection — refactoring it to accept an injected reader would ripple outside Plan 05-05's scope. Behavior is exercised indirectly via `pkg/verify/consent_test.go` + `pkg/verify/verifier_test.go` from Wave 1.
|
|
|
|
### pkg/output/table_test.go (new)
|
|
|
|
- `TestPrintFindings_NoVerification_Unchanged` — asserts VERIFY header absent when no findings are verified (backward compat lock).
|
|
- `TestPrintFindings_LiveVerification_ShowsCheck` — verified live finding produces VERIFY header and "live" text in output.
|
|
- `TestPrintFindings_Metadata_Rendered` — metadata pairs render on indented line, sorted alphabetically (org before tier).
|
|
|
|
Uses `os.Pipe` + `os.Stdout` swap to capture stdout, strips ANSI escape sequences via regex before assertions. All three pass.
|
|
|
|
## Verification Results
|
|
|
|
```
|
|
$ go build ./...
|
|
(clean)
|
|
|
|
$ go test ./...
|
|
ok github.com/salvacybersec/keyhunter/cmd 0.005s
|
|
ok github.com/salvacybersec/keyhunter/pkg/engine 0.242s
|
|
ok github.com/salvacybersec/keyhunter/pkg/engine/sources (cached)
|
|
ok github.com/salvacybersec/keyhunter/pkg/legal (cached)
|
|
ok github.com/salvacybersec/keyhunter/pkg/output 0.004s
|
|
ok github.com/salvacybersec/keyhunter/pkg/providers 0.944s
|
|
ok github.com/salvacybersec/keyhunter/pkg/storage (cached)
|
|
ok github.com/salvacybersec/keyhunter/pkg/verify 0.393s
|
|
|
|
$ go run . scan --help | grep verify
|
|
--verify actively verify found keys (opt-in, Phase 5)
|
|
--verify-timeout duration per-key verification HTTP timeout (default 10s) (default 10s)
|
|
--verify-workers int parallel workers for key verification (default 10) (default 10)
|
|
```
|
|
|
|
## Commits
|
|
|
|
| Hash | Message |
|
|
| ------- | ------------------------------------------------------------------------- |
|
|
| d537078 | test(05-05): add failing test for --verify-timeout/--verify-workers flags |
|
|
| 6fc0abe | feat(05-05): wire --verify into scan pipeline with consent gate |
|
|
| edba8fb | test(05-05): add failing tests for VERIFY column and metadata rendering |
|
|
| cc9dabe | feat(05-05): render VERIFY column and metadata line in output table |
|
|
|
|
## Deviations from Plan
|
|
|
|
None — plan executed as written. Test scope was explicitly scoped down to flag-registration per the plan's own escape hatch ("Prefer the lightweight flag-registration test to avoid pulling the full scan path into tests").
|
|
|
|
## Success Criteria Check
|
|
|
|
- [x] VRFY-01: consent prompt gates --verify on first use (via `verify.EnsureConsent` call before verifier)
|
|
- [x] VRFY-04: metadata displayed under finding when extracted (indented `↳` line)
|
|
- [x] VRFY-05: `--verify-timeout` and `--verify-workers` flags work (threaded through `NewHTTPVerifier` + `VerifyAll` workers arg)
|
|
- [x] Unverified scans unchanged from Phase 4 behavior (backward-compat test enforces identical header format)
|
|
- [x] `go build ./...` clean
|
|
- [x] `go test ./... -v` green across all modified packages
|
|
- [x] `go run . scan --help` shows `--verify`, `--verify-timeout`, `--verify-workers`
|
|
|
|
## Known Stubs
|
|
|
|
None. All wiring is live; no placeholders in the verified code paths.
|
|
|
|
## Self-Check: PASSED
|
|
|
|
All 5 key files exist on disk. All 4 commit hashes are present in git history.
|