From 52925020008ca37e6140febff34ceda4b04ec3fd Mon Sep 17 00:00:00 2001 From: salvacybersec Date: Sun, 5 Apr 2026 23:29:13 +0300 Subject: [PATCH] docs(06-01): complete formatter interface + TableFormatter plan --- .planning/REQUIREMENTS.md | 4 +- .planning/ROADMAP.md | 2 +- .planning/STATE.md | 24 ++-- .../06-output-reporting/06-01-SUMMARY.md | 133 ++++++++++++++++++ 4 files changed, 149 insertions(+), 14 deletions(-) create mode 100644 .planning/phases/06-output-reporting/06-01-SUMMARY.md diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 7948c26..508c2bf 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -50,12 +50,12 @@ Requirements for initial release. Each maps to roadmap phases. ### Output & Reporting -- [ ] **OUT-01**: Colored terminal table output (default) +- [x] **OUT-01**: Colored terminal table output (default) - [ ] **OUT-02**: JSON output format - [ ] **OUT-03**: SARIF output format (CI/CD compatible) - [ ] **OUT-04**: CSV output format - [ ] **OUT-05**: Key masking by default (first 8 + last 4 chars) with --unmask flag for full keys -- [ ] **OUT-06**: Exit codes: 0=clean, 1=keys found, 2=error +- [x] **OUT-06**: Exit codes: 0=clean, 1=keys found, 2=error ### Key Management diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 1358639..ab091d1 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -142,7 +142,7 @@ Plans: **Plans**: 6 plans Plans: -- [ ] 06-01-PLAN.md — Wave 0: Formatter interface, colors.go (TTY/NO_COLOR), refactor TableFormatter +- [x] 06-01-PLAN.md — Wave 0: Formatter interface, colors.go (TTY/NO_COLOR), refactor TableFormatter - [ ] 06-02-PLAN.md — JSONFormatter + CSVFormatter (full Finding fields, Unmask option) - [ ] 06-03-PLAN.md — SARIF 2.1.0 formatter with custom structs (rule dedup, level mapping) - [ ] 06-04-PLAN.md — pkg/storage/queries.go: Filters, ListFindingsFiltered, GetFinding, DeleteFinding diff --git a/.planning/STATE.md b/.planning/STATE.md index 4baa0ef..1980ca2 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -2,15 +2,15 @@ gsd_state_version: 1.0 milestone: v1.0 milestone_name: milestone -status: verifying -stopped_at: Completed 05-05-PLAN.md -last_updated: "2026-04-05T12:59:51.061Z" +status: executing +stopped_at: Completed 06-01-PLAN.md +last_updated: "2026-04-05T20:29:09.502Z" last_activity: 2026-04-05 progress: total_phases: 18 completed_phases: 5 - total_plans: 28 - completed_plans: 28 + total_plans: 34 + completed_plans: 29 percent: 20 --- @@ -21,13 +21,13 @@ progress: See: .planning/PROJECT.md (updated 2026-04-04) **Core value:** Detect leaked LLM API keys across more providers and more internet sources than any other tool, with active verification to confirm keys are real and alive. -**Current focus:** Phase 05 — verification-engine +**Current focus:** Phase 06 — output-reporting ## Current Position -Phase: 6 -Plan: Not started -Status: Phase complete — ready for verification +Phase: 06 (output-reporting) — EXECUTING +Plan: 2 of 6 +Status: Ready to execute Last activity: 2026-04-05 Progress: [██░░░░░░░░] 20% @@ -74,6 +74,7 @@ Progress: [██░░░░░░░░] 20% | Phase 05-verification-engine P02 | 7m | 2 tasks | 9 files | | Phase 05-verification-engine P03 | 245s | 2 tasks | 4 files | | Phase 05 P05 | 12min | 2 tasks | 5 files | +| Phase 06 P01 | 8m | 2 tasks | 7 files | ## Accumulated Context @@ -102,6 +103,7 @@ Recent decisions affecting current work: - [Phase 05-verification-engine]: verify.consent setting: granted is sticky across runs; declined is not — users who initially refuse can change mind without manual reset - [Phase 05-verification-engine]: Plan 05-03: HTTPVerifier classifies via YAML VerifySpec only; no per-provider branches. VerifyAll uses ants pool with per-finding Result guarantee. - [Phase 05]: Verification runs in batch mode after scan completes (collect -> verify -> persist) with Result->Finding back-assignment via provider+masked-key tuple +- [Phase 06]: Registry pattern for output formatters; TableFormatter strips ANSI when writer is not a TTY via zero-value lipgloss.Style ### Pending Todos @@ -116,6 +118,6 @@ None yet. ## Session Continuity -Last session: 2026-04-05T12:56:13.570Z -Stopped at: Completed 05-05-PLAN.md +Last session: 2026-04-05T20:29:05.176Z +Stopped at: Completed 06-01-PLAN.md Resume file: None diff --git a/.planning/phases/06-output-reporting/06-01-SUMMARY.md b/.planning/phases/06-output-reporting/06-01-SUMMARY.md new file mode 100644 index 0000000..8eecdc6 --- /dev/null +++ b/.planning/phases/06-output-reporting/06-01-SUMMARY.md @@ -0,0 +1,133 @@ +--- +phase: 06-output-reporting +plan: 01 +subsystem: pkg/output +tags: [output, formatter, registry, tty, colors, refactor] +requirements: [OUT-01, OUT-06] +dependency-graph: + requires: [pkg/engine.Finding] + provides: + - "output.Formatter interface" + - "output.Options struct" + - "output.Registry (Register/Get/Names/ErrUnknownFormat)" + - "output.TableFormatter" + - "output.IsTTY / output.ColorsEnabled" + affects: [cmd/scan.go PrintFindings wrapper] +tech-stack: + added: + - "github.com/mattn/go-isatty (promoted from indirect to direct)" + patterns: + - "Registry via init()-registered formatters" + - "TTY-aware color stripping using zero-value lipgloss.Style" + - "Deprecated wrapper (PrintFindings) delegating to new interface" +key-files: + created: + - pkg/output/formatter.go + - pkg/output/formatter_test.go + - pkg/output/colors.go + - pkg/output/colors_test.go + modified: + - pkg/output/table.go + - pkg/output/table_test.go + - go.mod +decisions: + - "Registry is unguarded: all registration happens at init() before main." + - "newTableStyles(false) returns zero lipgloss.Style values — lipgloss passes text through verbatim, guaranteeing no ANSI escapes on non-TTY writers." + - "PrintFindings(findings, unmask) kept as backward-compat wrapper so cmd/scan.go still compiles unchanged until Plan 06 (scan flag wiring)." + - "ColorsEnabled honours NO_COLOR (https://no-color.org/) before the TTY check." +metrics: + duration: ~8m + completed: 2026-04-05 + tasks: 2 + commits: 3 +--- + +# Phase 06 Plan 01: Formatter Interface + TableFormatter Refactor Summary + +Established the `output.Formatter` interface, a name-keyed registry, and TTY-aware color detection, then refactored the existing colored-table output to implement `Formatter` and register itself as `"table"`. This is the foundation every other Phase 6 formatter (JSON, SARIF, CSV) will plug into. + +## What Was Built + +### 1. Formatter interface + registry (`pkg/output/formatter.go`) + +- `Formatter` interface: `Format(findings []engine.Finding, w io.Writer, opts Options) error` +- `Options` struct: `{ Unmask, ToolName, ToolVersion }` +- `Register(name, f)` — called from `init()` functions +- `Get(name)` — returns `ErrUnknownFormat` wrapped via `fmt.Errorf("%w: %q", ...)` +- `Names()` — sorted list for `--output` help text +- `ErrUnknownFormat` sentinel, discoverable via `errors.Is` + +### 2. TTY detection (`pkg/output/colors.go`) + +- `IsTTY(*os.File) bool` — wraps `isatty.IsTerminal` + `isatty.IsCygwinTerminal`, nil-safe +- `ColorsEnabled(io.Writer) bool` — returns false when `NO_COLOR` is set, when writer is not an `*os.File`, or when the file is not a TTY +- Promoted `github.com/mattn/go-isatty v0.0.20` from indirect (via lipgloss) to a direct dependency in `go.mod` + +### 3. TableFormatter refactor (`pkg/output/table.go`) + +- `TableFormatter struct{}` implements `Formatter`, registered under `"table"` in `init()` +- Writes to the caller-supplied `io.Writer` (no more hardcoded `os.Stdout`) +- New `tableStyles` bundle + `newTableStyles(colored bool)` factory — when `colored==false` every style is a zero `lipgloss.Style`, which renders text verbatim with zero ANSI escapes +- Preserves the exact layout from Phase 5: PROVIDER/KEY/CONFIDENCE/SOURCE/LINE columns plus optional VERIFY column, indented sorted metadata line, footer `"N key(s) found."` +- Respects `Options.Unmask` to toggle between `KeyMasked` and `KeyValue` +- `PrintFindings(findings, unmask)` retained as a deprecated thin wrapper that delegates to `TableFormatter{}.Format(..., os.Stdout, Options{Unmask: unmask})` — keeps `cmd/scan.go` compiling untouched until Plan 06. + +## Tests + +All green on `go test ./pkg/output/... -count=1`: + +- **formatter_test.go**: `Register/Get` round-trip, `Get` unknown → `ErrUnknownFormat` via `errors.Is`, `Names()` sort ordering, `Options` zero-value defaults. +- **colors_test.go**: `ColorsEnabled(&bytes.Buffer{})==false`, `NO_COLOR=1` forces false, typed-nil writer does not panic, `IsTTY(nil)==false`. +- **table_test.go (new)**: + - Empty slice → exact `"No API keys found.\n"` + - Two findings (one verified) written to `bytes.Buffer` → output contains no `\x1b[` escapes, includes `VERIFY` column and `"2 key(s) found."` footer + - Unverified-only header does NOT contain `VERIFY` + - Verified row shows `live` + - Unmask=false renders `KeyMasked`; Unmask=true renders `KeyValue` + - VerifyMetadata `{z:1, a:2}` renders `a: 2` before `z: 1` + - `Get("table")` returns a `TableFormatter` +- **table_test.go (legacy)**: `PrintFindings` stdout-capture tests still pass (backward compat preserved). + +Full project suite (`go test ./...`) green: cmd, engine, engine/sources, legal, output, providers, storage, verify. + +## Key Decisions + +1. **Registry is unguarded.** All `Register` calls happen from package `init()` before `main`, which Go runs sequentially. Adding a mutex would be dead weight. + +2. **Plain styles via zero lipgloss.Style.** Instead of conditionally calling different render functions, `newTableStyles(false)` returns `lipgloss.NewStyle()` for every field. A zero `lipgloss.Style` passes text through verbatim, so the same `fmt.Fprintf(... style.header.Render("PROVIDER") ...)` code path produces colored output on a TTY and plain output to a `bytes.Buffer`. + +3. **Backward-compat wrapper.** `PrintFindings` stays so `cmd/scan.go` compiles without edits. Plan 06 will replace it with a registry lookup when wiring the `--output` flag. Deprecated comment in place. + +4. **NO_COLOR precedence.** `ColorsEnabled` checks `NO_COLOR` before the TTY check, matching the https://no-color.org/ spec ("should check for a NO_COLOR environment variable that, when present (regardless of its value), prevents the addition of ANSI color"). + +## Deviations from Plan + +None — plan executed exactly as written. + +## Commits + +| Commit | Type | Message | +| ------- | ---- | ---------------------------------------------------------- | +| 291c97e | feat | add Formatter interface, Registry, and TTY color detection | +| 8c37252 | test | add failing tests for TableFormatter refactor | +| 8e4db5d | feat | refactor table output into TableFormatter | + +## Foundation for Next Plans + +- **Plan 02** (JSON + SARIF): create `pkg/output/json.go` and `pkg/output/sarif.go`, each with an `init()` calling `Register`. Implement `Format` against `[]engine.Finding`. Use `Options.ToolName`/`Options.ToolVersion` for SARIF `tool.driver` metadata. +- **Plan 03** (CSV): identical pattern with `encoding/csv`. +- **Plan 06** (scan wiring): replace `output.PrintFindings(findings, unmask)` in `cmd/scan.go` with `output.Get(flag)` + `f.Format(findings, os.Stdout, Options{Unmask: unmask, ToolName: "keyhunter", ToolVersion: version})`. + +## Self-Check: PASSED + +- pkg/output/formatter.go — FOUND +- pkg/output/formatter_test.go — FOUND +- pkg/output/colors.go — FOUND +- pkg/output/colors_test.go — FOUND +- pkg/output/table.go — modified (TableFormatter present, `Register("table", ...)` in init) +- pkg/output/table_test.go — modified (TableFormatter test block present) +- go.mod — `github.com/mattn/go-isatty v0.0.20` in direct require block +- Commits 291c97e, 8c37252, 8e4db5d — FOUND in `git log` +- `go build ./...` — succeeds +- `go test ./pkg/output/... -count=1` — PASS +- `go test ./...` — PASS