Files
keyhunter/.planning/phases/06-output-reporting/06-02-SUMMARY.md
2026-04-05 23:34:51 +03:00

7.8 KiB

phase, plan, subsystem, tags, requirements, dependency-graph, tech-stack, key-files, decisions, metrics
phase plan subsystem tags requirements dependency-graph tech-stack key-files decisions metrics
06-output-reporting 02 pkg/output
output
formatter
json
csv
unmask
OUT-02
OUT-04
requires provides affects
pkg/output.Formatter
pkg/output.Options
pkg/output.Register
pkg/engine.Finding
output.JSONFormatter (registered as "json")
output.CSVFormatter (registered as "csv")
cmd/scan.go --output=json|csv (wired in later plan)
added patterns
encoding/json Encoder with SetIndent for pretty output
encoding/csv Writer for stable, auto-quoted flat rows
omitempty on verify_* JSON fields to keep unverified output compact
init()-time registration into the shared output.Registry
created modified
pkg/output/json.go
pkg/output/json_test.go
pkg/output/csv.go
pkg/output/csv_test.go
JSON `key` field carries masked value by default; `key_masked` always carries masked form regardless of Unmask so consumers can tell which mode produced the file.
Verification fields use `omitempty` so unverified scans stay compact; `verified: false` is always emitted as a required discriminator.
CSV `id` column is the zero-based slice index, not the DB rowid — formatter must run on in-memory findings before persistence.
CSV header order is frozen (id,provider,confidence,key,source,line,detected_at,verified,verify_status) — downstream consumers may index by position.
DetectedAt is serialised as RFC3339 in both formats for cross-format consistency.
duration completed tasks commits
~10m 2026-04-05 2 4

Phase 06 Plan 02: JSON + CSV Formatters Summary

Added JSONFormatter and CSVFormatter to pkg/output, both implementing the Formatter interface established in Plan 01 and both honoring the Unmask option. These are the machine-readable siblings of the colored table output: JSON for pipelines and tooling, CSV for spreadsheets and ad-hoc grep.

What Was Built

1. JSONFormatter (pkg/output/json.go)

  • Renders []engine.Finding as a JSON array with 2-space indent via json.Encoder.SetIndent("", " ").
  • Empty input produces "[]\n" exactly (important for deterministic CI diffs).
  • Private jsonFinding struct controls field names and serialization order: provider, key, key_masked, confidence, source, source_type, line, offset, detected_at, verified, verify_status?, verify_http_code?, verify_metadata?, verify_error?.
  • key column: masked by default, full KeyValue when Options.Unmask == true. key_masked is always the masked form.
  • detected_at serialized as RFC3339.
  • Verification fields tagged omitempty so unverified scans don't carry dead columns.
  • Registers itself as "json" in the output registry via init().

2. CSVFormatter (pkg/output/csv.go)

  • Writes a frozen header row followed by one flat row per finding using encoding/csv.Writer.
  • Header (stable, index-indexable): id,provider,confidence,key,source,line,detected_at,verified,verify_status.
  • id is the zero-based slice index (scan-time id; DB rowid is intentionally not used so this formatter can run on pre-persisted results).
  • key column honors Options.Unmask identically to JSON.
  • verified rendered as "true"/"false", line as a plain integer, detected_at as RFC3339.
  • Commas, quotes, and newlines in source paths are auto-escaped by encoding/csv.
  • Empty input still writes the header row (one line) — consumers can unconditionally parse a header.
  • Registers itself as "csv" in the output registry via init().

Test Coverage

pkg/output/json_test.go

  • TestJSONFormatter_EmptyFindings — exact "[]\n" output.
  • TestJSONFormatter_RegisteredUnderJSON — registry lookup returns JSONFormatter.
  • TestJSONFormatter_MaskedByDefaultkey == KeyMasked, verify fields absent (omitempty).
  • TestJSONFormatter_UnmaskRevealsKeykey == KeyValue, key_masked still masked.
  • TestJSONFormatter_VerifyFieldsPresent — verify_status/code/metadata round-trip when Verified=true.
  • TestJSONFormatter_Indented — 2-space array item indent, 4-space nested field indent.

pkg/output/csv_test.go

  • TestCSVFormatter_RegisteredUnderCSV — registry lookup.
  • TestCSVFormatter_HeaderOnly — empty findings produce exactly the 9-column header.
  • TestCSVFormatter_RowMasked — full row round-trips via csv.NewReader, masked key, correct types.
  • TestCSVFormatter_Unmask — Unmask=true reveals KeyValue.
  • TestCSVFormatter_QuotesCommaInSource"path, with, commas.txt" round-trips verbatim.
  • TestCSVFormatter_VerifiedRow — verified/verify_status columns populated.
  • TestCSVFormatter_MultipleRowsIncrementID — id column increments 0,1,… with slice position.

Commits

Task Phase Hash Message
1 RED c933673 test(06-02): add failing tests for JSONFormatter
1 GREEN 1644771 feat(06-02): implement JSONFormatter with Unmask support
2 RED b35881a test(06-02): add failing tests for CSVFormatter
2 GREEN 03249fb feat(06-02): implement CSVFormatter with Unmask support

Verification

$ go test ./pkg/output/... -count=1
ok  	github.com/salvacybersec/keyhunter/pkg/output	0.005s

$ go build ./...
(no output — clean build)

$ grep -h "Register(" pkg/output/*.go
	Register("csv", CSVFormatter{})
	Register("json", JSONFormatter{})
	Register("sarif", SARIFFormatter{})
	Register("table", TableFormatter{})

All four formatters (table, json, csv, sarif) now live in the registry — --output=json|csv works end to end once the scan command is wired up in a later plan.

Deviations from Plan

Fixed During Execution

1. [Rule 1 - Test bug] Incorrect indent assertion in initial JSON test

  • Found during: Task 1 GREEN run
  • Issue: TestJSONFormatter_Indented asserted "\n \"provider\"" (2-space indent) but json.Encoder.SetIndent("", " ") produces 4-space indent for nested object fields (array item at 2 spaces + 2 more for nested field).
  • Fix: Assertion now verifies both indent levels: "\n {" for array items and "\n \"provider\"" for nested fields.
  • Files modified: pkg/output/json_test.go
  • Commit: 1644771 (rolled into GREEN commit since the RED had not yet been runnable)

Plan body (the JSON encoder behavior) was implemented exactly as specified — only the test expectation was off by one indent level.

Out of Scope / Observed

  • During execution, a pkg/output/sarif_test.go file from the parallel Plan 03 executor was present in the working tree before pkg/output/sarif.go existed, causing the package to fail compilation for unrelated tests. Handled by temporarily moving it aside while running go test -run TestJSONFormatter|TestCSVFormatter, then restoring. By the time the final package-wide test run happened, Plan 03's sarif.go had landed and the full package compiled and tested cleanly. No files were modified as part of this mitigation; this is a parallel-execution artifact and not a deviation from the plan.

Known Stubs

None. JSON and CSV formatters emit real data from engine.Finding and wire through Options.Unmask. No placeholder fields, no TODO markers.

Self-Check: PASSED

  • FOUND: pkg/output/json.go
  • FOUND: pkg/output/json_test.go
  • FOUND: pkg/output/csv.go
  • FOUND: pkg/output/csv_test.go
  • FOUND: c933673 (test JSON RED)
  • FOUND: 1644771 (feat JSON GREEN)
  • FOUND: b35881a (test CSV RED)
  • FOUND: 03249fb (feat CSV GREEN)
  • FOUND: Register("json" in pkg/output/json.go
  • FOUND: Register("csv" in pkg/output/csv.go
  • VERIFIED: go test ./pkg/output/... -count=1 → ok
  • VERIFIED: go build ./... → clean