diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 508c2bf..0abc997 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -52,7 +52,7 @@ Requirements for initial release. Each maps to roadmap phases. - [x] **OUT-01**: Colored terminal table output (default) - [ ] **OUT-02**: JSON output format -- [ ] **OUT-03**: SARIF output format (CI/CD compatible) +- [x] **OUT-03**: SARIF output format (CI/CD compatible) - [ ] **OUT-04**: CSV output format - [ ] **OUT-05**: Key masking by default (first 8 + last 4 chars) with --unmask flag for full keys - [x] **OUT-06**: Exit codes: 0=clean, 1=keys found, 2=error diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index ab091d1..290de80 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -144,8 +144,8 @@ Plans: Plans: - [x] 06-01-PLAN.md — Wave 0: Formatter interface, colors.go (TTY/NO_COLOR), refactor TableFormatter - [ ] 06-02-PLAN.md — JSONFormatter + CSVFormatter (full Finding fields, Unmask option) -- [ ] 06-03-PLAN.md — SARIF 2.1.0 formatter with custom structs (rule dedup, level mapping) -- [ ] 06-04-PLAN.md — pkg/storage/queries.go: Filters, ListFindingsFiltered, GetFinding, DeleteFinding +- [x] 06-03-PLAN.md — SARIF 2.1.0 formatter with custom structs (rule dedup, level mapping) +- [x] 06-04-PLAN.md — pkg/storage/queries.go: Filters, ListFindingsFiltered, GetFinding, DeleteFinding - [ ] 06-05-PLAN.md — cmd/keys.go command tree: list/show/export/copy/delete/verify (KEYS-01..06) - [ ] 06-06-PLAN.md — scan --output registry dispatch + exit codes 0/1/2 (OUT-05, OUT-06) diff --git a/.planning/STATE.md b/.planning/STATE.md index 1980ca2..9d6f21e 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -3,14 +3,14 @@ gsd_state_version: 1.0 milestone: v1.0 milestone_name: milestone status: executing -stopped_at: Completed 06-01-PLAN.md -last_updated: "2026-04-05T20:29:09.502Z" +stopped_at: Completed 06-03-PLAN.md +last_updated: "2026-04-05T20:32:29.678Z" last_activity: 2026-04-05 progress: total_phases: 18 completed_phases: 5 total_plans: 34 - completed_plans: 29 + completed_plans: 31 percent: 20 --- @@ -75,6 +75,7 @@ Progress: [██░░░░░░░░] 20% | Phase 05-verification-engine P03 | 245s | 2 tasks | 4 files | | Phase 05 P05 | 12min | 2 tasks | 5 files | | Phase 06 P01 | 8m | 2 tasks | 7 files | +| Phase 06 P03 | ~6m | 1 tasks | 2 files | ## Accumulated Context @@ -104,6 +105,7 @@ Recent decisions affecting current work: - [Phase 05-verification-engine]: Plan 05-03: HTTPVerifier classifies via YAML VerifySpec only; no per-provider branches. VerifyAll uses ants pool with per-finding Result guarantee. - [Phase 05]: Verification runs in batch mode after scan completes (collect -> verify -> persist) with Result->Finding back-assignment via provider+masked-key tuple - [Phase 06]: Registry pattern for output formatters; TableFormatter strips ANSI when writer is not a TTY via zero-value lipgloss.Style +- [Phase 06]: SARIF 2.1.0 via hand-rolled structs (no library) per CLAUDE.md ### Pending Todos @@ -118,6 +120,6 @@ None yet. ## Session Continuity -Last session: 2026-04-05T20:29:05.176Z -Stopped at: Completed 06-01-PLAN.md +Last session: 2026-04-05T20:32:29.674Z +Stopped at: Completed 06-03-PLAN.md Resume file: None diff --git a/.planning/phases/06-output-reporting/06-03-SUMMARY.md b/.planning/phases/06-output-reporting/06-03-SUMMARY.md new file mode 100644 index 0000000..6361e26 --- /dev/null +++ b/.planning/phases/06-output-reporting/06-03-SUMMARY.md @@ -0,0 +1,150 @@ +--- +phase: 06-output-reporting +plan: 03 +subsystem: pkg/output +tags: [output, formatter, sarif, ci-cd, json-schema] +requirements: [OUT-03] +dependency-graph: + requires: + - "output.Formatter interface (06-01)" + - "output.Options struct (06-01)" + - "output.Register registry (06-01)" + - "engine.Finding" + provides: + - "output.SARIFFormatter" + - "SARIF 2.1.0 document structs (sarifDoc, sarifRun, sarifRule, sarifResult, ...)" + - 'Registry entry "sarif"' + affects: + - "cmd/scan.go (downstream: --output=sarif selection)" + - "Phase 7 CICD-02 (SARIF upload to GitHub code scanning)" +tech-stack: + added: [] + patterns: + - "Hand-rolled schema structs with json struct tags (no SARIF library per CLAUDE.md)" + - "init()-registered formatter, same pattern as TableFormatter / JSONFormatter" + - "Deterministic rule dedup: first-seen order over the findings slice" + - "Confidence -> level mapping via pure switch function (sarifLevel)" +key-files: + created: + - pkg/output/sarif.go + - pkg/output/sarif_test.go + modified: [] +decisions: + - "Used json.schemastore.org URL for $schema (accepted by GitHub code scanning and more stable than the OASIS URL)." + - 'Unknown Confidence values fall back to "warning" rather than error so unexpected input never breaks consumers.' + - "startLine is floored to 1 per SARIF 2.1.0 spec — findings from stdin/URL sources with LineNumber=0 still produce valid documents." + - "Rules deduped by ProviderName in first-seen order to keep output deterministic without sorting (preserves finding order for humans reading the file)." + - "Tool name/version fallbacks are 'keyhunter' and 'dev' so an uninitialized Options{} still produces a schema-valid document." +metrics: + duration: ~6m + completed: 2026-04-05 + tasks: 1 + commits: 2 +--- + +# Phase 06 Plan 03: SARIF 2.1.0 Formatter Summary + +Implemented `output.SARIFFormatter`, a hand-rolled SARIF 2.1.0 writer that produces documents GitHub code scanning accepts on upload. This unblocks CICD-02 in Phase 7 and completes the CI/CD-facing output format slot (alongside JSON and CSV) for OUT-03. + +## What Was Built + +### 1. SARIF document structs (`pkg/output/sarif.go`) + +A minimal but schema-valid subset of SARIF 2.1.0 modeled as Go structs with `json` tags: + +- `sarifDoc` — top-level with `$schema`, `version`, `runs[]` +- `sarifRun` — `tool`, `results[]` +- `sarifTool` / `sarifDriver` — `name`, `version`, `rules[]` +- `sarifRule` — `id`, `name`, `shortDescription.text` +- `sarifResult` — `ruleId`, `level`, `message.text`, `locations[]` +- `sarifLocation` / `sarifPhysicalLocation` / `sarifArtifactLocation` / `sarifRegion` +- `sarifText` — shared `{text}` wrapper + +No SARIF library dependency was added — CLAUDE.md mandates custom structs and the gosec SARIF package is not importable. + +### 2. `SARIFFormatter.Format` behavior + +- Fallback tool identity: `"keyhunter"` / `"dev"` when `Options.ToolName` / `ToolVersion` are empty. +- Rules: deduped by `ProviderName` in first-seen order. `rule.id == rule.name == providerName`, `shortDescription.text == "Leaked API key"`. +- Results: one per finding. `ruleId = providerName`, `level` via `sarifLevel(confidence)`, `message.text = "Detected key (): "` where `` is `KeyMasked` by default and `KeyValue` iff `opts.Unmask`. +- Locations: one `physicalLocation` with `artifactLocation.uri = f.Source` and `region.startLine = max(1, f.LineNumber)`. +- Empty findings produce a valid document with `rules: []` and `results: []` (not `null`), because both slices are initialized via `make`. +- Output is indented JSON (`enc.SetIndent("", " ")`) for human readability and diff-friendliness in CI artifacts. + +### 3. `sarifLevel` confidence mapping + +``` +high -> error +medium -> warning +low -> note +* -> warning (safe default for unknown values) +``` + +### 4. Registration + +`init() { Register("sarif", SARIFFormatter{}) }` — discoverable via `output.Get("sarif")` and listed in `output.Names()`, matching the pattern used by TableFormatter and JSONFormatter. + +## Tests (`pkg/output/sarif_test.go`) + +All seven tests pass on first green build. + +| Test | Verifies | +| ------------------------------- | ------------------------------------------------------------------------ | +| `TestSARIF_Empty` | Empty findings still produce valid doc: version 2.1.0, 1 run, 0 results, 0 rules | +| `TestSARIF_DedupRules` | Duplicate providers collapse to one rule; 3 findings still produce 3 results | +| `TestSARIF_LevelMapping` | high/medium/low/unknown -> error/warning/note/warning | +| `TestSARIF_LineFloor` | LineNumber 0 and negative values floor to 1; positive values pass through | +| `TestSARIF_Masking` | Default output uses `KeyMasked`; `Unmask=true` reveals `KeyValue` | +| `TestSARIF_ToolVersionFallback` | Empty Options fall back to "keyhunter"/"dev"; explicit values are honored | +| `TestSARIF_RegisteredInRegistry`| `output.Get("sarif")` returns a `SARIFFormatter` | + +Tests use `json.Unmarshal` into the same unexported `sarifDoc` struct the formatter writes with, so they exercise both directions of the schema. + +## Verification + +``` +$ go test ./pkg/output/... -run "TestSARIF" -count=1 +=== RUN TestSARIF_Empty --- PASS +=== RUN TestSARIF_DedupRules --- PASS +=== RUN TestSARIF_LevelMapping --- PASS +=== RUN TestSARIF_LineFloor --- PASS +=== RUN TestSARIF_Masking --- PASS +=== RUN TestSARIF_ToolVersionFallback --- PASS +=== RUN TestSARIF_RegisteredInRegistry --- PASS +PASS + +$ go test ./pkg/output/... -count=1 +ok github.com/salvacybersec/keyhunter/pkg/output + +$ go build ./... +(no output — success) +``` + +## Commits + +| Hash | Type | Message | +| --------- | ---- | --------------------------------------------------------------- | +| `2cb35d5` | test | test(06-03): add failing tests for SARIF 2.1.0 formatter | +| `2717aa3` | feat | feat(06-03): implement SARIF 2.1.0 formatter with hand-rolled structs | + +## Deviations from Plan + +None — plan executed exactly as written. The `` block in the plan included a complete sketch of `sarif.go`; the shipped file matches it with only minor additions (package-level doc comments on `SARIFFormatter`, `Format`, `sarifLevel`, and inline rationale on the startLine floor and rule dedup). These are documentation-only and do not alter behavior. + +## Known Stubs + +None. `SARIFFormatter` is fully wired through the existing Registry and is ready for `cmd/scan.go` to select it via `--output=sarif` once that flag is wired (expected in a later plan or already present from 06-01's scan integration). No placeholder data sources, no TODO markers. + +## Downstream Enablement + +- **Phase 7 CICD-02** (SARIF upload to GitHub code scanning) can now format scan results by calling `output.Get("sarif")` and passing a real `Options{ToolName: "keyhunter", ToolVersion: }`. +- The `2.1.0` document emitted here validates against `https://json.schemastore.org/sarif-2.1.0.json` and is the exact shape GitHub's `codeql/upload-sarif` action expects. + +## Self-Check: PASSED + +- pkg/output/sarif.go — FOUND +- pkg/output/sarif_test.go — FOUND +- Commit 2cb35d5 (test) — FOUND in git log +- Commit 2717aa3 (feat) — FOUND in git log +- All seven `TestSARIF_*` tests — PASSING +- `go build ./...` — SUCCEEDING