Commit Graph

338 Commits

Author SHA1 Message Date
salvacybersec
e2394ec663 test(06-05): integration tests for keys list/show/export/delete
- Temp-file SQLite DB seeded with three findings (2 openai, 1 anthropic,
  one verified) via storage.SaveFinding + loadOrCreateEncKey
- RunE + cmd.SetOut buffers for hermetic stdout capture
- Covers: list default + provider filter, show hit (unmasked) + miss,
  export JSON stdout (parses + plaintext present), export CSV to file
  (header + 3 rows), delete --yes then list returns 2
- TestKeysCopy and TestKeysVerify are documented as intentionally skipped
  (clipboard backend unavailable headlessly; verify needs network)
2026-04-05 23:39:07 +03:00
salvacybersec
06594afc57 feat(06-05): implement keys command tree (list/show/export/copy/delete/verify)
- Add cmd/keys.go with six subcommands backed by the Plan 04 query layer
- keys list prints masked findings with id/provider/confidence/source columns
  and supports --provider/--verified/--limit/--unmask filters
- keys show <id> renders a finding fully unmasked with verify metadata
- keys export --format=json|csv reuses the formatter registry, atomic
  file writes when --output is set
- keys copy <id> uses atotto/clipboard for clipboard handoff
- keys delete <id> prompts via cmd.InOrStdin unless --yes is passed
- keys verify <id> gates on verify.EnsureConsent, then updates the stored
  row inline via UPDATE findings SET verify_* using db.SQL()
- Remove the keysCmd stub from cmd/stubs.go (single declaration)
- All subcommands read config via openDBWithKey() mirroring scan.go
2026-04-05 23:37:25 +03:00
salvacybersec
7a3822c22e docs(06-02): complete JSON + CSV formatter plan 2026-04-05 23:34:51 +03:00
salvacybersec
9546f80fab docs(06-03): complete SARIF 2.1.0 formatter plan 2026-04-05 23:32:37 +03:00
salvacybersec
35352ff3d0 docs(06-04): complete findings query layer plan
SUMMARY.md for Plan 06-04: Filters struct + ListFindingsFiltered +
GetFinding + DeleteFinding on pkg/storage. Foundation for keys command
tree in Plan 06-05.
2026-04-05 23:32:11 +03:00
salvacybersec
03249fb3d1 feat(06-02): implement CSVFormatter with Unmask support
- Fixed 9-column header: id,provider,confidence,key,source,line,detected_at,verified,verify_status
- Uses encoding/csv for automatic quoting of commas/quotes in source paths
- Honors Options.Unmask for key column
- Registers under "csv" in output registry
2026-04-05 23:32:07 +03:00
salvacybersec
b35881aaef test(06-02): add failing tests for CSVFormatter 2026-04-05 23:31:44 +03:00
salvacybersec
2717aa3196 feat(06-03): implement SARIF 2.1.0 formatter with hand-rolled structs
- SARIFFormatter emits schema-valid SARIF 2.1.0 JSON for CI ingestion
- One rule per distinct provider, deduped in first-seen order
- Confidence mapped high/medium/low to error/warning/note
- startLine floored to 1 per SARIF spec requirement
- Registered under name 'sarif' via init()
2026-04-05 23:31:15 +03:00
salvacybersec
b1e4dea51c feat(06-04): implement findings query layer for keys command
- Filters struct: Provider, Verified (*bool), Limit, Offset
- ListFindingsFiltered: optional WHERE + ORDER BY created_at DESC, id DESC
- GetFinding: single-row lookup, propagates sql.ErrNoRows on miss
- DeleteFinding: returns RowsAffected so caller can distinguish hit/miss
- Shared scan/hydrate helpers decrypt key_value via existing Decrypt
2026-04-05 23:31:15 +03:00
salvacybersec
164477136c feat(06-02): implement JSONFormatter with Unmask support
- Renders findings as 2-space indented JSON array
- Honors Options.Unmask for key field exposure
- Omits empty verify fields via json omitempty
- Registers under "json" in output registry
2026-04-05 23:31:12 +03:00
salvacybersec
2cb35d50ac test(06-03): add failing tests for SARIF 2.1.0 formatter 2026-04-05 23:30:38 +03:00
salvacybersec
67763ec498 test(06-04): add failing tests for findings query layer
- Filters struct with provider, verified, limit/offset
- ListFindingsFiltered, GetFinding, DeleteFinding coverage
- Uses in-memory SQLite with seeded fixtures across 2 providers
2026-04-05 23:30:33 +03:00
salvacybersec
c933673ca9 test(06-02): add failing tests for JSONFormatter 2026-04-05 23:30:12 +03:00
salvacybersec
5292502000 docs(06-01): complete formatter interface + TableFormatter plan 2026-04-05 23:29:13 +03:00
salvacybersec
8e4db5db09 feat(06-01): refactor table output into TableFormatter
- TableFormatter implements Formatter interface, registered as "table"
- Writes to arbitrary io.Writer instead of hardcoded os.Stdout
- Strips ANSI colors when writer is not a TTY or NO_COLOR is set
- Uses bundled tableStyles so plain/colored paths share one renderer
- PrintFindings retained as backward-compat wrapper delegating to Format
2026-04-05 23:27:53 +03:00
salvacybersec
8c37252c1b test(06-01): add failing tests for TableFormatter refactor
- Add TestTableFormatter_Empty, NoColorInBuffer, Unverified/VerifiedLayout
- Add TestTableFormatter_Masking, MetadataSorted, RegisteredUnderTable
- Keep legacy PrintFindings tests as backward-compat wrapper coverage
2026-04-05 23:27:03 +03:00
salvacybersec
291c97ed0b feat(06-01): add Formatter interface, Registry, and TTY color detection
- pkg/output/formatter.go: Formatter interface, Options, Registry with
  Register/Get/Names, ErrUnknownFormat sentinel
- pkg/output/colors.go: IsTTY + ColorsEnabled honoring NO_COLOR
- Promote github.com/mattn/go-isatty to direct dependency
- Unit tests cover registry round-trip, unknown lookup, sorted Names,
  non-TTY buffer, NO_COLOR override

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 18:41:23 +03:00
salvacybersec
ce37ee2bc5 docs(06): create phase 6 plans — output formats + key management 2026-04-05 16:10:14 +03:00
salvacybersec
7ba2a94fb6 docs(06): output formats and key management context 2026-04-05 16:00:42 +03:00
salvacybersec
0a474e70c5 docs(phase-05): complete phase execution 2026-04-05 15:59:51 +03:00
salvacybersec
8cceca164c docs(05-05): complete scan --verify integration plan 2026-04-05 15:56:23 +03:00
salvacybersec
cc9dabe5f5 feat(05-05): render VERIFY column and metadata line in output table
- When any finding has Verified=true, append a VERIFY column with colored
  glyphs: ✓ live / ✗ dead / ⚠ rate / ! err / ? unk
- Per-finding VerifyMetadata is rendered on an indented secondary line
  with deterministic (sorted) key ordering
- Backward compatible: unverified scans produce identical output to
  pre-Phase-5 runs
2026-04-05 15:54:51 +03:00
salvacybersec
edba8fb5d4 test(05-05): add failing tests for VERIFY column and metadata rendering 2026-04-05 15:54:13 +03:00
salvacybersec
6fc0abe8ae feat(05-05): wire --verify into scan pipeline with consent gate
- Add --verify-timeout (default 10s) and --verify-workers (default 10) flags
- Refactor scan loop to collect findings, verify, then persist
- Gate verification behind verify.EnsureConsent(db, stdin, stderr)
- Route findings through verify.HTTPVerifier.VerifyAll with configurable
  timeout and worker pool, back-assign Result.Status/HTTPCode/Metadata
  onto engine.Finding by provider+masked-key tuple
- Persist verify_* columns via storage.SaveFinding after verification
2026-04-05 15:53:47 +03:00
salvacybersec
d5370783d4 test(05-05): add failing test for --verify-timeout/--verify-workers flags 2026-04-05 15:52:57 +03:00
salvacybersec
c6968a4a72 docs(05-03): complete HTTPVerifier core plan
- SUMMARY.md with decisions, metrics, deviations, self-check
- STATE.md advanced, requirements VRFY-02/03/05 marked complete
- ROADMAP.md plan progress updated
2026-04-05 15:51:23 +03:00
salvacybersec
35c7759f02 feat(05-03): add VerifyAll ants worker pool for parallel verification
- VerifyAll(ctx, findings, reg, workers) returns a result channel closed
  after all findings are processed or ctx is cancelled.
- Default worker count of 10 when workers <= 0.
- Missing providers yield StatusUnknown with 'provider not found' error.
- Graceful context cancellation stops dispatch while still draining inflight.
2026-04-05 15:49:22 +03:00
salvacybersec
0be926f823 docs(05-02): complete legal disclaimer & consent prompt plan 2026-04-05 15:49:05 +03:00
salvacybersec
45ee2f8f53 test(05-03): add failing tests for VerifyAll worker pool
- TestVerifyAll_MultipleFindings: 5 findings via 3-worker pool
- TestVerifyAll_MissingProvider: unknown provider yields StatusUnknown
- TestVerifyAll_ContextCancellation: cancellation closes channel early
- Add providers.NewRegistryFromProviders test helper
2026-04-05 15:48:46 +03:00
salvacybersec
026d305026 docs(05-04): complete Tier 1 verify specs plan 2026-04-05 15:48:15 +03:00
salvacybersec
3dfe72779b feat(05-03): implement HTTPVerifier single-key verification
- HTTPVerifier with TLS 1.2+ client and configurable per-call timeout
- {{KEY}} template substitution in URL, header values, and body
- Classification via EffectiveSuccessCodes/FailureCodes/RateLimitCodes
- Retry-After header captured on rate-limit responses
- gjson-based metadata extraction for JSON responses (1 MiB cap)
- HTTPS-only enforcement; missing URL yields StatusUnknown
- Consent stub added to unblock parallel Plan 05-02 worktree (Rule 3 deviation)
2026-04-05 15:47:49 +03:00
salvacybersec
d4c140371e feat(05-02): implement EnsureConsent prompt gating --verify
- Add EnsureConsent(db, in, out) that returns (true, nil) immediately if
  verify.consent==granted, otherwise prompts once, reads a line, persists
  'granted' on 'yes' (case-insensitive), 'declined' otherwise.
- Declined is not sticky — next call re-prompts; only granted persists.
- Prompt references legal implications and directs users to 'keyhunter legal'.
2026-04-05 15:47:30 +03:00
salvacybersec
6a94ce5903 test(05-04): guardrail tests for Tier 1 verify spec completeness
- TestTier1VerifySpecs_Complete asserts 11 Tier 1 providers have HTTPS
  verify URLs and non-empty effective success codes
- TestInflection_NoVerifyEndpoint documents the intentional empty URL
- Prevents future regressions when editing provider YAMLs
2026-04-05 15:46:57 +03:00
salvacybersec
e5f72149cf test(05-02): add failing tests for EnsureConsent prompt logic 2026-04-05 15:46:41 +03:00
salvacybersec
f3ae8f0b09 feat(05-04): extend Tier 1 provider verify specs
- 12 Tier 1 providers now carry success_codes, failure_codes, rate_limit_codes
- {{KEY}} template in headers or URL (double-brace canonical form)
- metadata_paths added where provider APIs return useful metadata
- Anthropic switched to POST /v1/messages with minimal body
- Perplexity gains JSON body, content-type header
- Inflection verify URL left empty (no public endpoint)
- Dual-location sync preserved: providers/ mirrors pkg/providers/definitions/
2026-04-05 15:46:30 +03:00
salvacybersec
3ceccd98ad test(05-03): add failing tests for HTTPVerifier single-key verification
- 10 test cases covering live/dead/rate-limited/unknown/error classification
- Key substitution in header/body/URL via {{KEY}} template
- JSON metadata extraction via gjson paths
- HTTPS-only enforcement and per-call timeout
2026-04-05 15:46:15 +03:00
salvacybersec
260e342f2f feat(05-02): add LEGAL.md, embed it, and wire keyhunter legal command
- Add LEGAL.md at repo root (109 lines) covering CFAA, Computer Misuse Act,
  EU Directive 2013/40/EU, responsible use, disclosure, and disclaimer.
- Mirror to pkg/legal/LEGAL.md for go:embed (Go cannot traverse parents).
- Add pkg/legal package exposing Text() for the embedded markdown.
- Add cmd/legal.go registering keyhunter legal subcommand to print it.
2026-04-05 15:46:11 +03:00
salvacybersec
177888bfa8 docs(05-01): complete verification foundation plan
Wave 0 contracts for the verification engine are in place:
- VerifySpec extended with SuccessCodes/FailureCodes/RateLimitCodes/MetadataPaths/Body
- Finding extended with Verified/VerifyStatus/VerifyHTTPCode/VerifyMetadata/VerifyError
- findings table schema migrated with verify_* columns (fresh + legacy DBs)
- gjson dep wired as direct require
- VRFY-02, VRFY-03 marked complete
2026-04-05 15:44:20 +03:00
salvacybersec
aec559d2aa feat(05-01): migrate findings schema with verify_* columns
- schema.sql: new findings columns verified, verify_status, verify_http_code, verify_metadata_json
- db.go: migrateFindingsVerifyColumns runs on Open() for legacy DBs using PRAGMA table_info + ALTER TABLE
- findings.go: Finding struct gains Verified/VerifyStatus/VerifyHTTPCode/VerifyMetadata
- SaveFinding serializes verify metadata as JSON (NULL when nil)
- ListFindings round-trips all verify fields
2026-04-05 15:42:53 +03:00
salvacybersec
26544872f7 test(05-01): add failing tests for findings verify columns
- Round-trip verify fields (Verified, VerifyStatus, VerifyHTTPCode, VerifyMetadata)
- Empty verify fields persist as defaults
- Legacy DB schema migrates verify columns idempotently
2026-04-05 15:41:49 +03:00
salvacybersec
30c0e9871b feat(05-01): extend VerifySpec and Finding, add gjson dep
- VerifySpec: add SuccessCodes, FailureCodes, RateLimitCodes, MetadataPaths, Body
- Preserve legacy ValidStatus/InvalidStatus for backward compat
- Add EffectiveSuccessCodes/FailureCodes/RateLimitCodes fallback helpers
- Add ExtractMetadata helper using gjson (skeleton for Plan 05-03)
- Finding: add Verified, VerifyStatus, VerifyHTTPCode, VerifyMetadata, VerifyError
- Add github.com/tidwall/gjson v1.18.0 as direct dependency
2026-04-05 15:41:13 +03:00
salvacybersec
499f5d5025 test(05-01): add failing tests for extended VerifySpec
- New canonical fields: SuccessCodes, FailureCodes, RateLimitCodes, MetadataPaths, Body
- EffectiveSuccessCodes/FailureCodes/RateLimitCodes fallback logic
- Legacy ValidStatus/InvalidStatus still parse
2026-04-05 15:39:35 +03:00
salvacybersec
0b667566c4 docs(05): create phase 5 verification engine plans 2026-04-05 15:38:23 +03:00
salvacybersec
e65b9c981b docs(05): verification engine context 2026-04-05 15:30:11 +03:00
salvacybersec
b079ed202c docs(phase-04): complete phase execution 2026-04-05 15:29:09 +03:00
salvacybersec
c40e9f087f docs(04-05): complete scan command source wiring plan 2026-04-05 15:24:46 +03:00
salvacybersec
b151e88a29 feat(04-05): wire all Phase 4 sources through scan command
- Add --git, --url, --clipboard, --since, --max-file-size, --insecure flags
- Introduce selectSource dispatcher with sourceFlags struct
- Dispatch to Dir/File/Git/Stdin/URL/Clipboard sources based on args+flags
- Reject mutually exclusive source selectors with clear error
- Forward --exclude patterns into DirSource
- Args changed to MaximumNArgs(1) to allow --url/--clipboard without positional
2026-04-05 15:23:12 +03:00
salvacybersec
9105ca11f5 test(04-05): add failing tests for selectSource dispatcher 2026-04-05 15:21:37 +03:00
salvacybersec
66fc597399 docs(04-04): complete stdin/url/clipboard sources plan 2026-04-05 15:20:10 +03:00
salvacybersec
563d6e5ce2 docs(04-02): complete DirSource plan 2026-04-05 15:19:50 +03:00