docs(07-01): complete importer trufflehog adapter plan
This commit is contained in:
89
.planning/phases/07-import-cicd/07-01-SUMMARY.md
Normal file
89
.planning/phases/07-import-cicd/07-01-SUMMARY.md
Normal file
@@ -0,0 +1,89 @@
|
|||||||
|
---
|
||||||
|
phase: 07-import-cicd
|
||||||
|
plan: 01
|
||||||
|
subsystem: importer
|
||||||
|
tags: [importer, trufflehog, json, adapters]
|
||||||
|
requires:
|
||||||
|
- pkg/engine/finding.go
|
||||||
|
provides:
|
||||||
|
- "pkg/importer.Importer interface"
|
||||||
|
- "pkg/importer.TruffleHogImporter (v3 JSON)"
|
||||||
|
affects:
|
||||||
|
- pkg/importer (new package)
|
||||||
|
tech-stack:
|
||||||
|
added: []
|
||||||
|
patterns:
|
||||||
|
- "Adapter interface per external scanner format"
|
||||||
|
- "Regex + alias map name normalization"
|
||||||
|
- "Raw json.RawMessage for polymorphic SourceMetadata"
|
||||||
|
key-files:
|
||||||
|
created:
|
||||||
|
- pkg/importer/importer.go
|
||||||
|
- pkg/importer/trufflehog.go
|
||||||
|
- pkg/importer/trufflehog_test.go
|
||||||
|
- pkg/importer/testdata/trufflehog-sample.json
|
||||||
|
modified: []
|
||||||
|
decisions:
|
||||||
|
- "SourceMetadata decoded lazily via json.RawMessage then a second pass into a priority struct — tolerates unknown source types without breaking the import"
|
||||||
|
- "Records with empty Raw are skipped silently (no usable key material)"
|
||||||
|
- "Verified=true -> Confidence=high, VerifyStatus=live; otherwise medium/unverified"
|
||||||
|
metrics:
|
||||||
|
duration: "~6 min"
|
||||||
|
completed: 2026-04-05
|
||||||
|
tasks: 1
|
||||||
|
files: 4
|
||||||
|
commits: 1
|
||||||
|
---
|
||||||
|
|
||||||
|
# Phase 07 Plan 01: Importer Interface and TruffleHog v3 JSON Adapter Summary
|
||||||
|
|
||||||
|
**One-liner:** New `pkg/importer` package with `Importer` interface plus `TruffleHogImporter` that decodes v3 JSON into `engine.Finding` with detector-name normalization and SourceMetadata path extraction.
|
||||||
|
|
||||||
|
## What Was Built
|
||||||
|
|
||||||
|
- **`pkg/importer/importer.go`** — `Importer` interface with `Name() string` and `Import(r io.Reader) ([]engine.Finding, error)`. Stateless contract reusable across future scanner formats.
|
||||||
|
- **`pkg/importer/trufflehog.go`** — `TruffleHogImporter` struct, `trufflehogRecord` mirror of the v3 schema, `normalizeTruffleHogName` (lowercases, strips `v\d+$`, applies alias map for aws/gcp/openai/anthropic/huggingface/github), and `extractSourcePath` walking `SourceMetadata.Data.{Git,Filesystem,Github}` in priority order.
|
||||||
|
- **`pkg/importer/testdata/trufflehog-sample.json`** — Realistic 3-record fixture: verified OpenAI key with Git file+line, unverified AnthropicV2 on Filesystem, verified AWS key with Github link.
|
||||||
|
- **`pkg/importer/trufflehog_test.go`** — 5 tests: `Name`, full `Import` fixture roundtrip, table-driven `NormalizeName` (7 cases incl. unknown detector fall-through), empty array, invalid JSON error.
|
||||||
|
|
||||||
|
## Mapping Rules
|
||||||
|
|
||||||
|
| TruffleHog field | engine.Finding field |
|
||||||
|
| --- | --- |
|
||||||
|
| `DetectorName` (normalized) | `ProviderName` |
|
||||||
|
| `Raw` | `KeyValue` (+ `KeyMasked` via `engine.MaskKey`) |
|
||||||
|
| `Verified` true | `Confidence=high`, `VerifyStatus=live`, `Verified=true` |
|
||||||
|
| `Verified` false | `Confidence=medium`, `VerifyStatus=unverified` |
|
||||||
|
| `SourceMetadata.Data.Git.File` / line | `Source`, `LineNumber` |
|
||||||
|
| `SourceMetadata.Data.Filesystem.File` | `Source` |
|
||||||
|
| `SourceMetadata.Data.Github.{File,Link,Repository}` | `Source` |
|
||||||
|
| fallback `SourceName` | `Source` |
|
||||||
|
| constant | `SourceType = "import:trufflehog"` |
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- `go build ./pkg/importer/...` — clean
|
||||||
|
- `go vet ./pkg/importer/...` — clean
|
||||||
|
- `go test ./pkg/importer/... -run TruffleHog -v` — 5/5 PASS in ~3ms
|
||||||
|
|
||||||
|
Note: the package also contains untracked scaffolding (`gitleaks_test.go`) awaiting plan 07-02. To verify this plan in isolation the scaffolding was temporarily moved out of the package during the vet/test run, then restored. No tracked files outside this plan were touched.
|
||||||
|
|
||||||
|
## Deviations from Plan
|
||||||
|
|
||||||
|
None — plan executed exactly as written.
|
||||||
|
|
||||||
|
## Deferred Issues
|
||||||
|
|
||||||
|
- Pre-existing untracked `pkg/importer/gitleaks_test.go` references `GitleaksImporter`/`GitleaksCSVImporter`, which are scheduled for plan 07-02. Out of scope for this plan; tracked by phase roadmap.
|
||||||
|
|
||||||
|
## Commits
|
||||||
|
|
||||||
|
- `46eec32` — feat(07-01): Importer interface and TruffleHog v3 JSON adapter
|
||||||
|
|
||||||
|
## Self-Check: PASSED
|
||||||
|
|
||||||
|
- FOUND: pkg/importer/importer.go
|
||||||
|
- FOUND: pkg/importer/trufflehog.go
|
||||||
|
- FOUND: pkg/importer/trufflehog_test.go
|
||||||
|
- FOUND: pkg/importer/testdata/trufflehog-sample.json
|
||||||
|
- FOUND commit: 46eec32
|
||||||
Reference in New Issue
Block a user