docs(07-01): complete importer trufflehog adapter plan

This commit is contained in:
salvacybersec
2026-04-05 23:55:58 +03:00
parent bd8eb9b611
commit 75becce3dd

View File

@@ -0,0 +1,89 @@
---
phase: 07-import-cicd
plan: 01
subsystem: importer
tags: [importer, trufflehog, json, adapters]
requires:
- pkg/engine/finding.go
provides:
- "pkg/importer.Importer interface"
- "pkg/importer.TruffleHogImporter (v3 JSON)"
affects:
- pkg/importer (new package)
tech-stack:
added: []
patterns:
- "Adapter interface per external scanner format"
- "Regex + alias map name normalization"
- "Raw json.RawMessage for polymorphic SourceMetadata"
key-files:
created:
- pkg/importer/importer.go
- pkg/importer/trufflehog.go
- pkg/importer/trufflehog_test.go
- pkg/importer/testdata/trufflehog-sample.json
modified: []
decisions:
- "SourceMetadata decoded lazily via json.RawMessage then a second pass into a priority struct — tolerates unknown source types without breaking the import"
- "Records with empty Raw are skipped silently (no usable key material)"
- "Verified=true -> Confidence=high, VerifyStatus=live; otherwise medium/unverified"
metrics:
duration: "~6 min"
completed: 2026-04-05
tasks: 1
files: 4
commits: 1
---
# Phase 07 Plan 01: Importer Interface and TruffleHog v3 JSON Adapter Summary
**One-liner:** New `pkg/importer` package with `Importer` interface plus `TruffleHogImporter` that decodes v3 JSON into `engine.Finding` with detector-name normalization and SourceMetadata path extraction.
## What Was Built
- **`pkg/importer/importer.go`** — `Importer` interface with `Name() string` and `Import(r io.Reader) ([]engine.Finding, error)`. Stateless contract reusable across future scanner formats.
- **`pkg/importer/trufflehog.go`** — `TruffleHogImporter` struct, `trufflehogRecord` mirror of the v3 schema, `normalizeTruffleHogName` (lowercases, strips `v\d+$`, applies alias map for aws/gcp/openai/anthropic/huggingface/github), and `extractSourcePath` walking `SourceMetadata.Data.{Git,Filesystem,Github}` in priority order.
- **`pkg/importer/testdata/trufflehog-sample.json`** — Realistic 3-record fixture: verified OpenAI key with Git file+line, unverified AnthropicV2 on Filesystem, verified AWS key with Github link.
- **`pkg/importer/trufflehog_test.go`** — 5 tests: `Name`, full `Import` fixture roundtrip, table-driven `NormalizeName` (7 cases incl. unknown detector fall-through), empty array, invalid JSON error.
## Mapping Rules
| TruffleHog field | engine.Finding field |
| --- | --- |
| `DetectorName` (normalized) | `ProviderName` |
| `Raw` | `KeyValue` (+ `KeyMasked` via `engine.MaskKey`) |
| `Verified` true | `Confidence=high`, `VerifyStatus=live`, `Verified=true` |
| `Verified` false | `Confidence=medium`, `VerifyStatus=unverified` |
| `SourceMetadata.Data.Git.File` / line | `Source`, `LineNumber` |
| `SourceMetadata.Data.Filesystem.File` | `Source` |
| `SourceMetadata.Data.Github.{File,Link,Repository}` | `Source` |
| fallback `SourceName` | `Source` |
| constant | `SourceType = "import:trufflehog"` |
## Verification
- `go build ./pkg/importer/...` — clean
- `go vet ./pkg/importer/...` — clean
- `go test ./pkg/importer/... -run TruffleHog -v` — 5/5 PASS in ~3ms
Note: the package also contains untracked scaffolding (`gitleaks_test.go`) awaiting plan 07-02. To verify this plan in isolation the scaffolding was temporarily moved out of the package during the vet/test run, then restored. No tracked files outside this plan were touched.
## Deviations from Plan
None — plan executed exactly as written.
## Deferred Issues
- Pre-existing untracked `pkg/importer/gitleaks_test.go` references `GitleaksImporter`/`GitleaksCSVImporter`, which are scheduled for plan 07-02. Out of scope for this plan; tracked by phase roadmap.
## Commits
- `46eec32` — feat(07-01): Importer interface and TruffleHog v3 JSON adapter
## Self-Check: PASSED
- FOUND: pkg/importer/importer.go
- FOUND: pkg/importer/trufflehog.go
- FOUND: pkg/importer/trufflehog_test.go
- FOUND: pkg/importer/testdata/trufflehog-sample.json
- FOUND commit: 46eec32