7.5 KiB
7.5 KiB
phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves
| phase | plan | type | wave | depends_on | files_modified | autonomous | requirements | must_haves | ||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 07-import-cicd | 02 | execute | 1 |
|
true |
|
|
Purpose: Gitleaks is the second major secret scanner; ingesting its output (both JSON and CSV flavors) lets users unify findings (IMP-02). Output: GitleaksImporter, GitleaksCSVImporter, tests, fixtures.
<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>
@.planning/phases/07-import-cicd/07-CONTEXT.md @pkg/engine/finding.go Contract defined by Plan 07-01 in pkg/importer/importer.go: ```go type Importer interface { Name() string Import(r io.Reader) ([]engine.Finding, error) } ``` NOTE: If pkg/importer/importer.go does not yet exist at execution time (waves 1 run in parallel), this plan's executor MUST first create that file with the interface above. The TruffleHog plan (07-01) will reuse the same file. Task 1: Gitleaks JSON + CSV parsers with fixtures pkg/importer/gitleaks.go, pkg/importer/gitleaks_test.go, pkg/importer/testdata/gitleaks-sample.json, pkg/importer/testdata/gitleaks-sample.csv - GitleaksImporter.Import parses a JSON array of records with fields: Description, StartLine, EndLine, StartColumn, EndColumn, Match, Secret, File, SymlinkFile, Commit, Entropy (float), Author, Email, Date, Message, Tags ([]string), RuleID, Fingerprint - Each JSON record maps to engine.Finding: ProviderName = normalizeGitleaksRuleID(RuleID) // "openai-api-key" -> "openai", "aws-access-token" -> "aws", "generic-api-key" -> "generic" KeyValue = Secret KeyMasked = engine.MaskKey(Secret) Confidence = "medium" (Gitleaks doesn't verify) Source = File (fallback SymlinkFile) SourceType = "import:gitleaks" LineNumber = StartLine DetectedAt = time.Now() Verified = false, VerifyStatus = "unverified" - GitleaksCSVImporter.Import reads CSV via encoding/csv. Header row mandatory; column order follows gitleaks default: RuleID,Commit,File,SymlinkFile,Secret,Match,StartLine,EndLine,StartColumn,EndColumn,Author,Message,Date,Email,Fingerprint,Tags. Parse header to index map so column order resilience is not required but header names must match. - normalizeGitleaksRuleID trims common suffixes: "-api-key", "-access-token", "-token", "-secret", "-key". E.g., "openai-api-key" -> "openai", "anthropic-api-key" -> "anthropic", "aws-access-token" -> "aws", "github-pat" -> "github-pat" (no suffix match, kept as-is but lowercased). - Empty array / empty CSV (header only): returns empty slice nil error. - Malformed JSON or CSV: returns wrapped error. - Name() methods return "gitleaks" and "gitleaks-csv" respectively. If pkg/importer/importer.go does not exist yet (parallel execution with 07-01), create it first with the Importer interface (see above).Create pkg/importer/gitleaks.go:
- Package `importer`; imports: encoding/csv, encoding/json, fmt, io, strconv, strings, time, engine pkg.
- Define `type GitleaksImporter struct{}` and `type GitleaksCSVImporter struct{}`.
- Define `type gitleaksRecord struct` with JSON tags matching the Gitleaks schema above.
- Implement `(GitleaksImporter) Name() string` -> "gitleaks"; Import decodes JSON array, loops building engine.Finding.
- Implement `(GitleaksCSVImporter) Name() string` -> "gitleaks-csv"; Import uses csv.NewReader(r), reads header row, builds `map[string]int` of column index, then loops reading records. Parses StartLine via strconv.Atoi; swallows parse errors by setting LineNumber=0.
- Implement `normalizeGitleaksRuleID(id string) string`:
```go
id = strings.ToLower(id)
suffixes := []string{"-api-key", "-access-token", "-secret-key", "-secret", "-token", "-key"}
for _, s := range suffixes {
if strings.HasSuffix(id, s) {
return strings.TrimSuffix(id, s)
}
}
return id
```
- Helper `buildGitleaksFinding(ruleID, secret, file, symlink string, startLine int) engine.Finding` shared between JSON and CSV paths:
- source := file; if source == "" { source = symlink }
- returns engine.Finding{ProviderName: normalizeGitleaksRuleID(ruleID), KeyValue: secret, KeyMasked: engine.MaskKey(secret), Confidence: "medium", Source: source, SourceType: "import:gitleaks", LineNumber: startLine, DetectedAt: time.Now(), VerifyStatus: "unverified"}
Create pkg/importer/testdata/gitleaks-sample.json — JSON array with 3 records covering:
- {"RuleID":"openai-api-key","Secret":"sk-proj-1234567890abcdef1234","File":"config/app.yml","StartLine":12, ...}
- {"RuleID":"aws-access-token","Secret":"AKIAIOSFODNN7EXAMPLE","File":"terraform/main.tf","StartLine":55, ...}
- {"RuleID":"generic-api-key","Secret":"xoxp-abcdefghijklmnopqrstuvwxyz","File":"scripts/deploy.sh","StartLine":3, ...}
Create pkg/importer/testdata/gitleaks-sample.csv with header row and the same 3 rows (in Gitleaks default column order).
Create pkg/importer/gitleaks_test.go:
- TestGitleaksImporter_JSON: loads fixture, expects 3 findings, findings[0].ProviderName=="openai", findings[1].ProviderName=="aws", Source/LineNumber correct.
- TestGitleaksImporter_CSV: loads CSV fixture, same 3 findings, same assertions.
- TestGitleaksImporter_NormalizeRuleID: table — {"openai-api-key","openai"}, {"aws-access-token","aws"}, {"anthropic-api-key","anthropic"}, {"generic-api-key","generic"}, {"github-pat","github-pat"}.
- TestGitleaksImporter_EmptyArray, TestGitleaksImporter_EmptyCSV (header only).
- TestGitleaksImporter_InvalidJSON returns error.
- Name() assertions for both importers.
<success_criteria> Gitleaks output (JSON and CSV) ingests into normalized engine.Finding records with correct provider name mapping and line number extraction. </success_criteria>
After completion, create `.planning/phases/07-import-cicd/07-02-SUMMARY.md`.