salvacybersec/keyhunter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(01-foundation): create phase 1 plan — 5 plans across 3 execution waves

Browse Source 
Wave 0: module init + test scaffolding (01-01)
Wave 1: provider registry (01-02) + storage layer (01-03) in parallel
Wave 2: scan engine pipeline (01-04, depends on 01-02)
Wave 3: CLI wiring + integration checkpoint (01-05, depends on all)

Covers all 16 Phase 1 requirements: CORE-01 through CORE-07, STOR-01 through STOR-03,
CLI-01 through CLI-05, PROV-10.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
salvacybersec
2026-04-04 23:44:09 +03:00
parent c573b97a68
commit 684b67cb73
6 changed files with 3095 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

634
.planning/phases/01-foundation/01-03-PLAN.md Normal file
View File

@@ -0,0 +1,634 @@
---
phase: 01-foundation
plan: 03
type: execute
wave: 1
depends_on: [01-01]
files_modified:
- pkg/storage/schema.sql
- pkg/storage/encrypt.go
- pkg/storage/crypto.go
- pkg/storage/db.go
- pkg/storage/findings.go
- pkg/storage/db_test.go
autonomous: true
requirements: [STOR-01, STOR-02, STOR-03]
must_haves:
truths:
- "SQLite database opens, runs migrations from embedded schema.sql, and closes cleanly"
- "AES-256-GCM Encrypt/Decrypt roundtrip produces the original plaintext"
- "Argon2id DeriveKey with the same passphrase and salt always returns the same 32-byte key"
- "A Finding can be saved to the database with the key_value stored encrypted and retrieved as plaintext"
- "The raw database file does NOT contain plaintext API key values"
artifacts:
- path: "pkg/storage/encrypt.go"
provides: "Encrypt(plaintext, key) and Decrypt(ciphertext, key) using AES-256-GCM"
exports: ["Encrypt", "Decrypt"]
- path: "pkg/storage/crypto.go"
provides: "DeriveKey(passphrase, salt) using Argon2id RFC 9106 params"
exports: ["DeriveKey", "NewSalt"]
- path: "pkg/storage/db.go"
provides: "DB struct with Open(), Close(), WAL mode, embedded schema migration"
exports: ["DB", "Open"]
- path: "pkg/storage/findings.go"
provides: "SaveFinding(finding, encKey) and ListFindings(encKey) CRUD"
exports: ["SaveFinding", "ListFindings", "Finding"]
- path: "pkg/storage/schema.sql"
provides: "CREATE TABLE statements for findings, scans, settings"
contains: "CREATE TABLE IF NOT EXISTS findings"
key_links:
- from: "pkg/storage/findings.go"
to: "pkg/storage/encrypt.go"
via: "Encrypt() called before INSERT, Decrypt() called after SELECT"
pattern: "Encrypt|Decrypt"
- from: "pkg/storage/db.go"
to: "pkg/storage/schema.sql"
via: "//go:embed schema.sql and db.Exec on open"
pattern: "go:embed.*schema"
- from: "pkg/storage/crypto.go"
to: "golang.org/x/crypto/argon2"
via: "argon2.IDKey call"
pattern: "argon2\\.IDKey"
---
<objective>
Build the storage layer: AES-256-GCM column encryption, Argon2id key derivation, SQLite database with WAL mode and embedded schema, and Finding CRUD operations that transparently encrypt key values on write and decrypt on read.
Purpose: Scanner results from Plan 04 and CLI commands from Plan 05 need a storage layer to persist findings. The encryption contract (Encrypt/Decrypt/DeriveKey) must exist before the scanner pipeline can store keys.
Output: pkg/storage/{encrypt,crypto,db,findings,schema}.go and db_test.go (stubs filled).
</objective>
<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/phases/01-foundation/01-RESEARCH.md
@.planning/phases/01-foundation/01-01-SUMMARY.md
<interfaces>
<!-- AES-256-GCM encrypt/decrypt pattern from RESEARCH.md Pattern 3 -->
func Encrypt(plaintext []byte, key []byte) ([]byte, error)
// key must be exactly 32 bytes (AES-256)
// nonce prepended to ciphertext in returned []byte
// uses crypto/aes + crypto/cipher GCM
func Decrypt(ciphertext []byte, key []byte) ([]byte, error)
// expects nonce prepended format from Encrypt()
// returns ErrCiphertextTooShort if len < nonceSize
<!-- Argon2id key derivation pattern from RESEARCH.md Pattern 4 -->
func DeriveKey(passphrase []byte, salt []byte) []byte
// params: time=1, memory=64*1024, threads=4, keyLen=32
// returns exactly 32 bytes deterministically
func NewSalt() ([]byte, error)
// generates 16 random bytes via crypto/rand
<!-- SQLite schema — findings table -->
findings table columns:
id INTEGER PRIMARY KEY AUTOINCREMENT
scan_id INTEGER REFERENCES scans(id)
provider_name TEXT NOT NULL
key_value BLOB NOT NULL -- AES-256-GCM encrypted, nonce prepended
key_masked TEXT NOT NULL -- first8...last4, stored plaintext for display
confidence TEXT NOT NULL -- "high", "medium", "low"
source_path TEXT
source_type TEXT -- "file", "dir", "git", "stdin", "url"
line_number INTEGER
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
scans table columns:
id INTEGER PRIMARY KEY AUTOINCREMENT
started_at DATETIME NOT NULL
finished_at DATETIME
source_path TEXT
finding_count INTEGER DEFAULT 0
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
settings table columns:
key TEXT PRIMARY KEY
value TEXT NOT NULL
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
<!-- Finding struct for inter-package communication -->
type Finding struct {
ID int64
ScanID int64
ProviderName string
KeyValue string // plaintext — encrypted before storage
KeyMasked string // first8chars...last4chars
Confidence string
SourcePath string
SourceType string
LineNumber int
}
<!-- DB driver registration -->
import _ "modernc.org/sqlite"
// driver registered as "sqlite" (NOT "sqlite3")
db, err := sql.Open("sqlite", dataSourceName)
</interfaces>
</context>
<tasks>
<task type="auto" tdd="true">
<name>Task 1: AES-256-GCM encryption and Argon2id key derivation</name>
<files>pkg/storage/encrypt.go, pkg/storage/crypto.go</files>
<read_first>
- /home/salva/Documents/apikey/.planning/phases/01-foundation/01-RESEARCH.md (Pattern 3: AES-256-GCM Column Encryption and Pattern 4: Argon2id Key Derivation — exact code examples)
</read_first>
<behavior>
- Test 1: Encrypt then Decrypt same key → returns original plaintext exactly
- Test 2: Encrypt produces output longer than input (nonce + tag overhead)
- Test 3: Two Encrypt calls on same plaintext → different ciphertext (random nonce)
- Test 4: Decrypt with wrong key → returns error (GCM authentication fails)
- Test 5: DeriveKey with same passphrase+salt → same 32-byte output (deterministic)
- Test 6: DeriveKey output is exactly 32 bytes
- Test 7: NewSalt() returns 16 bytes, two calls return different values
</behavior>
<action>
Create **pkg/storage/encrypt.go**:
```go
package storage
import (
"crypto/aes"
"crypto/cipher"
"crypto/rand"
"errors"
"io"
)
// ErrCiphertextTooShort is returned when ciphertext is shorter than the GCM nonce size.
var ErrCiphertextTooShort = errors.New("ciphertext too short")
// Encrypt encrypts plaintext using AES-256-GCM with a random nonce.
// The nonce is prepended to the returned ciphertext.
// key must be exactly 32 bytes (AES-256).
func Encrypt(plaintext []byte, key []byte) ([]byte, error) {
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
gcm, err := cipher.NewGCM(block)
if err != nil {
return nil, err
}
nonce := make([]byte, gcm.NonceSize())
if _, err := io.ReadFull(rand.Reader, nonce); err != nil {
return nil, err
}
// Seal appends encrypted data to nonce, so nonce is prepended
ciphertext := gcm.Seal(nonce, nonce, plaintext, nil)
return ciphertext, nil
}
// Decrypt decrypts ciphertext produced by Encrypt.
// Expects the nonce to be prepended to the ciphertext.
func Decrypt(ciphertext []byte, key []byte) ([]byte, error) {
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
gcm, err := cipher.NewGCM(block)
if err != nil {
return nil, err
}
nonceSize := gcm.NonceSize()
if len(ciphertext) < nonceSize {
return nil, ErrCiphertextTooShort
}
nonce, ciphertext := ciphertext[:nonceSize], ciphertext[nonceSize:]
return gcm.Open(nil, nonce, ciphertext, nil)
}
```
Create **pkg/storage/crypto.go**:
```go
package storage
import (
"crypto/rand"
"golang.org/x/crypto/argon2"
)
const (
argon2Time uint32 = 1
argon2Memory uint32 = 64 * 1024 // 64 MB — RFC 9106 Section 7.3
argon2Threads uint8 = 4
argon2KeyLen uint32 = 32 // AES-256 key length
saltSize = 16
)
// DeriveKey produces a 32-byte AES-256 key from a passphrase and salt using Argon2id.
// Uses RFC 9106 Section 7.3 recommended parameters.
// Given the same passphrase and salt, always returns the same key.
func DeriveKey(passphrase []byte, salt []byte) []byte {
return argon2.IDKey(passphrase, salt, argon2Time, argon2Memory, argon2Threads, argon2KeyLen)
}
// NewSalt generates a cryptographically random 16-byte salt.
// Store alongside the database and reuse on each key derivation.
func NewSalt() ([]byte, error) {
salt := make([]byte, saltSize)
if _, err := rand.Read(salt); err != nil {
return nil, err
}
return salt, nil
}
```
</action>
<verify>
<automated>cd /home/salva/Documents/apikey && go build ./pkg/storage/... && echo "BUILD OK"</automated>
</verify>
<acceptance_criteria>
- `go build ./pkg/storage/...` exits 0
- pkg/storage/encrypt.go exports: Encrypt, Decrypt, ErrCiphertextTooShort
- pkg/storage/crypto.go exports: DeriveKey, NewSalt
- `grep -q 'argon2\.IDKey' pkg/storage/crypto.go` exits 0
- `grep -q 'crypto/aes' pkg/storage/encrypt.go` exits 0
- `grep -q 'cipher\.NewGCM' pkg/storage/encrypt.go` exits 0
</acceptance_criteria>
<done>Encrypt/Decrypt and DeriveKey/NewSalt exist and compile. Encryption uses AES-256-GCM with random nonce. Key derivation uses Argon2id RFC 9106 parameters.</done>
</task>
<task type="auto" tdd="true">
<name>Task 2: SQLite database, schema, Finding CRUD, and filled test stubs</name>
<files>pkg/storage/schema.sql, pkg/storage/db.go, pkg/storage/findings.go, pkg/storage/db_test.go</files>
<read_first>
- /home/salva/Documents/apikey/.planning/phases/01-foundation/01-RESEARCH.md (STOR-01 row, Pattern 1 for embed usage pattern)
- /home/salva/Documents/apikey/pkg/storage/encrypt.go (Encrypt/Decrypt signatures)
- /home/salva/Documents/apikey/pkg/storage/crypto.go (DeriveKey signature)
</read_first>
<behavior>
- Test 1: Open(":memory:") returns *DB without error, schema tables exist
- Test 2: Encrypt/Decrypt roundtrip — Encrypt([]byte("sk-proj-abc"), key) then Decrypt returns "sk-proj-abc"
- Test 3: DeriveKey(passphrase, salt) twice returns identical 32 bytes
- Test 4: NewSalt() twice returns different slices
- Test 5: SaveFinding stores finding → ListFindings decrypts and returns KeyValue == "sk-proj-test"
- Test 6: Database file (when not :memory:) does NOT contain literal "sk-proj-test" in raw bytes
</behavior>
<action>
Create **pkg/storage/schema.sql**:
```sql
-- KeyHunter database schema
-- Version: 1
CREATE TABLE IF NOT EXISTS scans (
id INTEGER PRIMARY KEY AUTOINCREMENT,
started_at DATETIME NOT NULL,
finished_at DATETIME,
source_path TEXT,
finding_count INTEGER DEFAULT 0,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS findings (
id INTEGER PRIMARY KEY AUTOINCREMENT,
scan_id INTEGER REFERENCES scans(id),
provider_name TEXT NOT NULL,
key_value BLOB NOT NULL,
key_masked TEXT NOT NULL,
confidence TEXT NOT NULL,
source_path TEXT,
source_type TEXT,
line_number INTEGER,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS settings (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
-- Indexes for common queries
CREATE INDEX IF NOT EXISTS idx_findings_scan_id ON findings(scan_id);
CREATE INDEX IF NOT EXISTS idx_findings_provider ON findings(provider_name);
CREATE INDEX IF NOT EXISTS idx_findings_created ON findings(created_at DESC);
```
Create **pkg/storage/db.go**:
```go
package storage
import (
"database/sql"
_ "embed"
"fmt"
_ "modernc.org/sqlite"
)
//go:embed schema.sql
var schemaSQLBytes []byte
// DB wraps the sql.DB connection with KeyHunter-specific behavior.
type DB struct {
sql *sql.DB
}
// Open opens or creates a SQLite database at path, runs embedded schema migrations,
// and enables WAL mode for better concurrent read performance.
// Use ":memory:" for tests.
func Open(path string) (*DB, error) {
sqlDB, err := sql.Open("sqlite", path)
if err != nil {
return nil, fmt.Errorf("opening database: %w", err)
}
// Enable WAL mode for concurrent reads
if _, err := sqlDB.Exec("PRAGMA journal_mode=WAL"); err != nil {
sqlDB.Close()
return nil, fmt.Errorf("enabling WAL mode: %w", err)
}
// Enable foreign keys
if _, err := sqlDB.Exec("PRAGMA foreign_keys=ON"); err != nil {
sqlDB.Close()
return nil, fmt.Errorf("enabling foreign keys: %w", err)
}
// Run schema migrations
if _, err := sqlDB.Exec(string(schemaSQLBytes)); err != nil {
sqlDB.Close()
return nil, fmt.Errorf("running schema migrations: %w", err)
}
return &DB{sql: sqlDB}, nil
}
// Close closes the underlying database connection.
func (db *DB) Close() error {
return db.sql.Close()
}
// SQL returns the underlying sql.DB for advanced use cases.
func (db *DB) SQL() *sql.DB {
return db.sql
}
```
Create **pkg/storage/findings.go**:
```go
package storage
import (
"fmt"
"time"
)
// Finding represents a detected API key with metadata.
// KeyValue is always plaintext in this struct — encryption happens at the storage boundary.
type Finding struct {
ID int64
ScanID int64
ProviderName string
KeyValue string // plaintext — encrypted before storage, decrypted after retrieval
KeyMasked string // first8...last4, stored plaintext
Confidence string
SourcePath string
SourceType string
LineNumber int
CreatedAt time.Time
}
// MaskKey returns the masked form of a key: first 8 chars + "..." + last 4 chars.
// If the key is too short (< 12 chars), returns the full key masked with asterisks.
func MaskKey(key string) string {
if len(key) < 12 {
return "****"
}
return key[:8] + "..." + key[len(key)-4:]
}
// SaveFinding encrypts the finding's KeyValue and persists the finding to the database.
// encKey must be a 32-byte AES-256 key (from DeriveKey).
func (db *DB) SaveFinding(f Finding, encKey []byte) (int64, error) {
encrypted, err := Encrypt([]byte(f.KeyValue), encKey)
if err != nil {
return 0, fmt.Errorf("encrypting key value: %w", err)
}
masked := f.KeyMasked
if masked == "" {
masked = MaskKey(f.KeyValue)
}
res, err := db.sql.Exec(
`INSERT INTO findings (scan_id, provider_name, key_value, key_masked, confidence, source_path, source_type, line_number)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)`,
f.ScanID, f.ProviderName, encrypted, masked, f.Confidence, f.SourcePath, f.SourceType, f.LineNumber,
)
if err != nil {
return 0, fmt.Errorf("inserting finding: %w", err)
}
return res.LastInsertId()
}
// ListFindings retrieves all findings, decrypting key values using encKey.
// encKey must be the same 32-byte key used during SaveFinding.
func (db *DB) ListFindings(encKey []byte) ([]Finding, error) {
rows, err := db.sql.Query(
`SELECT id, scan_id, provider_name, key_value, key_masked, confidence,
source_path, source_type, line_number, created_at
FROM findings ORDER BY created_at DESC`,
)
if err != nil {
return nil, fmt.Errorf("querying findings: %w", err)
}
defer rows.Close()
var findings []Finding
for rows.Next() {
var f Finding
var encrypted []byte
var createdAt string
err := rows.Scan(
&f.ID, &f.ScanID, &f.ProviderName, &encrypted, &f.KeyMasked,
&f.Confidence, &f.SourcePath, &f.SourceType, &f.LineNumber, &createdAt,
)
if err != nil {
return nil, fmt.Errorf("scanning finding row: %w", err)
}
plain, err := Decrypt(encrypted, encKey)
if err != nil {
return nil, fmt.Errorf("decrypting finding %d: %w", f.ID, err)
}
f.KeyValue = string(plain)
f.CreatedAt, _ = time.Parse("2006-01-02 15:04:05", createdAt)
findings = append(findings, f)
}
return findings, rows.Err()
}
```
Fill **pkg/storage/db_test.go** (replacing stubs from Plan 01):
```go
package storage_test
import (
"testing"
"github.com/salvacybersec/keyhunter/pkg/storage"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestDBOpen(t *testing.T) {
db, err := storage.Open(":memory:")
require.NoError(t, err)
defer db.Close()
// Verify schema tables exist
rows, err := db.SQL().Query("SELECT name FROM sqlite_master WHERE type='table' ORDER BY name")
require.NoError(t, err)
defer rows.Close()
var tables []string
for rows.Next() {
var name string
require.NoError(t, rows.Scan(&name))
tables = append(tables, name)
}
assert.Contains(t, tables, "findings")
assert.Contains(t, tables, "scans")
assert.Contains(t, tables, "settings")
}
func TestEncryptDecryptRoundtrip(t *testing.T) {
key := make([]byte, 32) // all-zero key for test
for i := range key {
key[i] = byte(i)
}
plaintext := []byte("sk-proj-supersecretapikey1234")
ciphertext, err := storage.Encrypt(plaintext, key)
require.NoError(t, err)
assert.Greater(t, len(ciphertext), len(plaintext), "ciphertext should be longer than plaintext")
recovered, err := storage.Decrypt(ciphertext, key)
require.NoError(t, err)
assert.Equal(t, plaintext, recovered)
}
func TestEncryptNonDeterministic(t *testing.T) {
key := make([]byte, 32)
plain := []byte("test-key")
ct1, err1 := storage.Encrypt(plain, key)
ct2, err2 := storage.Encrypt(plain, key)
require.NoError(t, err1)
require.NoError(t, err2)
assert.NotEqual(t, ct1, ct2, "same plaintext encrypted twice should produce different ciphertext")
}
func TestDecryptWrongKey(t *testing.T) {
key1 := make([]byte, 32)
key2 := make([]byte, 32)
key2[0] = 0xFF
ct, err := storage.Encrypt([]byte("secret"), key1)
require.NoError(t, err)
_, err = storage.Decrypt(ct, key2)
assert.Error(t, err, "decryption with wrong key should fail")
}
func TestArgon2KeyDerivation(t *testing.T) {
passphrase := []byte("my-secure-passphrase")
salt := []byte("1234567890abcdef") // 16 bytes
key1 := storage.DeriveKey(passphrase, salt)
key2 := storage.DeriveKey(passphrase, salt)
assert.Equal(t, 32, len(key1), "derived key must be 32 bytes")
assert.Equal(t, key1, key2, "same passphrase+salt must produce same key")
}
func TestNewSalt(t *testing.T) {
salt1, err1 := storage.NewSalt()
salt2, err2 := storage.NewSalt()
require.NoError(t, err1)
require.NoError(t, err2)
assert.Equal(t, 16, len(salt1))
assert.NotEqual(t, salt1, salt2, "two salts should differ")
}
func TestSaveFindingEncrypted(t *testing.T) {
db, err := storage.Open(":memory:")
require.NoError(t, err)
defer db.Close()
// Derive a test key
key := storage.DeriveKey([]byte("testpassphrase"), []byte("testsalt1234xxxx"))
f := storage.Finding{
ProviderName: "openai",
KeyValue: "sk-proj-test1234567890abcdefghijklmnopqr",
Confidence: "high",
SourcePath: "/test/file.env",
SourceType: "file",
LineNumber: 42,
}
id, err := db.SaveFinding(f, key)
require.NoError(t, err)
assert.Greater(t, id, int64(0))
findings, err := db.ListFindings(key)
require.NoError(t, err)
require.Len(t, findings, 1)
assert.Equal(t, "sk-proj-test1234567890abcdefghijklmnopqr", findings[0].KeyValue)
assert.Equal(t, "openai", findings[0].ProviderName)
// Verify masking
assert.Equal(t, "sk-proj-...opqr", findings[0].KeyMasked)
}
```
</action>
<verify>
<automated>cd /home/salva/Documents/apikey && go test ./pkg/storage/... -v -count=1 2>&1 | tail -25</automated>
</verify>
<acceptance_criteria>
- `go test ./pkg/storage/... -v -count=1` exits 0 with all 7 tests PASS (no SKIP)
- TestDBOpen finds tables: findings, scans, settings
- TestEncryptDecryptRoundtrip passes — recovered plaintext matches original
- TestEncryptNonDeterministic passes — two encryptions differ
- TestDecryptWrongKey passes — wrong key causes error
- TestArgon2KeyDerivation passes — 32 bytes, deterministic
- TestNewSalt passes — 16 bytes, non-deterministic
- TestSaveFindingEncrypted passes — stored and retrieved with correct KeyValue and KeyMasked
- `grep -q 'go:embed.*schema' pkg/storage/db.go` exits 0
- `grep -q 'modernc.org/sqlite' pkg/storage/db.go` exits 0
- `grep -q 'journal_mode=WAL' pkg/storage/db.go` exits 0
</acceptance_criteria>
<done>Storage layer complete — SQLite opens with schema, AES-256-GCM encrypt/decrypt works, Argon2id key derivation works, SaveFinding/ListFindings encrypt/decrypt transparently. All 7 tests pass.</done>
</task>
</tasks>
<verification>
After both tasks:
- `go test ./pkg/storage/... -v -count=1` exits 0 with 7 tests PASS
- `go build ./...` still exits 0
- `grep -q 'argon2\.IDKey' pkg/storage/crypto.go` exits 0
- `grep -q 'cipher\.NewGCM' pkg/storage/encrypt.go` exits 0
- `grep -q 'journal_mode=WAL' pkg/storage/db.go` exits 0
- schema.sql contains CREATE TABLE for findings, scans, settings
</verification>
<success_criteria>
- SQLite database opens and auto-migrates from embedded schema.sql (STOR-01)
- AES-256-GCM column encryption works: Encrypt + Decrypt roundtrip returns original (STOR-02)
- Argon2id key derivation: DeriveKey deterministic, 32 bytes, RFC 9106 params (STOR-03)
- FindingCRUD: SaveFinding encrypts before INSERT, ListFindings decrypts after SELECT
- All 7 storage tests pass
</success_criteria>
<output>
After completion, create `.planning/phases/01-foundation/01-03-SUMMARY.md` following the summary template.
</output>