960 lines
34 KiB
Markdown
960 lines
34 KiB
Markdown
---
|
|
phase: 01-foundation
|
|
plan: 05
|
|
type: execute
|
|
wave: 3
|
|
depends_on: [01-02, 01-03, 01-04]
|
|
files_modified:
|
|
- cmd/root.go
|
|
- cmd/scan.go
|
|
- cmd/providers.go
|
|
- cmd/config.go
|
|
- cmd/stubs.go
|
|
- pkg/config/config.go
|
|
- pkg/output/table.go
|
|
autonomous: true
|
|
requirements: [CLI-01, CLI-02, CLI-03, CLI-04, CLI-05]
|
|
|
|
must_haves:
|
|
truths:
|
|
- "`keyhunter scan ./testdata/samples/openai_key.txt` runs the pipeline and prints a finding"
|
|
- "`keyhunter providers list` prints a table with at least 3 providers"
|
|
- "`keyhunter providers info openai` prints OpenAI provider details"
|
|
- "`keyhunter config init` creates ~/.keyhunter.yaml without error"
|
|
- "`keyhunter config set workers 16` persists the value to ~/.keyhunter.yaml"
|
|
- "`keyhunter --help` shows all top-level commands: scan, verify, import, recon, keys, serve, dorks, hook, schedule, providers, config"
|
|
- "Findings are stored with a per-installation salt loaded from the settings table — not a hardcoded salt"
|
|
- "Raw sqlite3 query on the database file does NOT return plaintext key values"
|
|
artifacts:
|
|
- path: "cmd/root.go"
|
|
provides: "Cobra root command with PersistentPreRunE config loading"
|
|
contains: "cobra.Command"
|
|
- path: "cmd/scan.go"
|
|
provides: "scan command wiring Engine + FileSource + output table + salt from settings"
|
|
exports: ["scanCmd"]
|
|
- path: "cmd/stubs.go"
|
|
provides: "stub commands for verify, import, recon, keys, serve, dorks, hook, schedule"
|
|
exports: ["verifyCmd", "importCmd", "reconCmd", "keysCmd", "serveCmd", "dorksCmd", "hookCmd", "scheduleCmd"]
|
|
- path: "cmd/providers.go"
|
|
provides: "providers list/info/stats subcommands using Registry"
|
|
exports: ["providersCmd"]
|
|
- path: "cmd/config.go"
|
|
provides: "config init/set/get subcommands using Viper"
|
|
exports: ["configCmd"]
|
|
- path: "pkg/config/config.go"
|
|
provides: "Config struct with Load() and defaults"
|
|
exports: ["Config", "Load"]
|
|
- path: "pkg/output/table.go"
|
|
provides: "lipgloss terminal table for printing Findings"
|
|
exports: ["PrintFindings"]
|
|
key_links:
|
|
- from: "cmd/scan.go"
|
|
to: "pkg/engine/engine.go"
|
|
via: "engine.NewEngine(registry).Scan() called in RunE"
|
|
pattern: "engine\\.NewEngine"
|
|
- from: "cmd/scan.go"
|
|
to: "pkg/storage/db.go"
|
|
via: "storage.Open() called, SaveFinding for each result"
|
|
pattern: "storage\\.Open"
|
|
- from: "cmd/scan.go"
|
|
to: "pkg/storage/crypto.go"
|
|
via: "loadOrCreateSalt() reads salt from settings table via storage.GetSetting/SetSetting, then calls storage.DeriveKey"
|
|
pattern: "DeriveKey|GetSetting|SetSetting"
|
|
- from: "cmd/root.go"
|
|
to: "github.com/spf13/viper"
|
|
via: "viper.SetConfigFile in PersistentPreRunE"
|
|
pattern: "viper\\.SetConfigFile"
|
|
- from: "cmd/providers.go"
|
|
to: "pkg/providers/registry.go"
|
|
via: "Registry.List(), Registry.Get(), Registry.Stats() called"
|
|
pattern: "registry\\.List|registry\\.Get|registry\\.Stats"
|
|
---
|
|
|
|
<objective>
|
|
Wire all subsystems together through the Cobra CLI: scan command (engine + storage + output), providers list/info/stats commands, config init/set/get commands, and 8 stub commands for future phases. This is the integration layer — all business logic lives in pkg/, cmd/ only wires.
|
|
|
|
Purpose: Satisfies all Phase 1 CLI requirements and delivers the first working `keyhunter scan` command that completes the end-to-end success criteria.
|
|
Output: cmd/{root,scan,providers,config,stubs}.go, pkg/config/config.go, pkg/output/table.go.
|
|
</objective>
|
|
|
|
<execution_context>
|
|
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
|
@$HOME/.claude/get-shit-done/templates/summary.md
|
|
</execution_context>
|
|
|
|
<context>
|
|
@.planning/phases/01-foundation/01-RESEARCH.md
|
|
@.planning/phases/01-foundation/01-02-SUMMARY.md
|
|
@.planning/phases/01-foundation/01-03-SUMMARY.md
|
|
@.planning/phases/01-foundation/01-04-SUMMARY.md
|
|
|
|
<interfaces>
|
|
<!-- Engine (from Plan 04) -->
|
|
package engine
|
|
type ScanConfig struct { Workers int; Verify bool; Unmask bool }
|
|
func NewEngine(registry *providers.Registry) *Engine
|
|
func (e *Engine) Scan(ctx context.Context, src sources.Source, cfg ScanConfig) (<-chan Finding, error)
|
|
|
|
<!-- FileSource (from Plan 04) -->
|
|
package sources
|
|
func NewFileSource(path string) *FileSource
|
|
|
|
<!-- Finding type (from Plan 04) -->
|
|
type Finding struct {
|
|
ProviderName string
|
|
KeyValue string
|
|
KeyMasked string
|
|
Confidence string
|
|
Source string
|
|
LineNumber int
|
|
}
|
|
|
|
<!-- Storage (from Plan 03) -->
|
|
package storage
|
|
func Open(path string) (*DB, error)
|
|
func (db *DB) SaveFinding(f Finding, encKey []byte) (int64, error)
|
|
func DeriveKey(passphrase []byte, salt []byte) []byte
|
|
func NewSalt() ([]byte, error)
|
|
|
|
<!-- Storage settings CRUD — must be added to pkg/storage/db.go in this plan -->
|
|
func (db *DB) GetSetting(key string) (string, bool, error)
|
|
func (db *DB) SetSetting(key string, value string) error
|
|
|
|
<!-- Registry (from Plan 02) -->
|
|
package providers
|
|
func NewRegistry() (*Registry, error)
|
|
func (r *Registry) List() []Provider
|
|
func (r *Registry) Get(name string) (Provider, bool)
|
|
func (r *Registry) Stats() RegistryStats
|
|
|
|
<!-- Config defaults -->
|
|
DBPath: ~/.keyhunter/keyhunter.db
|
|
ConfigPath: ~/.keyhunter.yaml
|
|
Workers: runtime.NumCPU() * 8
|
|
Passphrase: (prompt if not in env KEYHUNTER_PASSPHRASE — Phase 1: use empty string as dev default)
|
|
|
|
<!-- Viper config keys -->
|
|
"database.path" → DBPath
|
|
"scan.workers" → Workers
|
|
"encryption.passphrase" → Passphrase (sensitive — warn in help)
|
|
|
|
<!-- lipgloss table output -->
|
|
Columns: PROVIDER | MASKED KEY | CONFIDENCE | SOURCE | LINE
|
|
Colors: use lipgloss.NewStyle().Foreground() for confidence: high=green, medium=yellow, low=red
|
|
|
|
<!-- Salt wiring strategy (BLOCKER 4 fix) -->
|
|
On first scan, call storage.NewSalt(), hex-encode it, store in settings table with key "encryption.salt".
|
|
On subsequent scans, read the salt from the settings table.
|
|
This ensures all users have a unique per-installation salt instead of a shared hardcoded salt.
|
|
The helper function loadOrCreateSalt(db *storage.DB) ([]byte, error) handles both cases.
|
|
</interfaces>
|
|
</context>
|
|
|
|
<tasks>
|
|
|
|
<task type="auto" tdd="false">
|
|
<name>Task 1: Config package, output table, root command, and settings helpers</name>
|
|
<files>pkg/config/config.go, pkg/output/table.go, cmd/root.go, pkg/storage/settings.go</files>
|
|
<read_first>
|
|
- /home/salva/Documents/apikey/.planning/phases/01-foundation/01-RESEARCH.md (CLI-01, CLI-02, CLI-03 rows, Standard Stack: cobra v1.10.2 + viper v1.21.0)
|
|
- /home/salva/Documents/apikey/pkg/engine/finding.go (Finding struct fields for output)
|
|
- /home/salva/Documents/apikey/pkg/storage/db.go (DB struct, to add GetSetting/SetSetting)
|
|
</read_first>
|
|
<action>
|
|
Create **pkg/config/config.go**:
|
|
```go
|
|
package config
|
|
|
|
import (
|
|
"os"
|
|
"path/filepath"
|
|
"runtime"
|
|
)
|
|
|
|
// Config holds all KeyHunter runtime configuration.
|
|
// Values are populated from ~/.keyhunter.yaml, environment variables, and CLI flags (in that precedence order).
|
|
type Config struct {
|
|
DBPath string // path to SQLite database file
|
|
ConfigPath string // path to config YAML file
|
|
Workers int // number of scanner worker goroutines
|
|
Passphrase string // encryption passphrase (sensitive)
|
|
}
|
|
|
|
// Load returns a Config with defaults applied.
|
|
// Callers should override individual fields after Load() using viper-bound values.
|
|
func Load() Config {
|
|
home, _ := os.UserHomeDir()
|
|
return Config{
|
|
DBPath: filepath.Join(home, ".keyhunter", "keyhunter.db"),
|
|
ConfigPath: filepath.Join(home, ".keyhunter.yaml"),
|
|
Workers: runtime.NumCPU() * 8,
|
|
Passphrase: "", // Phase 1: empty passphrase; Phase 6+ will prompt
|
|
}
|
|
}
|
|
```
|
|
|
|
Create **pkg/storage/settings.go** — adds GetSetting/SetSetting to the storage package:
|
|
```go
|
|
package storage
|
|
|
|
import (
|
|
"database/sql"
|
|
"fmt"
|
|
)
|
|
|
|
// GetSetting retrieves a value from the settings table.
|
|
// Returns (value, true, nil) if found, ("", false, nil) if not found, ("", false, err) on error.
|
|
func (db *DB) GetSetting(key string) (string, bool, error) {
|
|
var value string
|
|
err := db.sql.QueryRow("SELECT value FROM settings WHERE key = ?", key).Scan(&value)
|
|
if err == sql.ErrNoRows {
|
|
return "", false, nil
|
|
}
|
|
if err != nil {
|
|
return "", false, fmt.Errorf("getting setting %q: %w", key, err)
|
|
}
|
|
return value, true, nil
|
|
}
|
|
|
|
// SetSetting inserts or updates a key-value pair in the settings table.
|
|
func (db *DB) SetSetting(key, value string) error {
|
|
_, err := db.sql.Exec(
|
|
`INSERT INTO settings (key, value, updated_at) VALUES (?, ?, CURRENT_TIMESTAMP)
|
|
ON CONFLICT(key) DO UPDATE SET value = excluded.value, updated_at = CURRENT_TIMESTAMP`,
|
|
key, value,
|
|
)
|
|
if err != nil {
|
|
return fmt.Errorf("setting %q: %w", key, err)
|
|
}
|
|
return nil
|
|
}
|
|
```
|
|
|
|
Create **pkg/output/table.go**:
|
|
```go
|
|
package output
|
|
|
|
import (
|
|
"fmt"
|
|
"os"
|
|
|
|
"github.com/charmbracelet/lipgloss"
|
|
"github.com/salvacybersec/keyhunter/pkg/engine"
|
|
)
|
|
|
|
var (
|
|
styleHigh = lipgloss.NewStyle().Foreground(lipgloss.Color("2")) // green
|
|
styleMedium = lipgloss.NewStyle().Foreground(lipgloss.Color("3")) // yellow
|
|
styleLow = lipgloss.NewStyle().Foreground(lipgloss.Color("1")) // red
|
|
styleHeader = lipgloss.NewStyle().Bold(true).Underline(true)
|
|
)
|
|
|
|
// PrintFindings writes findings as a colored terminal table to stdout.
|
|
// If unmask is true, KeyValue is shown; otherwise KeyMasked is shown.
|
|
func PrintFindings(findings []engine.Finding, unmask bool) {
|
|
if len(findings) == 0 {
|
|
fmt.Println("No API keys found.")
|
|
return
|
|
}
|
|
|
|
// Header
|
|
fmt.Fprintf(os.Stdout, "%-20s %-40s %-10s %-30s %s\n",
|
|
styleHeader.Render("PROVIDER"),
|
|
styleHeader.Render("KEY"),
|
|
styleHeader.Render("CONFIDENCE"),
|
|
styleHeader.Render("SOURCE"),
|
|
styleHeader.Render("LINE"),
|
|
)
|
|
fmt.Println(lipgloss.NewStyle().Foreground(lipgloss.Color("8")).Render(
|
|
"──────────────────────────────────────────────────────────────────────────────────────────────────────────",
|
|
))
|
|
|
|
for _, f := range findings {
|
|
keyDisplay := f.KeyMasked
|
|
if unmask {
|
|
keyDisplay = f.KeyValue
|
|
}
|
|
|
|
confStyle := styleLow
|
|
switch f.Confidence {
|
|
case "high":
|
|
confStyle = styleHigh
|
|
case "medium":
|
|
confStyle = styleMedium
|
|
}
|
|
|
|
fmt.Fprintf(os.Stdout, "%-20s %-40s %-10s %-30s %d\n",
|
|
f.ProviderName,
|
|
keyDisplay,
|
|
confStyle.Render(f.Confidence),
|
|
truncate(f.Source, 28),
|
|
f.LineNumber,
|
|
)
|
|
}
|
|
fmt.Printf("\n%d key(s) found.\n", len(findings))
|
|
}
|
|
|
|
func truncate(s string, max int) string {
|
|
if len(s) <= max {
|
|
return s
|
|
}
|
|
return "..." + s[len(s)-max+3:]
|
|
}
|
|
```
|
|
|
|
Create **cmd/root.go** (replaces the stub from Plan 01):
|
|
```go
|
|
package cmd
|
|
|
|
import (
|
|
"fmt"
|
|
"os"
|
|
"path/filepath"
|
|
|
|
"github.com/spf13/cobra"
|
|
"github.com/spf13/viper"
|
|
)
|
|
|
|
var cfgFile string
|
|
|
|
// rootCmd is the base command when called without any subcommands.
|
|
var rootCmd = &cobra.Command{
|
|
Use: "keyhunter",
|
|
Short: "KeyHunter — detect leaked LLM API keys across 108+ providers",
|
|
Long: `KeyHunter scans files, git history, and internet sources for leaked LLM API keys.
|
|
Supports 108+ providers with Aho-Corasick pre-filtering and regex + entropy detection.`,
|
|
SilenceUsage: true,
|
|
}
|
|
|
|
// Execute is the entry point called by main.go.
|
|
func Execute() {
|
|
if err := rootCmd.Execute(); err != nil {
|
|
os.Exit(1)
|
|
}
|
|
}
|
|
|
|
func init() {
|
|
cobra.OnInitialize(initConfig)
|
|
rootCmd.PersistentFlags().StringVar(&cfgFile, "config", "", "config file (default: ~/.keyhunter.yaml)")
|
|
rootCmd.AddCommand(scanCmd)
|
|
rootCmd.AddCommand(providersCmd)
|
|
rootCmd.AddCommand(configCmd)
|
|
// Stub commands for future phases (per CLI-01 requirement of 11 commands)
|
|
rootCmd.AddCommand(verifyCmd)
|
|
rootCmd.AddCommand(importCmd)
|
|
rootCmd.AddCommand(reconCmd)
|
|
rootCmd.AddCommand(keysCmd)
|
|
rootCmd.AddCommand(serveCmd)
|
|
rootCmd.AddCommand(dorksCmd)
|
|
rootCmd.AddCommand(hookCmd)
|
|
rootCmd.AddCommand(scheduleCmd)
|
|
}
|
|
|
|
func initConfig() {
|
|
if cfgFile != "" {
|
|
viper.SetConfigFile(cfgFile)
|
|
} else {
|
|
home, err := os.UserHomeDir()
|
|
if err != nil {
|
|
fmt.Fprintln(os.Stderr, "warning: cannot determine home directory:", err)
|
|
return
|
|
}
|
|
viper.SetConfigName(".keyhunter")
|
|
viper.SetConfigType("yaml")
|
|
viper.AddConfigPath(home)
|
|
viper.AddConfigPath(".")
|
|
}
|
|
|
|
viper.SetEnvPrefix("KEYHUNTER")
|
|
viper.AutomaticEnv()
|
|
|
|
// Defaults
|
|
viper.SetDefault("scan.workers", 0) // 0 = auto (CPU*8)
|
|
viper.SetDefault("database.path", filepath.Join(mustHomeDir(), ".keyhunter", "keyhunter.db"))
|
|
|
|
// Config file is optional — ignore if not found
|
|
_ = viper.ReadInConfig()
|
|
}
|
|
|
|
func mustHomeDir() string {
|
|
h, _ := os.UserHomeDir()
|
|
return h
|
|
}
|
|
```
|
|
</action>
|
|
<verify>
|
|
<automated>cd /home/salva/Documents/apikey && go build ./... && ./keyhunter --help 2>&1 | grep -E "scan|providers|config|verify|recon|keys|serve|dorks|hook|schedule" && echo "HELP OK"</automated>
|
|
</verify>
|
|
<acceptance_criteria>
|
|
- `go build ./...` exits 0
|
|
- `./keyhunter --help` shows scan, providers, config, verify, import, recon, keys, serve, dorks, hook, schedule in command list
|
|
- pkg/config/config.go exports Config and Load
|
|
- pkg/output/table.go exports PrintFindings
|
|
- pkg/storage/settings.go exports GetSetting and SetSetting
|
|
- cmd/root.go declares rootCmd, Execute(), and adds all 11 subcommands
|
|
- `grep -q 'viper\.SetConfigFile\|viper\.SetConfigName' cmd/root.go` exits 0
|
|
- lipgloss used for header and confidence coloring
|
|
</acceptance_criteria>
|
|
<done>Root command registers all 11 CLI commands. Config package, output table, and settings helpers exist. `keyhunter --help` shows all commands.</done>
|
|
</task>
|
|
|
|
<task type="auto" tdd="false">
|
|
<name>Task 2: scan, providers, config subcommands, and stub commands</name>
|
|
<files>cmd/scan.go, cmd/providers.go, cmd/config.go, cmd/stubs.go</files>
|
|
<read_first>
|
|
- /home/salva/Documents/apikey/.planning/phases/01-foundation/01-RESEARCH.md (CLI-04, CLI-05 rows, Pattern 2 pipeline usage)
|
|
- /home/salva/Documents/apikey/cmd/root.go (rootCmd, viper setup)
|
|
- /home/salva/Documents/apikey/pkg/engine/engine.go (Engine.Scan, ScanConfig)
|
|
- /home/salva/Documents/apikey/pkg/storage/db.go (Open, SaveFinding)
|
|
- /home/salva/Documents/apikey/pkg/storage/settings.go (GetSetting, SetSetting)
|
|
- /home/salva/Documents/apikey/pkg/providers/registry.go (NewRegistry, List, Get, Stats)
|
|
</read_first>
|
|
<action>
|
|
Create **cmd/scan.go**:
|
|
```go
|
|
package cmd
|
|
|
|
import (
|
|
"context"
|
|
"encoding/hex"
|
|
"encoding/json"
|
|
"fmt"
|
|
"os"
|
|
"path/filepath"
|
|
"runtime"
|
|
|
|
"github.com/spf13/cobra"
|
|
"github.com/spf13/viper"
|
|
"github.com/salvacybersec/keyhunter/pkg/config"
|
|
"github.com/salvacybersec/keyhunter/pkg/engine"
|
|
"github.com/salvacybersec/keyhunter/pkg/engine/sources"
|
|
"github.com/salvacybersec/keyhunter/pkg/output"
|
|
"github.com/salvacybersec/keyhunter/pkg/providers"
|
|
"github.com/salvacybersec/keyhunter/pkg/storage"
|
|
)
|
|
|
|
var (
|
|
flagWorkers int
|
|
flagVerify bool
|
|
flagUnmask bool
|
|
flagOutput string
|
|
flagExclude []string
|
|
)
|
|
|
|
var scanCmd = &cobra.Command{
|
|
Use: "scan <path>",
|
|
Short: "Scan a file or directory for leaked API keys",
|
|
Args: cobra.ExactArgs(1),
|
|
RunE: func(cmd *cobra.Command, args []string) error {
|
|
target := args[0]
|
|
|
|
// Load config
|
|
cfg := config.Load()
|
|
if viper.GetInt("scan.workers") > 0 {
|
|
cfg.Workers = viper.GetInt("scan.workers")
|
|
}
|
|
|
|
// Workers flag overrides config
|
|
workers := flagWorkers
|
|
if workers <= 0 {
|
|
workers = cfg.Workers
|
|
}
|
|
if workers <= 0 {
|
|
workers = runtime.NumCPU() * 8
|
|
}
|
|
|
|
// Initialize registry
|
|
reg, err := providers.NewRegistry()
|
|
if err != nil {
|
|
return fmt.Errorf("loading providers: %w", err)
|
|
}
|
|
|
|
// Initialize engine
|
|
eng := engine.NewEngine(reg)
|
|
src := sources.NewFileSource(target)
|
|
|
|
scanCfg := engine.ScanConfig{
|
|
Workers: workers,
|
|
Verify: flagVerify,
|
|
Unmask: flagUnmask,
|
|
}
|
|
|
|
// Open database (ensure directory exists)
|
|
dbPath := viper.GetString("database.path")
|
|
if dbPath == "" {
|
|
dbPath = cfg.DBPath
|
|
}
|
|
if err := os.MkdirAll(filepath.Dir(dbPath), 0700); err != nil {
|
|
return fmt.Errorf("creating database directory: %w", err)
|
|
}
|
|
db, err := storage.Open(dbPath)
|
|
if err != nil {
|
|
return fmt.Errorf("opening database: %w", err)
|
|
}
|
|
defer db.Close()
|
|
|
|
// Derive encryption key using a per-installation salt stored in settings table.
|
|
// On first run, NewSalt() generates a random salt and stores it.
|
|
// On subsequent runs, the same salt is loaded — ensuring consistent encryption.
|
|
encKey, err := loadOrCreateEncKey(db, cfg.Passphrase)
|
|
if err != nil {
|
|
return fmt.Errorf("preparing encryption key: %w", err)
|
|
}
|
|
|
|
// Run scan
|
|
ch, err := eng.Scan(context.Background(), src, scanCfg)
|
|
if err != nil {
|
|
return fmt.Errorf("starting scan: %w", err)
|
|
}
|
|
|
|
var findings []engine.Finding
|
|
for f := range ch {
|
|
findings = append(findings, f)
|
|
// Persist to storage
|
|
storeFinding := storage.Finding{
|
|
ProviderName: f.ProviderName,
|
|
KeyValue: f.KeyValue,
|
|
KeyMasked: f.KeyMasked,
|
|
Confidence: f.Confidence,
|
|
SourcePath: f.Source,
|
|
SourceType: f.SourceType,
|
|
LineNumber: f.LineNumber,
|
|
}
|
|
if _, err := db.SaveFinding(storeFinding, encKey); err != nil {
|
|
fmt.Fprintf(os.Stderr, "warning: failed to save finding: %v\n", err)
|
|
}
|
|
}
|
|
|
|
// Output
|
|
switch flagOutput {
|
|
case "json":
|
|
// Return valid empty JSON array when no findings; full JSON in Phase 6.
|
|
enc := json.NewEncoder(os.Stdout)
|
|
enc.SetIndent("", " ")
|
|
type jsonFinding struct {
|
|
Provider string `json:"provider"`
|
|
KeyMasked string `json:"key_masked"`
|
|
Confidence string `json:"confidence"`
|
|
Source string `json:"source"`
|
|
Line int `json:"line"`
|
|
}
|
|
out := make([]jsonFinding, 0, len(findings))
|
|
for _, f := range findings {
|
|
out = append(out, jsonFinding{
|
|
Provider: f.ProviderName,
|
|
KeyMasked: f.KeyMasked,
|
|
Confidence: f.Confidence,
|
|
Source: f.Source,
|
|
Line: f.LineNumber,
|
|
})
|
|
}
|
|
if err := enc.Encode(out); err != nil {
|
|
return fmt.Errorf("encoding JSON output: %w", err)
|
|
}
|
|
default:
|
|
output.PrintFindings(findings, flagUnmask)
|
|
}
|
|
|
|
// Exit code semantics (CLI-05 / OUT-06): 0=clean, 1=found, 2=error
|
|
if len(findings) > 0 {
|
|
os.Exit(1)
|
|
}
|
|
return nil
|
|
},
|
|
}
|
|
|
|
// loadOrCreateEncKey loads the per-installation salt from the settings table.
|
|
// On first run it generates a new random salt with storage.NewSalt() and persists it.
|
|
// The salt is hex-encoded in the settings table under key "encryption.salt".
|
|
func loadOrCreateEncKey(db *storage.DB, passphrase string) ([]byte, error) {
|
|
const saltKey = "encryption.salt"
|
|
saltHex, found, err := db.GetSetting(saltKey)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("reading salt from settings: %w", err)
|
|
}
|
|
var salt []byte
|
|
if !found {
|
|
// First run: generate and persist a new random salt.
|
|
salt, err = storage.NewSalt()
|
|
if err != nil {
|
|
return nil, fmt.Errorf("generating salt: %w", err)
|
|
}
|
|
if err := db.SetSetting(saltKey, hex.EncodeToString(salt)); err != nil {
|
|
return nil, fmt.Errorf("storing salt: %w", err)
|
|
}
|
|
} else {
|
|
salt, err = hex.DecodeString(saltHex)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("decoding stored salt: %w", err)
|
|
}
|
|
}
|
|
return storage.DeriveKey([]byte(passphrase), salt), nil
|
|
}
|
|
|
|
func init() {
|
|
scanCmd.Flags().IntVar(&flagWorkers, "workers", 0, "number of worker goroutines (default: CPU*8)")
|
|
scanCmd.Flags().BoolVar(&flagVerify, "verify", false, "actively verify found keys (opt-in, Phase 5)")
|
|
scanCmd.Flags().BoolVar(&flagUnmask, "unmask", false, "show full key values (default: masked)")
|
|
scanCmd.Flags().StringVar(&flagOutput, "output", "table", "output format: table, json (full JSON output in Phase 6)")
|
|
scanCmd.Flags().StringSliceVar(&flagExclude, "exclude", nil, "glob patterns to exclude (e.g. *.min.js)")
|
|
viper.BindPFlag("scan.workers", scanCmd.Flags().Lookup("workers"))
|
|
}
|
|
```
|
|
|
|
Create **cmd/providers.go**:
|
|
```go
|
|
package cmd
|
|
|
|
import (
|
|
"fmt"
|
|
"os"
|
|
"strings"
|
|
|
|
"github.com/charmbracelet/lipgloss"
|
|
"github.com/spf13/cobra"
|
|
"github.com/salvacybersec/keyhunter/pkg/providers"
|
|
)
|
|
|
|
var providersCmd = &cobra.Command{
|
|
Use: "providers",
|
|
Short: "Manage and inspect provider definitions",
|
|
}
|
|
|
|
var providersListCmd = &cobra.Command{
|
|
Use: "list",
|
|
Short: "List all loaded provider definitions",
|
|
RunE: func(cmd *cobra.Command, args []string) error {
|
|
reg, err := providers.NewRegistry()
|
|
if err != nil {
|
|
return err
|
|
}
|
|
bold := lipgloss.NewStyle().Bold(true)
|
|
fmt.Fprintf(os.Stdout, "%-20s %-6s %-8s %s\n",
|
|
bold.Render("NAME"), bold.Render("TIER"), bold.Render("PATTERNS"), bold.Render("KEYWORDS"))
|
|
fmt.Println(strings.Repeat("─", 70))
|
|
for _, p := range reg.List() {
|
|
fmt.Fprintf(os.Stdout, "%-20s %-6d %-8d %s\n",
|
|
p.Name, p.Tier, len(p.Patterns), strings.Join(p.Keywords, ", "))
|
|
}
|
|
stats := reg.Stats()
|
|
fmt.Printf("\nTotal: %d providers\n", stats.Total)
|
|
return nil
|
|
},
|
|
}
|
|
|
|
var providersInfoCmd = &cobra.Command{
|
|
Use: "info <name>",
|
|
Short: "Show detailed info for a provider",
|
|
Args: cobra.ExactArgs(1),
|
|
RunE: func(cmd *cobra.Command, args []string) error {
|
|
reg, err := providers.NewRegistry()
|
|
if err != nil {
|
|
return err
|
|
}
|
|
p, ok := reg.Get(args[0])
|
|
if !ok {
|
|
return fmt.Errorf("provider %q not found", args[0])
|
|
}
|
|
fmt.Printf("Name: %s\n", p.Name)
|
|
fmt.Printf("Display Name: %s\n", p.DisplayName)
|
|
fmt.Printf("Tier: %d\n", p.Tier)
|
|
fmt.Printf("Last Verified: %s\n", p.LastVerified)
|
|
fmt.Printf("Keywords: %s\n", strings.Join(p.Keywords, ", "))
|
|
fmt.Printf("Patterns: %d\n", len(p.Patterns))
|
|
for i, pat := range p.Patterns {
|
|
fmt.Printf(" [%d] regex=%s confidence=%s entropy_min=%.1f\n",
|
|
i+1, pat.Regex, pat.Confidence, pat.EntropyMin)
|
|
}
|
|
if p.Verify.URL != "" {
|
|
fmt.Printf("Verify URL: %s %s\n", p.Verify.Method, p.Verify.URL)
|
|
}
|
|
return nil
|
|
},
|
|
}
|
|
|
|
var providersStatsCmd = &cobra.Command{
|
|
Use: "stats",
|
|
Short: "Show provider statistics",
|
|
RunE: func(cmd *cobra.Command, args []string) error {
|
|
reg, err := providers.NewRegistry()
|
|
if err != nil {
|
|
return err
|
|
}
|
|
stats := reg.Stats()
|
|
fmt.Printf("Total providers: %d\n", stats.Total)
|
|
fmt.Printf("By tier:\n")
|
|
for tier := 1; tier <= 9; tier++ {
|
|
if count := stats.ByTier[tier]; count > 0 {
|
|
fmt.Printf(" Tier %d: %d\n", tier, count)
|
|
}
|
|
}
|
|
fmt.Printf("By confidence:\n")
|
|
for conf, count := range stats.ByConfidence {
|
|
fmt.Printf(" %s: %d\n", conf, count)
|
|
}
|
|
return nil
|
|
},
|
|
}
|
|
|
|
func init() {
|
|
providersCmd.AddCommand(providersListCmd)
|
|
providersCmd.AddCommand(providersInfoCmd)
|
|
providersCmd.AddCommand(providersStatsCmd)
|
|
}
|
|
```
|
|
|
|
Create **cmd/config.go**:
|
|
```go
|
|
package cmd
|
|
|
|
import (
|
|
"fmt"
|
|
"os"
|
|
"path/filepath"
|
|
|
|
"github.com/spf13/cobra"
|
|
"github.com/spf13/viper"
|
|
)
|
|
|
|
var configCmd = &cobra.Command{
|
|
Use: "config",
|
|
Short: "Manage KeyHunter configuration",
|
|
}
|
|
|
|
var configInitCmd = &cobra.Command{
|
|
Use: "init",
|
|
Short: "Create default configuration file at ~/.keyhunter.yaml",
|
|
RunE: func(cmd *cobra.Command, args []string) error {
|
|
home, err := os.UserHomeDir()
|
|
if err != nil {
|
|
return fmt.Errorf("cannot determine home directory: %w", err)
|
|
}
|
|
configPath := filepath.Join(home, ".keyhunter.yaml")
|
|
|
|
// Set defaults before writing
|
|
viper.SetDefault("scan.workers", 0)
|
|
viper.SetDefault("database.path", filepath.Join(home, ".keyhunter", "keyhunter.db"))
|
|
|
|
if err := viper.WriteConfigAs(configPath); err != nil {
|
|
return fmt.Errorf("writing config: %w", err)
|
|
}
|
|
fmt.Printf("Config initialized: %s\n", configPath)
|
|
return nil
|
|
},
|
|
}
|
|
|
|
var configSetCmd = &cobra.Command{
|
|
Use: "set <key> <value>",
|
|
Short: "Set a configuration value",
|
|
Args: cobra.ExactArgs(2),
|
|
RunE: func(cmd *cobra.Command, args []string) error {
|
|
key, value := args[0], args[1]
|
|
viper.Set(key, value)
|
|
if err := viper.WriteConfig(); err != nil {
|
|
// If config file doesn't exist yet, create it
|
|
home, _ := os.UserHomeDir()
|
|
configPath := filepath.Join(home, ".keyhunter.yaml")
|
|
if err2 := viper.WriteConfigAs(configPath); err2 != nil {
|
|
return fmt.Errorf("writing config: %w", err2)
|
|
}
|
|
}
|
|
fmt.Printf("Set %s = %s\n", key, value)
|
|
return nil
|
|
},
|
|
}
|
|
|
|
var configGetCmd = &cobra.Command{
|
|
Use: "get <key>",
|
|
Short: "Get a configuration value",
|
|
Args: cobra.ExactArgs(1),
|
|
RunE: func(cmd *cobra.Command, args []string) error {
|
|
val := viper.Get(args[0])
|
|
if val == nil {
|
|
return fmt.Errorf("key %q not found", args[0])
|
|
}
|
|
fmt.Printf("%v\n", val)
|
|
return nil
|
|
},
|
|
}
|
|
|
|
func init() {
|
|
configCmd.AddCommand(configInitCmd)
|
|
configCmd.AddCommand(configSetCmd)
|
|
configCmd.AddCommand(configGetCmd)
|
|
}
|
|
```
|
|
|
|
Create **cmd/stubs.go** — stub commands for the 8 phases not yet implemented.
|
|
These satisfy CLI-01 (11 commands) and print a clear "not implemented" message so users
|
|
know the command exists but is pending a future phase.
|
|
|
|
```go
|
|
package cmd
|
|
|
|
import (
|
|
"fmt"
|
|
|
|
"github.com/spf13/cobra"
|
|
)
|
|
|
|
// notImplemented returns a RunE function that prints a "not yet implemented" message.
|
|
// Each stub command is registered in root.go and satisfies CLI-01's 11-command requirement.
|
|
func notImplemented(name, phase string) func(cmd *cobra.Command, args []string) error {
|
|
return func(cmd *cobra.Command, args []string) error {
|
|
fmt.Printf("%s: not implemented in this phase (coming in %s)\n", name, phase)
|
|
return nil
|
|
}
|
|
}
|
|
|
|
var verifyCmd = &cobra.Command{
|
|
Use: "verify",
|
|
Short: "Actively verify found API keys (Phase 5)",
|
|
RunE: notImplemented("verify", "Phase 5"),
|
|
}
|
|
|
|
var importCmd = &cobra.Command{
|
|
Use: "import",
|
|
Short: "Import findings from TruffleHog or Gitleaks output (Phase 7)",
|
|
RunE: notImplemented("import", "Phase 7"),
|
|
}
|
|
|
|
var reconCmd = &cobra.Command{
|
|
Use: "recon",
|
|
Short: "Run OSINT recon across internet sources (Phase 9+)",
|
|
RunE: notImplemented("recon", "Phase 9"),
|
|
}
|
|
|
|
var keysCmd = &cobra.Command{
|
|
Use: "keys",
|
|
Short: "Manage stored keys (list, export, delete) (Phase 6)",
|
|
RunE: notImplemented("keys", "Phase 6"),
|
|
}
|
|
|
|
var serveCmd = &cobra.Command{
|
|
Use: "serve",
|
|
Short: "Start the web dashboard (Phase 18)",
|
|
RunE: notImplemented("serve", "Phase 18"),
|
|
}
|
|
|
|
var dorksCmd = &cobra.Command{
|
|
Use: "dorks",
|
|
Short: "Manage and run dork queries (Phase 8)",
|
|
RunE: notImplemented("dorks", "Phase 8"),
|
|
}
|
|
|
|
var hookCmd = &cobra.Command{
|
|
Use: "hook",
|
|
Short: "Install or manage git pre-commit hooks (Phase 7)",
|
|
RunE: notImplemented("hook", "Phase 7"),
|
|
}
|
|
|
|
var scheduleCmd = &cobra.Command{
|
|
Use: "schedule",
|
|
Short: "Manage scheduled recurring scans (Phase 17)",
|
|
RunE: notImplemented("schedule", "Phase 17"),
|
|
}
|
|
```
|
|
</action>
|
|
<verify>
|
|
<automated>cd /home/salva/Documents/apikey && go build -o keyhunter . && ./keyhunter providers list && ./keyhunter providers info openai && echo "PROVIDERS OK"</automated>
|
|
</verify>
|
|
<acceptance_criteria>
|
|
- `go build -o keyhunter .` exits 0
|
|
- `./keyhunter --help` shows all 11 commands: scan, verify, import, recon, keys, serve, dorks, hook, schedule, providers, config
|
|
- `./keyhunter providers list` prints table with >= 3 rows including "openai"
|
|
- `./keyhunter providers info openai` prints Name, Tier, Keywords, Patterns, Verify URL
|
|
- `./keyhunter providers stats` prints "Total providers: 3" or more
|
|
- `./keyhunter config init` creates or updates ~/.keyhunter.yaml
|
|
- `./keyhunter config set scan.workers 16` exits 0
|
|
- `./keyhunter verify` prints "not implemented in this phase"
|
|
- `./keyhunter recon` prints "not implemented in this phase"
|
|
- `./keyhunter scan testdata/samples/openai_key.txt` exits 1 (keys found) and prints a table row with "openai"
|
|
- `./keyhunter scan testdata/samples/no_keys.txt` exits 0 and prints "No API keys found."
|
|
- `./keyhunter scan --output json testdata/samples/no_keys.txt` exits 0 and prints `[]` (valid JSON)
|
|
- Second run of `./keyhunter scan testdata/samples/openai_key.txt` uses the SAME salt (loaded from settings table)
|
|
- `grep -q 'viper\.BindPFlag' cmd/scan.go` exits 0
|
|
- `grep -q 'loadOrCreateEncKey' cmd/scan.go` exits 0
|
|
</acceptance_criteria>
|
|
<done>Full CLI works: scan finds and persists keys with per-installation salt, providers list/info/stats work, config init/set/get work, 8 stub commands registered and respond. Phase 1 success criteria all met.</done>
|
|
</task>
|
|
|
|
<task type="checkpoint:human-verify" gate="blocking">
|
|
<what-built>
|
|
Complete Phase 1 implementation:
|
|
- Provider registry with 3 YAML definitions, Aho-Corasick automaton, schema validation
|
|
- Storage layer with AES-256-GCM encryption, Argon2id key derivation, SQLite WAL mode, per-installation salt
|
|
- Three-stage scan engine: keyword pre-filter → regex + entropy detector → finding channel
|
|
- CLI: keyhunter scan, providers list/info/stats, config init/set/get
|
|
- 8 stub commands for future phases (verify, import, recon, keys, serve, dorks, hook, schedule)
|
|
</what-built>
|
|
<how-to-verify>
|
|
Run these commands from the project root and confirm each expected output:
|
|
|
|
1. `cd /home/salva/Documents/apikey && go test ./... -v -count=1`
|
|
Expected: All tests PASS, zero FAIL, zero SKIP (except original stubs now filled)
|
|
|
|
2. `./keyhunter scan testdata/samples/openai_key.txt`
|
|
Expected: Exit code 1, table printed with 1 row showing "openai" provider, masked key
|
|
|
|
3. `./keyhunter scan testdata/samples/no_keys.txt`
|
|
Expected: Exit code 0, "No API keys found." printed
|
|
|
|
4. `./keyhunter scan --output json testdata/samples/no_keys.txt`
|
|
Expected: Exit code 0, valid JSON printed: `[]`
|
|
|
|
5. `./keyhunter providers list`
|
|
Expected: Table with openai, anthropic, huggingface rows
|
|
|
|
6. `./keyhunter providers info openai`
|
|
Expected: Name, Tier 1, Keywords including "sk-proj-", Pattern regex shown
|
|
|
|
7. `./keyhunter config init`
|
|
Expected: "Config initialized: ~/.keyhunter.yaml" and the file exists
|
|
|
|
8. `./keyhunter config set scan.workers 16 && ./keyhunter config get scan.workers`
|
|
Expected: "Set scan.workers = 16" then "16"
|
|
|
|
9. `./keyhunter verify`
|
|
Expected: "verify: not implemented in this phase (coming in Phase 5)"
|
|
|
|
10. Build the binary with production flags:
|
|
`CGO_ENABLED=0 go build -ldflags="-s -w" -o keyhunter-prod .`
|
|
Expected: Builds without error, binary produced
|
|
</how-to-verify>
|
|
<resume-signal>Type "approved" if all 10 checks pass, or describe which check failed and what output you saw.</resume-signal>
|
|
</task>
|
|
|
|
</tasks>
|
|
|
|
<verification>
|
|
Full Phase 1 integration check:
|
|
- `go test ./... -count=1` exits 0
|
|
- `./keyhunter scan testdata/samples/openai_key.txt` exits 1 with findings table
|
|
- `./keyhunter scan testdata/samples/no_keys.txt` exits 0 with "No API keys found."
|
|
- `./keyhunter scan --output json testdata/samples/no_keys.txt` prints valid JSON `[]`
|
|
- `./keyhunter providers list` shows 3+ providers
|
|
- `./keyhunter config init` creates ~/.keyhunter.yaml
|
|
- `./keyhunter verify` prints "not implemented in this phase"
|
|
- `CGO_ENABLED=0 go build -ldflags="-s -w" -o keyhunter-prod .` exits 0
|
|
</verification>
|
|
|
|
<success_criteria>
|
|
- Cobra CLI with all 11 commands: scan, verify, import, recon, keys, serve, dorks, hook, schedule, providers, config (CLI-01)
|
|
- `keyhunter config init` creates ~/.keyhunter.yaml (CLI-02)
|
|
- `keyhunter config set key value` persists (CLI-03)
|
|
- `keyhunter providers list/info/stats` work (CLI-04)
|
|
- scan flags: --workers, --verify, --unmask, --output, --exclude (CLI-05)
|
|
- Per-installation salt stored in settings table; no hardcoded salt in production code
|
|
- JSON output returns valid JSON (not a comment string)
|
|
- All Phase 1 success criteria from ROADMAP.md satisfied:
|
|
1. `keyhunter scan ./somefile` runs three-stage pipeline and returns findings with provider names
|
|
2. Findings persisted to SQLite with AES-256 encrypted key_value; raw db does not contain plaintext
|
|
3. `keyhunter config init` and `config set` work
|
|
4. `keyhunter providers list/info` return provider metadata from YAML
|
|
5. Provider YAML has format_version and last_verified, validated at load time
|
|
</success_criteria>
|
|
|
|
<output>
|
|
After completion, create `.planning/phases/01-foundation/01-05-SUMMARY.md` following the summary template.
|
|
</output>
|