diff --git a/.planning/STATE.md b/.planning/STATE.md index c1406e2..b51ea73 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -4,13 +4,13 @@ milestone: v1.0 milestone_name: milestone status: executing stopped_at: Completed 08-07-PLAN.md -last_updated: "2026-04-05T21:25:47.473Z" +last_updated: "2026-04-05T21:32:47.810Z" last_activity: 2026-04-05 progress: total_phases: 18 - completed_phases: 7 + completed_phases: 8 total_plans: 47 - completed_plans: 46 + completed_plans: 47 percent: 20 --- @@ -25,8 +25,8 @@ See: .planning/PROJECT.md (updated 2026-04-04) ## Current Position -Phase: 08 (dork-engine) — EXECUTING -Plan: 4 of 7 +Phase: 9 +Plan: Not started Status: Ready to execute Last activity: 2026-04-05 diff --git a/.planning/phases/08-dork-engine/08-VERIFICATION.md b/.planning/phases/08-dork-engine/08-VERIFICATION.md new file mode 100644 index 0000000..33f8a45 --- /dev/null +++ b/.planning/phases/08-dork-engine/08-VERIFICATION.md @@ -0,0 +1,111 @@ +--- +phase: 08-dork-engine +verified: 2026-04-05T00:00:00Z +status: passed +score: 4/4 must-haves verified +--- + +# Phase 08: Dork Engine Verification Report + +**Phase Goal:** Users can run, manage, and extend a library of 150+ built-in YAML dorks across GitHub, Google, Shodan, Censys, ZoomEye, FOFA, GitLab, and Bing. +**Verified:** 2026-04-05 +**Status:** passed +**Re-verification:** No — initial verification + +## Goal Achievement + +### Observable Truths (from Success Criteria + plan must_haves) + +| # | Truth | Status | Evidence | +|---|-------|--------|----------| +| 1 | `keyhunter dorks list` shows 150+ built-in dorks with source and category columns | VERIFIED | Built binary; `dorks list` prints 150 data rows + header + summary. Columns ID / SOURCE / CATEGORY / NAME present. | +| 2 | `keyhunter dorks run --source=github --category=frontier` executes against GitHub code search | VERIFIED | Command reaches `dorks.GitHubExecutor` and returns `ErrMissingAuth` with setup hint when GITHUB_TOKEN unset — i.e. wired all the way through; `pkg/dorks/github.go` targets `api.github.com/search/code` and handles Retry-After. | +| 3 | `keyhunter dorks add --source=google --query=... --description=...` persists a custom dork | VERIFIED | Live add created row 1, list shows `*custom-verify-test` (custom marker), export includes it. Round-trip through `storage.SaveCustomDork` / `ListCustomDorks`. | +| 4 | `keyhunter dorks export --format=json` exports all dorks including custom | VERIFIED | JSON export emitted with embedded + custom entries; custom-verify-test appears in output. | +| 5 | Embedded corpus holds >= 150 dorks across 8 sources and 5 categories | VERIFIED | Exact count = 150. Sources: github 50, google 30, shodan 20, censys 15, zoomeye 10, fofa 10, gitlab 10, bing 5. Categories frontier/specialized/infrastructure/emerging/enterprise all present. | +| 6 | Guardrail test locks 150 threshold + per-source minimums + ID uniqueness | VERIFIED | `TestDorkCountGuardrail`, `TestDorkCountPerSource`, `TestDorkCategoriesPresent`, `TestDorkIDsUnique` all PASS. | + +**Score:** 6/6 truths verified (success criteria 1-4 plus underlying corpus/guardrail truths) + +### Required Artifacts + +| Artifact | Expected | Status | Details | +|----------|----------|--------|---------| +| `pkg/dorks/schema.go` | Dork struct + Validate | VERIFIED | Present, used by loader/tests. | +| `pkg/dorks/loader.go` | go:embed walker | VERIFIED | `//go:embed definitions/*` on embed.FS. | +| `pkg/dorks/registry.go` | Registry with List/Get/Stats/ListBySource/ListByCategory | VERIFIED | `NewRegistry()` exercised by tests + CLI. | +| `pkg/dorks/executor.go` | Executor interface + ErrSourceNotImplemented | VERIFIED | Dispatched from cmd/dorks.go run. | +| `pkg/dorks/github.go` | GitHubExecutor hitting api.github.com/search/code | VERIFIED | Auth handling, Retry-After parsing, ErrMissingAuth all present. | +| `pkg/dorks/count_test.go` | Guardrail test | VERIFIED | 4 guardrail tests (count, per-source, categories, unique IDs). | +| `pkg/dorks/definitions/github/*.yaml` | >= 50 GitHub dorks across 5 categories | VERIFIED | 50 entries across frontier/specialized/infrastructure/emerging/enterprise. | +| `pkg/dorks/definitions/google/*.yaml` | >= 30 Google dorks | VERIFIED | 30 entries across 3 category files. | +| `pkg/dorks/definitions/shodan/*.yaml` | >= 20 Shodan dorks | VERIFIED | 20 entries (frontier 6 + infrastructure 14). | +| `pkg/dorks/definitions/censys/all.yaml` | >= 15 | VERIFIED | 15 entries. | +| `pkg/dorks/definitions/zoomeye/all.yaml` | >= 10 | VERIFIED | 10 entries. | +| `pkg/dorks/definitions/fofa/all.yaml` | >= 10 | VERIFIED | 10 entries. | +| `pkg/dorks/definitions/gitlab/all.yaml` | >= 10 | VERIFIED | 10 entries. | +| `pkg/dorks/definitions/bing/all.yaml` | >= 5 | VERIFIED | 5 entries. | +| `pkg/storage/custom_dorks.go` | Save/List/Delete/Get CustomDork | VERIFIED | All CRUD referenced from cmd/dorks.go and exercised in live test. | +| `pkg/storage/schema.sql` | custom_dorks table DDL | VERIFIED | `CREATE TABLE IF NOT EXISTS custom_dorks` at line 44. | +| `cmd/dorks.go` | dorks command tree (list/info/export/run/add/delete) | VERIFIED | All 6 subcommands registered via `dorksCmd.AddCommand`. | + +### Key Link Verification + +| From | To | Via | Status | Details | +|------|----|-----|--------|---------| +| `pkg/dorks/loader.go` | `definitions/*/*.yaml` | `//go:embed definitions/*` | WIRED | Loader walks embed.FS subdirs; 150 dorks load at runtime (verified via binary + guardrail test). | +| `cmd/dorks.go` | `pkg/dorks.Registry` | `dorks.NewRegistry()` | WIRED | Called in 5 subcommand handlers (list, info, export, run, add validation, delete). | +| `cmd/dorks.go run` | `pkg/dorks.GitHubExecutor` | `NewGitHubExecutor(viper.GetString("dorks.github.token"))` | WIRED | Observed in live run: executor produced its own auth error message, confirming the call path reaches github.go. | +| `cmd/dorks.go add/delete` | `storage.DB custom_dorks` | `SaveCustomDork` / `DeleteCustomDork` / `ListCustomDorks` | WIRED | Live add returned `row 1`; list+export surfaced the custom entry. | +| `pkg/storage/schema.sql` | custom_dorks table | `CREATE TABLE IF NOT EXISTS custom_dorks` | WIRED | Table created on DB init; round-trip verified. | +| `pkg/dorks/github.go` | `https://api.github.com/search/code` | `net/http` | WIRED | Endpoint literal present; Retry-After + 401/403/429 handling implemented; unit test uses httptest. | +| `pkg/dorks/count_test.go` | `pkg/dorks.NewRegistry()` | direct call on embedded FS | WIRED | All 4 guardrail tests pass against real embedded corpus. | + +### Data-Flow Trace (Level 4) + +| Artifact | Data Variable | Source | Produces Real Data | Status | +|----------|---------------|--------|---------------------|--------| +| `cmd/dorks.go list` | registry.List() + db.ListCustomDorks() | embedded YAML + sqlite custom_dorks | Yes (150 embedded observed; custom row observed after add) | FLOWING | +| `cmd/dorks.go export` | merged registry+custom list | same | Yes (JSON contained both) | FLOWING | +| `cmd/dorks.go run` | dispatch to Executor.Execute | live HTTP to api.github.com/search/code | Real path exercised (auth gate reached) | FLOWING | + +### Behavioral Spot-Checks + +| Behavior | Command | Result | Status | +|----------|---------|--------|--------| +| Guardrail tests | `go test -run TestDorkCountGuardrail ./pkg/dorks/` | PASS | PASS | +| Full package tests | `go test ./pkg/dorks/... ./pkg/storage/... ./cmd/...` | ok (all 3 packages) | PASS | +| Binary builds | `go build -o /tmp/keyhunter .` | success | PASS | +| List 150 dorks | `keyhunter dorks list` | 150 data rows + header + summary line | PASS | +| Filter by source+category | `keyhunter dorks list --source=github --category=frontier` | 15 rows, all github/frontier | PASS | +| Run hits GitHubExecutor | `keyhunter dorks run --source=github --category=frontier` | ErrMissingAuth with setup hint from github.go | PASS | +| Export JSON | `keyhunter dorks export --format=json` | Valid JSON array starting at bing entries | PASS | +| Add persists | `keyhunter dorks add --source=google --category=frontier --id=custom-verify-test ...` | "Added custom dork 'custom-verify-test' (row 1)" | PASS | +| Custom shows in list | `keyhunter dorks list` after add | `*custom-verify-test` row present | PASS | +| Custom in export | `keyhunter dorks export --format=json` after add | 2 matches for custom-verify-test | PASS | + +### Requirements Coverage + +| Requirement | Source Plan | Description | Status | Evidence | +|-------------|------------|-------------|--------|----------| +| DORK-01 | 08-01, 08-02, 08-03, 08-04 | YAML schema + 150 embedded dorks | SATISFIED | Schema exists; 150 dorks load via guardrail test + binary. | +| DORK-02 | 08-02, 08-03, 08-04, 08-05, 08-07 | 150+ threshold + GitHub executor | SATISFIED | Count test passes; GitHubExecutor live-wired. | +| DORK-03 | 08-01, 08-06 | Custom dork CRUD via CLI + storage | SATISFIED | Live add/list/export round-trip verified. | +| DORK-04 | 08-02, 08-03, 08-04, 08-06 | List/export/manage dorks across 8 sources | SATISFIED | All 8 sources present; list/export/run/add/delete subcommands registered and functional. | + +### Anti-Patterns Found + +None. No TODO/FIXME/placeholder strings in phase files, no stub returns, no hardcoded empty results. The `dorks run --source=shodan` path intentionally returns `ErrSourceNotImplemented` per plan 08-06 design (GitHub is the only live executor in this phase). + +### Human Verification Required + +None — all 4 Success Criteria verified via built binary + live commands. Live GitHub API execution with a real token is optional follow-up but not required for goal achievement; the wiring to api.github.com is proven by the auth error surfaced through the real code path and by the httptest suite in `pkg/dorks/github_test.go`. + +### Gaps Summary + +No gaps. All 4 Success Criteria verified by running the built binary; the 150-dork corpus is locked by a guardrail test; per-source minimums, category coverage, and ID uniqueness are enforced by additional tests; custom dork CRUD round-trip observed end-to-end; GitHubExecutor reached via `dorks run` and its error contract honored. Phase 08 goal achieved. + +--- + +_Verified: 2026-04-05_ +_Verifier: Claude (gsd-verifier)_