docs(phase-08): complete phase execution
This commit is contained in:
salvacybersec
@@ -4,13 +4,13 @@ milestone: v1.0
|
|||||||
milestone_name: milestone
|
milestone_name: milestone
|
||||||
status: executing
|
status: executing
|
||||||
stopped_at: Completed 08-07-PLAN.md
|
stopped_at: Completed 08-07-PLAN.md
|
||||||
last_updated: "2026-04-05T21:25:47.473Z"
|
last_updated: "2026-04-05T21:32:47.810Z"
|
||||||
last_activity: 2026-04-05
|
last_activity: 2026-04-05
|
||||||
progress:
|
progress:
|
||||||
total_phases: 18
|
total_phases: 18
|
||||||
completed_phases: 7
|
completed_phases: 8
|
||||||
total_plans: 47
|
total_plans: 47
|
||||||
completed_plans: 46
|
completed_plans: 47
|
||||||
percent: 20
|
percent: 20
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -25,8 +25,8 @@ See: .planning/PROJECT.md (updated 2026-04-04)
|
|||||||
|
|
||||||
## Current Position
|
## Current Position
|
||||||
|
|
||||||
Phase: 08 (dork-engine) — EXECUTING
|
Phase: 9
|
||||||
Plan: 4 of 7
|
Plan: Not started
|
||||||
Status: Ready to execute
|
Status: Ready to execute
|
||||||
Last activity: 2026-04-05
|
Last activity: 2026-04-05
|
||||||
|
|
||||||
|
|||||||
111
.planning/phases/08-dork-engine/08-VERIFICATION.md
Normal file
111
.planning/phases/08-dork-engine/08-VERIFICATION.md
Normal file
@@ -0,0 +1,111 @@
|
|||||||
|
---
|
||||||
|
phase: 08-dork-engine
|
||||||
|
verified: 2026-04-05T00:00:00Z
|
||||||
|
status: passed
|
||||||
|
score: 4/4 must-haves verified
|
||||||
|
---
|
||||||
|
|
||||||
|
# Phase 08: Dork Engine Verification Report
|
||||||
|
|
||||||
|
**Phase Goal:** Users can run, manage, and extend a library of 150+ built-in YAML dorks across GitHub, Google, Shodan, Censys, ZoomEye, FOFA, GitLab, and Bing.
|
||||||
|
**Verified:** 2026-04-05
|
||||||
|
**Status:** passed
|
||||||
|
**Re-verification:** No — initial verification
|
||||||
|
|
||||||
|
## Goal Achievement
|
||||||
|
|
||||||
|
### Observable Truths (from Success Criteria + plan must_haves)
|
||||||
|
|
||||||
|
| # | Truth | Status | Evidence |
|
||||||
|
|---|-------|--------|----------|
|
||||||
|
| 1 | `keyhunter dorks list` shows 150+ built-in dorks with source and category columns | VERIFIED | Built binary; `dorks list` prints 150 data rows + header + summary. Columns ID / SOURCE / CATEGORY / NAME present. |
|
||||||
|
| 2 | `keyhunter dorks run --source=github --category=frontier` executes against GitHub code search | VERIFIED | Command reaches `dorks.GitHubExecutor` and returns `ErrMissingAuth` with setup hint when GITHUB_TOKEN unset — i.e. wired all the way through; `pkg/dorks/github.go` targets `api.github.com/search/code` and handles Retry-After. |
|
||||||
|
| 3 | `keyhunter dorks add --source=google --query=... --description=...` persists a custom dork | VERIFIED | Live add created row 1, list shows `*custom-verify-test` (custom marker), export includes it. Round-trip through `storage.SaveCustomDork` / `ListCustomDorks`. |
|
||||||
|
| 4 | `keyhunter dorks export --format=json` exports all dorks including custom | VERIFIED | JSON export emitted with embedded + custom entries; custom-verify-test appears in output. |
|
||||||
|
| 5 | Embedded corpus holds >= 150 dorks across 8 sources and 5 categories | VERIFIED | Exact count = 150. Sources: github 50, google 30, shodan 20, censys 15, zoomeye 10, fofa 10, gitlab 10, bing 5. Categories frontier/specialized/infrastructure/emerging/enterprise all present. |
|
||||||
|
| 6 | Guardrail test locks 150 threshold + per-source minimums + ID uniqueness | VERIFIED | `TestDorkCountGuardrail`, `TestDorkCountPerSource`, `TestDorkCategoriesPresent`, `TestDorkIDsUnique` all PASS. |
|
||||||
|
|
||||||
|
**Score:** 6/6 truths verified (success criteria 1-4 plus underlying corpus/guardrail truths)
|
||||||
|
|
||||||
|
### Required Artifacts
|
||||||
|
|
||||||
|
| Artifact | Expected | Status | Details |
|
||||||
|
|----------|----------|--------|---------|
|
||||||
|
| `pkg/dorks/schema.go` | Dork struct + Validate | VERIFIED | Present, used by loader/tests. |
|
||||||
|
| `pkg/dorks/loader.go` | go:embed walker | VERIFIED | `//go:embed definitions/*` on embed.FS. |
|
||||||
|
| `pkg/dorks/registry.go` | Registry with List/Get/Stats/ListBySource/ListByCategory | VERIFIED | `NewRegistry()` exercised by tests + CLI. |
|
||||||
|
| `pkg/dorks/executor.go` | Executor interface + ErrSourceNotImplemented | VERIFIED | Dispatched from cmd/dorks.go run. |
|
||||||
|
| `pkg/dorks/github.go` | GitHubExecutor hitting api.github.com/search/code | VERIFIED | Auth handling, Retry-After parsing, ErrMissingAuth all present. |
|
||||||
|
| `pkg/dorks/count_test.go` | Guardrail test | VERIFIED | 4 guardrail tests (count, per-source, categories, unique IDs). |
|
||||||
|
| `pkg/dorks/definitions/github/*.yaml` | >= 50 GitHub dorks across 5 categories | VERIFIED | 50 entries across frontier/specialized/infrastructure/emerging/enterprise. |
|
||||||
|
| `pkg/dorks/definitions/google/*.yaml` | >= 30 Google dorks | VERIFIED | 30 entries across 3 category files. |
|
||||||
|
| `pkg/dorks/definitions/shodan/*.yaml` | >= 20 Shodan dorks | VERIFIED | 20 entries (frontier 6 + infrastructure 14). |
|
||||||
|
| `pkg/dorks/definitions/censys/all.yaml` | >= 15 | VERIFIED | 15 entries. |
|
||||||
|
| `pkg/dorks/definitions/zoomeye/all.yaml` | >= 10 | VERIFIED | 10 entries. |
|
||||||
|
| `pkg/dorks/definitions/fofa/all.yaml` | >= 10 | VERIFIED | 10 entries. |
|
||||||
|
| `pkg/dorks/definitions/gitlab/all.yaml` | >= 10 | VERIFIED | 10 entries. |
|
||||||
|
| `pkg/dorks/definitions/bing/all.yaml` | >= 5 | VERIFIED | 5 entries. |
|
||||||
|
| `pkg/storage/custom_dorks.go` | Save/List/Delete/Get CustomDork | VERIFIED | All CRUD referenced from cmd/dorks.go and exercised in live test. |
|
||||||
|
| `pkg/storage/schema.sql` | custom_dorks table DDL | VERIFIED | `CREATE TABLE IF NOT EXISTS custom_dorks` at line 44. |
|
||||||
|
| `cmd/dorks.go` | dorks command tree (list/info/export/run/add/delete) | VERIFIED | All 6 subcommands registered via `dorksCmd.AddCommand`. |
|
||||||
|
|
||||||
|
### Key Link Verification
|
||||||
|
|
||||||
|
| From | To | Via | Status | Details |
|
||||||
|
|------|----|-----|--------|---------|
|
||||||
|
| `pkg/dorks/loader.go` | `definitions/*/*.yaml` | `//go:embed definitions/*` | WIRED | Loader walks embed.FS subdirs; 150 dorks load at runtime (verified via binary + guardrail test). |
|
||||||
|
| `cmd/dorks.go` | `pkg/dorks.Registry` | `dorks.NewRegistry()` | WIRED | Called in 5 subcommand handlers (list, info, export, run, add validation, delete). |
|
||||||
|
| `cmd/dorks.go run` | `pkg/dorks.GitHubExecutor` | `NewGitHubExecutor(viper.GetString("dorks.github.token"))` | WIRED | Observed in live run: executor produced its own auth error message, confirming the call path reaches github.go. |
|
||||||
|
| `cmd/dorks.go add/delete` | `storage.DB custom_dorks` | `SaveCustomDork` / `DeleteCustomDork` / `ListCustomDorks` | WIRED | Live add returned `row 1`; list+export surfaced the custom entry. |
|
||||||
|
| `pkg/storage/schema.sql` | custom_dorks table | `CREATE TABLE IF NOT EXISTS custom_dorks` | WIRED | Table created on DB init; round-trip verified. |
|
||||||
|
| `pkg/dorks/github.go` | `https://api.github.com/search/code` | `net/http` | WIRED | Endpoint literal present; Retry-After + 401/403/429 handling implemented; unit test uses httptest. |
|
||||||
|
| `pkg/dorks/count_test.go` | `pkg/dorks.NewRegistry()` | direct call on embedded FS | WIRED | All 4 guardrail tests pass against real embedded corpus. |
|
||||||
|
|
||||||
|
### Data-Flow Trace (Level 4)
|
||||||
|
|
||||||
|
| Artifact | Data Variable | Source | Produces Real Data | Status |
|
||||||
|
|----------|---------------|--------|---------------------|--------|
|
||||||
|
| `cmd/dorks.go list` | registry.List() + db.ListCustomDorks() | embedded YAML + sqlite custom_dorks | Yes (150 embedded observed; custom row observed after add) | FLOWING |
|
||||||
|
| `cmd/dorks.go export` | merged registry+custom list | same | Yes (JSON contained both) | FLOWING |
|
||||||
|
| `cmd/dorks.go run` | dispatch to Executor.Execute | live HTTP to api.github.com/search/code | Real path exercised (auth gate reached) | FLOWING |
|
||||||
|
|
||||||
|
### Behavioral Spot-Checks
|
||||||
|
|
||||||
|
| Behavior | Command | Result | Status |
|
||||||
|
|----------|---------|--------|--------|
|
||||||
|
| Guardrail tests | `go test -run TestDorkCountGuardrail ./pkg/dorks/` | PASS | PASS |
|
||||||
|
| Full package tests | `go test ./pkg/dorks/... ./pkg/storage/... ./cmd/...` | ok (all 3 packages) | PASS |
|
||||||
|
| Binary builds | `go build -o /tmp/keyhunter .` | success | PASS |
|
||||||
|
| List 150 dorks | `keyhunter dorks list` | 150 data rows + header + summary line | PASS |
|
||||||
|
| Filter by source+category | `keyhunter dorks list --source=github --category=frontier` | 15 rows, all github/frontier | PASS |
|
||||||
|
| Run hits GitHubExecutor | `keyhunter dorks run --source=github --category=frontier` | ErrMissingAuth with setup hint from github.go | PASS |
|
||||||
|
| Export JSON | `keyhunter dorks export --format=json` | Valid JSON array starting at bing entries | PASS |
|
||||||
|
| Add persists | `keyhunter dorks add --source=google --category=frontier --id=custom-verify-test ...` | "Added custom dork 'custom-verify-test' (row 1)" | PASS |
|
||||||
|
| Custom shows in list | `keyhunter dorks list` after add | `*custom-verify-test` row present | PASS |
|
||||||
|
| Custom in export | `keyhunter dorks export --format=json` after add | 2 matches for custom-verify-test | PASS |
|
||||||
|
|
||||||
|
### Requirements Coverage
|
||||||
|
|
||||||
|
| Requirement | Source Plan | Description | Status | Evidence |
|
||||||
|
|-------------|------------|-------------|--------|----------|
|
||||||
|
| DORK-01 | 08-01, 08-02, 08-03, 08-04 | YAML schema + 150 embedded dorks | SATISFIED | Schema exists; 150 dorks load via guardrail test + binary. |
|
||||||
|
| DORK-02 | 08-02, 08-03, 08-04, 08-05, 08-07 | 150+ threshold + GitHub executor | SATISFIED | Count test passes; GitHubExecutor live-wired. |
|
||||||
|
| DORK-03 | 08-01, 08-06 | Custom dork CRUD via CLI + storage | SATISFIED | Live add/list/export round-trip verified. |
|
||||||
|
| DORK-04 | 08-02, 08-03, 08-04, 08-06 | List/export/manage dorks across 8 sources | SATISFIED | All 8 sources present; list/export/run/add/delete subcommands registered and functional. |
|
||||||
|
|
||||||
|
### Anti-Patterns Found
|
||||||
|
|
||||||
|
None. No TODO/FIXME/placeholder strings in phase files, no stub returns, no hardcoded empty results. The `dorks run --source=shodan` path intentionally returns `ErrSourceNotImplemented` per plan 08-06 design (GitHub is the only live executor in this phase).
|
||||||
|
|
||||||
|
### Human Verification Required
|
||||||
|
|
||||||
|
None — all 4 Success Criteria verified via built binary + live commands. Live GitHub API execution with a real token is optional follow-up but not required for goal achievement; the wiring to api.github.com is proven by the auth error surfaced through the real code path and by the httptest suite in `pkg/dorks/github_test.go`.
|
||||||
|
|
||||||
|
### Gaps Summary
|
||||||
|
|
||||||
|
No gaps. All 4 Success Criteria verified by running the built binary; the 150-dork corpus is locked by a guardrail test; per-source minimums, category coverage, and ID uniqueness are enforced by additional tests; custom dork CRUD round-trip observed end-to-end; GitHubExecutor reached via `dorks run` and its error contract honored. Phase 08 goal achieved.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
_Verified: 2026-04-05_
|
||||||
|
_Verifier: Claude (gsd-verifier)_
|
||||||
Reference in New Issue
Block a user