9.0 KiB
9.0 KiB
phase, verified, status, score
| phase | verified | status | score |
|---|---|---|---|
| 08-dork-engine | 2026-04-05T00:00:00Z | passed | 4/4 must-haves verified |
Phase 08: Dork Engine Verification Report
Phase Goal: Users can run, manage, and extend a library of 150+ built-in YAML dorks across GitHub, Google, Shodan, Censys, ZoomEye, FOFA, GitLab, and Bing. Verified: 2026-04-05 Status: passed Re-verification: No — initial verification
Goal Achievement
Observable Truths (from Success Criteria + plan must_haves)
| # | Truth | Status | Evidence |
|---|---|---|---|
| 1 | keyhunter dorks list shows 150+ built-in dorks with source and category columns |
VERIFIED | Built binary; dorks list prints 150 data rows + header + summary. Columns ID / SOURCE / CATEGORY / NAME present. |
| 2 | keyhunter dorks run --source=github --category=frontier executes against GitHub code search |
VERIFIED | Command reaches dorks.GitHubExecutor and returns ErrMissingAuth with setup hint when GITHUB_TOKEN unset — i.e. wired all the way through; pkg/dorks/github.go targets api.github.com/search/code and handles Retry-After. |
| 3 | keyhunter dorks add --source=google --query=... --description=... persists a custom dork |
VERIFIED | Live add created row 1, list shows *custom-verify-test (custom marker), export includes it. Round-trip through storage.SaveCustomDork / ListCustomDorks. |
| 4 | keyhunter dorks export --format=json exports all dorks including custom |
VERIFIED | JSON export emitted with embedded + custom entries; custom-verify-test appears in output. |
| 5 | Embedded corpus holds >= 150 dorks across 8 sources and 5 categories | VERIFIED | Exact count = 150. Sources: github 50, google 30, shodan 20, censys 15, zoomeye 10, fofa 10, gitlab 10, bing 5. Categories frontier/specialized/infrastructure/emerging/enterprise all present. |
| 6 | Guardrail test locks 150 threshold + per-source minimums + ID uniqueness | VERIFIED | TestDorkCountGuardrail, TestDorkCountPerSource, TestDorkCategoriesPresent, TestDorkIDsUnique all PASS. |
Score: 6/6 truths verified (success criteria 1-4 plus underlying corpus/guardrail truths)
Required Artifacts
| Artifact | Expected | Status | Details |
|---|---|---|---|
pkg/dorks/schema.go |
Dork struct + Validate | VERIFIED | Present, used by loader/tests. |
pkg/dorks/loader.go |
go:embed walker | VERIFIED | //go:embed definitions/* on embed.FS. |
pkg/dorks/registry.go |
Registry with List/Get/Stats/ListBySource/ListByCategory | VERIFIED | NewRegistry() exercised by tests + CLI. |
pkg/dorks/executor.go |
Executor interface + ErrSourceNotImplemented | VERIFIED | Dispatched from cmd/dorks.go run. |
pkg/dorks/github.go |
GitHubExecutor hitting api.github.com/search/code | VERIFIED | Auth handling, Retry-After parsing, ErrMissingAuth all present. |
pkg/dorks/count_test.go |
Guardrail test | VERIFIED | 4 guardrail tests (count, per-source, categories, unique IDs). |
pkg/dorks/definitions/github/*.yaml |
>= 50 GitHub dorks across 5 categories | VERIFIED | 50 entries across frontier/specialized/infrastructure/emerging/enterprise. |
pkg/dorks/definitions/google/*.yaml |
>= 30 Google dorks | VERIFIED | 30 entries across 3 category files. |
pkg/dorks/definitions/shodan/*.yaml |
>= 20 Shodan dorks | VERIFIED | 20 entries (frontier 6 + infrastructure 14). |
pkg/dorks/definitions/censys/all.yaml |
>= 15 | VERIFIED | 15 entries. |
pkg/dorks/definitions/zoomeye/all.yaml |
>= 10 | VERIFIED | 10 entries. |
pkg/dorks/definitions/fofa/all.yaml |
>= 10 | VERIFIED | 10 entries. |
pkg/dorks/definitions/gitlab/all.yaml |
>= 10 | VERIFIED | 10 entries. |
pkg/dorks/definitions/bing/all.yaml |
>= 5 | VERIFIED | 5 entries. |
pkg/storage/custom_dorks.go |
Save/List/Delete/Get CustomDork | VERIFIED | All CRUD referenced from cmd/dorks.go and exercised in live test. |
pkg/storage/schema.sql |
custom_dorks table DDL | VERIFIED | CREATE TABLE IF NOT EXISTS custom_dorks at line 44. |
cmd/dorks.go |
dorks command tree (list/info/export/run/add/delete) | VERIFIED | All 6 subcommands registered via dorksCmd.AddCommand. |
Key Link Verification
| From | To | Via | Status | Details |
|---|---|---|---|---|
pkg/dorks/loader.go |
definitions/*/*.yaml |
//go:embed definitions/* |
WIRED | Loader walks embed.FS subdirs; 150 dorks load at runtime (verified via binary + guardrail test). |
cmd/dorks.go |
pkg/dorks.Registry |
dorks.NewRegistry() |
WIRED | Called in 5 subcommand handlers (list, info, export, run, add validation, delete). |
cmd/dorks.go run |
pkg/dorks.GitHubExecutor |
NewGitHubExecutor(viper.GetString("dorks.github.token")) |
WIRED | Observed in live run: executor produced its own auth error message, confirming the call path reaches github.go. |
cmd/dorks.go add/delete |
storage.DB custom_dorks |
SaveCustomDork / DeleteCustomDork / ListCustomDorks |
WIRED | Live add returned row 1; list+export surfaced the custom entry. |
pkg/storage/schema.sql |
custom_dorks table | CREATE TABLE IF NOT EXISTS custom_dorks |
WIRED | Table created on DB init; round-trip verified. |
pkg/dorks/github.go |
https://api.github.com/search/code |
net/http |
WIRED | Endpoint literal present; Retry-After + 401/403/429 handling implemented; unit test uses httptest. |
pkg/dorks/count_test.go |
pkg/dorks.NewRegistry() |
direct call on embedded FS | WIRED | All 4 guardrail tests pass against real embedded corpus. |
Data-Flow Trace (Level 4)
| Artifact | Data Variable | Source | Produces Real Data | Status |
|---|---|---|---|---|
cmd/dorks.go list |
registry.List() + db.ListCustomDorks() | embedded YAML + sqlite custom_dorks | Yes (150 embedded observed; custom row observed after add) | FLOWING |
cmd/dorks.go export |
merged registry+custom list | same | Yes (JSON contained both) | FLOWING |
cmd/dorks.go run |
dispatch to Executor.Execute | live HTTP to api.github.com/search/code | Real path exercised (auth gate reached) | FLOWING |
Behavioral Spot-Checks
| Behavior | Command | Result | Status |
|---|---|---|---|
| Guardrail tests | go test -run TestDorkCountGuardrail ./pkg/dorks/ |
PASS | PASS |
| Full package tests | go test ./pkg/dorks/... ./pkg/storage/... ./cmd/... |
ok (all 3 packages) | PASS |
| Binary builds | go build -o /tmp/keyhunter . |
success | PASS |
| List 150 dorks | keyhunter dorks list |
150 data rows + header + summary line | PASS |
| Filter by source+category | keyhunter dorks list --source=github --category=frontier |
15 rows, all github/frontier | PASS |
| Run hits GitHubExecutor | keyhunter dorks run --source=github --category=frontier |
ErrMissingAuth with setup hint from github.go | PASS |
| Export JSON | keyhunter dorks export --format=json |
Valid JSON array starting at bing entries | PASS |
| Add persists | keyhunter dorks add --source=google --category=frontier --id=custom-verify-test ... |
"Added custom dork 'custom-verify-test' (row 1)" | PASS |
| Custom shows in list | keyhunter dorks list after add |
*custom-verify-test row present |
PASS |
| Custom in export | keyhunter dorks export --format=json after add |
2 matches for custom-verify-test | PASS |
Requirements Coverage
| Requirement | Source Plan | Description | Status | Evidence |
|---|---|---|---|---|
| DORK-01 | 08-01, 08-02, 08-03, 08-04 | YAML schema + 150 embedded dorks | SATISFIED | Schema exists; 150 dorks load via guardrail test + binary. |
| DORK-02 | 08-02, 08-03, 08-04, 08-05, 08-07 | 150+ threshold + GitHub executor | SATISFIED | Count test passes; GitHubExecutor live-wired. |
| DORK-03 | 08-01, 08-06 | Custom dork CRUD via CLI + storage | SATISFIED | Live add/list/export round-trip verified. |
| DORK-04 | 08-02, 08-03, 08-04, 08-06 | List/export/manage dorks across 8 sources | SATISFIED | All 8 sources present; list/export/run/add/delete subcommands registered and functional. |
Anti-Patterns Found
None. No TODO/FIXME/placeholder strings in phase files, no stub returns, no hardcoded empty results. The dorks run --source=shodan path intentionally returns ErrSourceNotImplemented per plan 08-06 design (GitHub is the only live executor in this phase).
Human Verification Required
None — all 4 Success Criteria verified via built binary + live commands. Live GitHub API execution with a real token is optional follow-up but not required for goal achievement; the wiring to api.github.com is proven by the auth error surfaced through the real code path and by the httptest suite in pkg/dorks/github_test.go.
Gaps Summary
No gaps. All 4 Success Criteria verified by running the built binary; the 150-dork corpus is locked by a guardrail test; per-source minimums, category coverage, and ID uniqueness are enforced by additional tests; custom dork CRUD round-trip observed end-to-end; GitHubExecutor reached via dorks run and its error contract honored. Phase 08 goal achieved.
Verified: 2026-04-05 Verifier: Claude (gsd-verifier)