keyhunter/.planning/phases/08-dork-engine/08-01-SUMMARY.md

phase, plan, subsystem, tags, requires, provides, affects, tech-stack, key-files, key-decisions, patterns-established, requirements-completed, duration, completed

phase	plan	subsystem	tags	requires	provides	affects	tech-stack	key-files	key-decisions	patterns-established	requirements-completed	duration	completed
08-dork-engine	01	dork-engine	dorks yaml go-embed sqlite registry executor	phase provides 01-foundation pkg/providers go:embed loader + Registry pattern, pkg/storage.Open + schema.sql migration harness	pkg/dorks package (schema, go:embed loader, Registry, Executor interface, Runner dispatcher) ErrSourceNotImplemented / ErrMissingAuth sentinel errors for per-source executors custom_dorks SQLite table with SaveCustomDork / ListCustomDorks / GetCustomDork / GetCustomDorkByDorkID / DeleteCustomDork CRUD pkg/dorks/definitions/ tree ready for Wave 2 YAML batches	08-02 08-03 08-04 08-05 08-06 08-07 09-osint 10-osint	added patterns go:embed dir/* directive tolerating empty tree via fs.SkipAll on ErrNotExist Registry mirroring pkg/providers: byID + bySource + byCategory index maps Runner pattern for per-source Executor dispatch with sentinel-error fallbacks JSON-serialised tags stored in SQLite TEXT column with rowScanner interface	created modified pkg/dorks/schema.go pkg/dorks/loader.go pkg/dorks/registry.go pkg/dorks/executor.go pkg/dorks/registry_test.go pkg/dorks/definitions/.gitkeep dorks/.gitkeep pkg/storage/custom_dorks.go pkg/storage/custom_dorks_test.go pkg/storage/schema.sql	Used //go:embed definitions/* (not definitions/*.yaml) so the placeholder .gitkeep keeps the embed directive valid pre-YAML Loader tolerates a definitions tree containing only .gitkeep by skipping non-.yaml entries and returning nil dorks without error Runner + Executor interface split (separate from Registry) so Plan 08-05 can wire GitHub live without touching registry internals custom_dorks.dork_id UNIQUE constraint at SQL layer prevents user duplicate IDs; embedded dork IDs are orthogonal (read-only) Tags serialised as JSON in a TEXT column (no join table) to keep CRUD trivial for a user-authored data set	Embedded YAML registries in KeyHunter follow the pkg/providers blueprint: schema.go + loader.go (go:embed) + registry.go with index maps + per-thing tests using NewRegistryFromX synthetic fixtures Per-source executor dispatch: interface + Runner.Register + ErrSourceNotImplemented sentinel for deferred backends	DORK-01 DORK-03	~15min	2026-04-05

Phase 08 Plan 01: Dork Engine Foundation Summary

pkg/dorks package mirroring the pkg/providers go:embed pattern with Registry, Executor interface, Runner dispatch, and custom_dorks SQLite CRUD — ready for Wave 2 YAML batches and the Plan 08-05 GitHub executor.

Performance

• Duration: ~15 min
• Started: 2026-04-05
• Completed: 2026-04-05
• Tasks: 2
• Files created: 9
• Files modified: 1

Accomplishments

• pkg/dorks package compiles and exposes NewRegistry, NewRegistryFromDorks, Get, List, ListBySource, ListByCategory, Stats, Runner, Executor, Match, ErrSourceNotImplemented, ErrMissingAuth
• go:embed definitions/* loader tolerates an empty tree (only .gitkeep), so Wave 2 plans can drop YAML files into pkg/dorks/definitions/{source}/ with zero loader changes
• custom_dorks table + CRUD fully exercised via round-trip tests on storage.Open(":memory:")
• All tests green, go build ./... clean, no regressions in existing storage tests

Task Commits

1. Task 1: pkg/dorks foundation (schema, loader, registry, executor, tests) — fd6efbb (feat)
2. Task 2: custom_dorks storage table + CRUD + tests — 01062b8 (feat)

Note: Tests were co-located with implementation in a single commit per task; NewRegistryFromDorks served as the synthetic-fixture entry point so the embedded FS was never required during testing.

Files Created/Modified

• pkg/dorks/schema.go — Dork struct, ValidSources/ValidCategories, Validate(), Stats
• pkg/dorks/loader.go — //go:embed definitions/* loader tolerating empty tree, validates every parsed dork, wraps errors with file path
• pkg/dorks/registry.go — Registry with byID, bySource, byCategory indexes and List/Get/ListBySource/ListByCategory/Stats
• pkg/dorks/executor.go — Match, Executor interface, Runner dispatcher, ErrSourceNotImplemented, ErrMissingAuth
• pkg/dorks/registry_test.go — 9 test functions covering registry methods, Validate, Runner dispatch, empty-tree NewRegistry
• pkg/dorks/definitions/.gitkeep — keeps embed directive valid pre-YAML
• dorks/.gitkeep — placeholder at repo root per plan layout
• pkg/storage/schema.sql — added custom_dorks CREATE TABLE + idx_custom_dorks_source + idx_custom_dorks_category
• pkg/storage/custom_dorks.go — CustomDork struct + SaveCustomDork / ListCustomDorks / GetCustomDork / GetCustomDorkByDorkID / DeleteCustomDork, JSON-encoded tags, shared rowScanner helper
• pkg/storage/custom_dorks_test.go — 7 tests: round-trip, newest-first list, not-found, by-dork-id lookup, delete + double-delete no-op, unique constraint, schema migration idempotency

Decisions Made

• definitions/* instead of definitions/*.yaml: The .yaml-only form fails compilation when the directory contains only .gitkeep. Using * and filtering by extension at walk time allows the foundation plan to ship with zero YAML and for Wave 2 plans to add files incrementally.
• Separate Runner from Registry: The executor plumbing is its own type so downstream plans (08-05 GitHub, 09-16 OSINT) can register implementations without coupling to the YAML index. Registry stays read-only and cache-safe.
• Tags as JSON TEXT, not a join table: A junction table for ~150 user-authored rows worth of tags is not justified. encoding/json round-trip is trivial and keeps the CRUD API flat.
• dork_id UNIQUE at SQL layer: Enforces user-visible ID uniqueness deterministically; tested via duplicate-insert failure.

Deviations from Plan

None — plan executed exactly as written. The only nuance was the deliberate use of //go:embed definitions/* instead of //go:embed definitions (the plan action step called out this exact contingency and the implementation picked the tolerant form).

Issues Encountered

None.

User Setup Required

None — no external service configuration required for this plan.

Next Phase Readiness

• Wave 2 plans (08-02 / 08-03 / 08-04 / 08-07) can drop YAML files into pkg/dorks/definitions/{source}/ and they will load automatically via NewRegistry()
• Plan 08-05 (GitHub live executor) can implement dorks.Executor and Runner.Register without modifying registry code
• Plan 08-06 (CLI dorks add/list/delete/info) has pkg/storage.CustomDork CRUD available immediately
• ErrSourceNotImplemented gives Plan 08-05's CLI a clean error path for non-GitHub sources until OSINT phases 9-16 arrive

Self-Check: PASSED

• pkg/dorks/schema.go — FOUND
• pkg/dorks/loader.go — FOUND
• pkg/dorks/registry.go — FOUND
• pkg/dorks/executor.go — FOUND
• pkg/dorks/registry_test.go — FOUND
• pkg/dorks/definitions/.gitkeep — FOUND
• dorks/.gitkeep — FOUND
• pkg/storage/schema.sql — FOUND (modified)
• pkg/storage/custom_dorks.go — FOUND
• pkg/storage/custom_dorks_test.go — FOUND
• commit fd6efbb — FOUND
• commit 01062b8 — FOUND
• go build ./... — PASSED
• go test ./pkg/dorks/... ./pkg/storage/... — PASSED

---

Phase: 08-dork-engine Completed: 2026-04-05