--- phase: 18-web-dashboard plan: 03 type: execute wave: 2 depends_on: ["18-01", "18-02"] files_modified: - pkg/web/templates/keys.html - pkg/web/templates/providers.html - pkg/web/templates/recon.html - pkg/web/templates/dorks.html - pkg/web/templates/settings.html - pkg/web/templates/scan.html - pkg/web/handlers.go - pkg/web/server.go - cmd/serve.go - pkg/web/handlers_test.go autonomous: false requirements: [WEB-03, WEB-04, WEB-05, WEB-06, WEB-07, WEB-08] must_haves: truths: - "User can browse keys with filtering, click Reveal to unmask, click Copy" - "User can view provider list with statistics" - "User can launch recon sweep from web UI and see live results via SSE" - "User can view and manage dorks" - "User can view and edit settings" - "User can trigger scan from web UI and see live progress" - "keyhunter serve --port=8080 starts full web dashboard" artifacts: - path: "pkg/web/templates/keys.html" provides: "Keys listing page with filter, reveal, copy" - path: "pkg/web/templates/providers.html" provides: "Provider listing with stats" - path: "pkg/web/templates/recon.html" provides: "Recon launcher with SSE live results" - path: "pkg/web/templates/dorks.html" provides: "Dork listing and management" - path: "pkg/web/templates/settings.html" provides: "Config editor" - path: "pkg/web/templates/scan.html" provides: "Scan launcher with SSE live progress" - path: "cmd/serve.go" provides: "HTTP server wired into CLI" key_links: - from: "pkg/web/templates/keys.html" to: "/api/v1/keys" via: "htmx hx-get for filtering and pagination" pattern: "hx-get.*api/v1/keys" - from: "pkg/web/templates/recon.html" to: "/api/v1/recon/progress" via: "EventSource SSE connection" pattern: "EventSource.*recon/progress" - from: "pkg/web/templates/scan.html" to: "/api/v1/scan/progress" via: "EventSource SSE connection" pattern: "EventSource.*scan/progress" - from: "cmd/serve.go" to: "pkg/web" via: "web.NewServer(cfg) + ListenAndServe" pattern: "web\\.NewServer" --- Create all remaining HTML pages (keys, providers, recon, dorks, scan, settings) using htmx for interactivity and SSE for live updates, then wire the HTTP server into cmd/serve.go so `keyhunter serve` launches the full dashboard. Purpose: Completes the user-facing web dashboard and makes it accessible via the CLI. Output: Full dashboard with all pages + cmd/serve.go wiring. @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md @.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/18-web-dashboard/18-CONTEXT.md @.planning/phases/18-web-dashboard/18-01-SUMMARY.md @.planning/phases/18-web-dashboard/18-02-SUMMARY.md ```go // pkg/web/server.go type Config struct { DB *storage.DB EncKey []byte Providers *providers.Registry Dorks *dorks.Registry ReconEngine *recon.Engine Port int AuthUser string AuthPass string AuthToken string } type Server struct { router chi.Router; cfg Config; tmpl *template.Template; sse *SSEHub } func NewServer(cfg Config) (*Server, error) func (s *Server) ListenAndServe() error func (s *Server) Router() chi.Router ``` ```go // pkg/web/embed.go var staticFiles embed.FS // //go:embed static/* var templateFiles embed.FS // //go:embed templates/* ``` ```go // pkg/web/auth.go func AuthMiddleware(user, pass, token string) func(http.Handler) http.Handler ``` ```go // pkg/web/api.go func (s *Server) mountAPI(r chi.Router) // mounts /api/v1/* func writeJSON(w http.ResponseWriter, status int, v interface{}) ``` ```go // pkg/web/sse.go type SSEHub struct { ... } func NewSSEHub() *SSEHub func (h *SSEHub) Broadcast(evt SSEEvent) type SSEEvent struct { Type string; Data interface{} } ``` ```go var servePort int var serveTelegram bool var serveCmd = &cobra.Command{ Use: "serve", ... } // Currently only starts Telegram bot — needs HTTP server wiring ``` ```go func openDBWithKey() (*storage.DB, []byte, error) // returns DB + encryption key ``` Task 1: HTML pages with htmx interactivity + page handlers pkg/web/templates/keys.html, pkg/web/templates/providers.html, pkg/web/templates/recon.html, pkg/web/templates/dorks.html, pkg/web/templates/settings.html, pkg/web/templates/scan.html, pkg/web/handlers.go, pkg/web/server.go, pkg/web/handlers_test.go 1. **keys.html** — extends layout (WEB-04): - Filter bar: provider dropdown (populated server-side from registry), confidence dropdown, text filter. Use `hx-get="/keys" hx-target="#keys-table" hx-include="[name='provider'],[name='confidence']"` for htmx-driven filtering. - Keys table: ID, Provider, Masked Key, Source, Confidence, Verified, Date columns - "Reveal" button per row: uses a small inline script or htmx `hx-get="/api/v1/keys/{id}"` that replaces the masked value cell. Since API always returns masked, the Reveal button uses a `data-key` attribute with the masked key from server render; for actual reveal, a dedicated handler `/keys/{id}/reveal` renders the unmasked key value (server-side, not API — the web dashboard can show unmasked to authenticated users). - "Copy" button: `navigator.clipboard.writeText()` on the revealed key value - "Delete" button: `hx-delete="/api/v1/keys/{id}" hx-confirm="Delete this key?" hx-target="closest tr" hx-swap="outerHTML"` — removes row on success - Pagination: "Load more" button via `hx-get="/keys?offset=N" hx-target="#keys-table" hx-swap="beforeend"` 2. **providers.html** — extends layout (WEB-06): - Stats summary bar: total count, per-category counts in badges - Provider table: Name, Category, Confidence, Keywords count, Has Verify - Filter by category via htmx dropdown - Click provider name -> expand row with details (patterns, verify endpoint) via `hx-get="/api/v1/providers/{name}" hx-target="#detail-{name}"` 3. **scan.html** — extends layout (WEB-03): - Form: Path input, verify checkbox, workers number input - "Start Scan" button: `hx-post="/api/v1/scan"` with JSON body, shows progress section - Progress section (hidden until scan starts): connects to SSE via inline script: `const es = new EventSource('/api/v1/scan/progress');` `es.addEventListener('scan:finding', (e) => { /* append row */ });` `es.addEventListener('scan:complete', (e) => { es.close(); });` - Results table: populated live via SSE events 4. **recon.html** — extends layout (WEB-05): - Source checkboxes: populated from `recon.Engine.List()`, grouped by category - Query input, stealth toggle, respect-robots toggle - "Sweep" button: `hx-post="/api/v1/recon"` triggers sweep - Live results via SSE (same pattern as scan.html with recon event types) - Results displayed as cards showing provider, masked key, source 5. **dorks.html** — extends layout (WEB-07): - Dork list table: ID, Source, Category, Query (truncated), Description - Filter by source dropdown - "Add Dork" form: source, category, query, description fields. `hx-post="/api/v1/dorks"` to create. - Stats bar: total dorks, per-source counts 6. **settings.html** — extends layout (WEB-08): - Config form populated from viper settings (rendered server-side) - Key fields: database path, encryption, telegram token (masked), default workers, verify timeout - "Save" button: `hx-put="/api/v1/config"` with form data as JSON - Success/error toast notification via htmx `hx-swap-oob` 7. **Update handlers.go** — add page handlers: - `handleKeys(w, r)` — render keys.html with initial data (first 50 findings, provider list for filter dropdown) - `handleKeyReveal(w, r)` — GET /keys/{id}/reveal — returns unmasked key value as HTML fragment (for htmx swap) - `handleProviders(w, r)` — render providers.html with provider list + stats - `handleScan(w, r)` — render scan.html - `handleRecon(w, r)` — render recon.html with source list - `handleDorks(w, r)` — render dorks.html with dork list + stats - `handleSettings(w, r)` — render settings.html with current config 8. **Update server.go** — register new routes in the router: - `GET /keys` -> handleKeys - `GET /keys/{id}/reveal` -> handleKeyReveal - `GET /providers` -> handleProviders - `GET /scan` -> handleScan - `GET /recon` -> handleRecon - `GET /dorks` -> handleDorks - `GET /settings` -> handleSettings 9. **Create handlers_test.go**: - Test each page handler returns 200 with expected content - Test keys page contains "keys-table" div - Test providers page lists provider names - Test key reveal returns unmasked value cd /home/salva/Documents/apikey && go test ./pkg/web/... -v -count=1 All 6 page templates render correctly, htmx attributes are present for interactive features, SSE JavaScript is embedded in scan and recon pages, page handlers serve data from real packages, all tests pass Task 2: Wire HTTP server into cmd/serve.go cmd/serve.go 1. Update cmd/serve.go RunE function: - Import `github.com/salvacybersec/keyhunter/pkg/web` - Import `github.com/salvacybersec/keyhunter/pkg/dorks` - After existing DB/provider/recon setup, create web server: ```go reg, err := providers.NewRegistry() dorkReg, err := dorks.NewRegistry() reconEng := recon.NewEngine() // ... (register recon sources if needed) srv, err := web.NewServer(web.Config{ DB: db, EncKey: encKey, Providers: reg, Dorks: dorkReg, ReconEngine: reconEng, Port: servePort, AuthUser: viper.GetString("web.auth_user"), AuthPass: viper.GetString("web.auth_pass"), AuthToken: viper.GetString("web.auth_token"), }) ``` - Start HTTP server in a goroutine: `go srv.ListenAndServe()` - Keep existing Telegram bot start logic (conditioned on --telegram flag) - Update the port message: `fmt.Printf("KeyHunter dashboard running at http://localhost:%d\n", servePort)` - The existing `<-ctx.Done()` already handles graceful shutdown 2. Add serve flags: - `--no-web` flag (default false) to disable web dashboard (for telegram-only mode) - `--auth-user`, `--auth-pass`, `--auth-token` flags bound to viper `web.auth_user`, `web.auth_pass`, `web.auth_token` 3. Ensure the DB is opened unconditionally (it currently only opens when --telegram is set): - Move `openDBWithKey()` call before the telegram conditional - Both web server and telegram bot share the same DB instance cd /home/salva/Documents/apikey && go build -o /dev/null ./cmd/... && echo "build OK" `keyhunter serve` starts HTTP server on port 8080 with full dashboard, --telegram additionally starts bot, --port changes listen port, --auth-user/pass/token enable auth, `go build ./cmd/...` succeeds Task 3: Visual verification of complete web dashboard Human verifies the full dashboard renders and functions correctly in browser. cd /home/salva/Documents/apikey && go build -o /dev/null ./cmd/... && go test ./pkg/web/... -count=1 All pages render, navigation works, API returns JSON, server starts and stops cleanly Complete web dashboard: overview, keys (with reveal/copy/delete), providers, scan (with SSE live progress), recon (with SSE live results), dorks, and settings pages. HTTP server wired into `keyhunter serve`. 1. Run: `cd /home/salva/Documents/apikey && go run . serve --port=9090` 2. Open browser: http://localhost:9090 3. Verify overview page shows stat cards and navigation bar 4. Click "Keys" — verify table renders (may be empty if no scans done) 5. Click "Providers" — verify 108+ providers listed with categories 6. Click "Dorks" — verify dork list renders 7. Click "Settings" — verify config form renders 8. Test API: `curl http://localhost:9090/api/v1/stats` — verify JSON response 9. Test API: `curl http://localhost:9090/api/v1/providers | head -c 200` — verify provider JSON 10. Stop server with Ctrl+C — verify clean shutdown Type "approved" or describe issues - `go build ./cmd/...` compiles without errors - `go test ./pkg/web/... -v` — all tests pass - `keyhunter serve --port=9090` starts and serves dashboard at http://localhost:9090 - All 7 pages render (overview, keys, providers, scan, recon, dorks, settings) - Navigation links work - htmx interactions work (filtering, delete) - SSE streams work (scan and recon progress) - API endpoints return proper JSON - All 7 HTML pages render with proper layout and navigation - Keys page supports filtering, reveal, copy, delete via htmx - Scan and recon pages show live progress via SSE - Providers page shows 108+ providers with stats - Settings page reads/writes config - cmd/serve.go starts HTTP server + optional Telegram bot - Auth middleware protects dashboard when credentials configured After completion, create `.planning/phases/18-web-dashboard/18-03-SUMMARY.md`