79ec763233
- 18-02-SUMMARY.md with 2 task commits - STATE.md updated with position and decisions - Requirements WEB-03, WEB-09, WEB-11 marked complete
132 lines
4.9 KiB
Markdown
132 lines
4.9 KiB
Markdown
---
|
|
phase: 18-web-dashboard
|
|
plan: 02
|
|
subsystem: api
|
|
tags: [chi, rest-api, sse, json, http, server-sent-events]
|
|
|
|
requires:
|
|
- phase: 01-foundation
|
|
provides: "storage DB, providers registry, encryption"
|
|
- phase: 08-dork-engine
|
|
provides: "dorks registry and custom dork storage"
|
|
- phase: 09-osint-infrastructure
|
|
provides: "recon engine"
|
|
provides:
|
|
- "REST API at /api/v1/* for keys, providers, scan, recon, dorks, config"
|
|
- "SSE hub for live scan/recon progress streaming"
|
|
- "Server struct with dependency injection for all web handlers"
|
|
affects: [18-web-dashboard, serve-command]
|
|
|
|
tech-stack:
|
|
added: [chi-v5]
|
|
patterns: [api-json-wrappers, sse-hub-broadcast, dependency-injected-server]
|
|
|
|
key-files:
|
|
created:
|
|
- pkg/web/server.go
|
|
- pkg/web/api.go
|
|
- pkg/web/sse.go
|
|
- pkg/web/api_test.go
|
|
- pkg/web/sse_test.go
|
|
modified:
|
|
- pkg/storage/schema.sql
|
|
- go.mod
|
|
- go.sum
|
|
|
|
key-decisions:
|
|
- "JSON wrapper structs (apiKey, apiProvider, apiDork) with explicit JSON tags since domain structs only have yaml tags"
|
|
- "API never exposes raw key values -- KeyValue always empty string in JSON responses"
|
|
- "Single SSEHub shared between scan and recon progress endpoints, events distinguished by Type prefix"
|
|
|
|
patterns-established:
|
|
- "API wrapper pattern: domain structs -> apiX structs with JSON tags for consistent camelCase API"
|
|
- "writeJSON/readJSON helpers for DRY HTTP response handling"
|
|
- "ServerConfig struct for dependency injection into all web handlers"
|
|
|
|
requirements-completed: [WEB-03, WEB-09, WEB-11]
|
|
|
|
duration: 7min
|
|
completed: 2026-04-06
|
|
---
|
|
|
|
# Phase 18 Plan 02: REST API + SSE Hub Summary
|
|
|
|
**Complete REST API at /api/v1/* with 14 endpoints (keys, providers, scan, recon, dorks, config) plus SSE hub for live event streaming**
|
|
|
|
## Performance
|
|
|
|
- **Duration:** 7 min
|
|
- **Started:** 2026-04-06T14:59:58Z
|
|
- **Completed:** 2026-04-06T15:06:51Z
|
|
- **Tasks:** 2
|
|
- **Files modified:** 7
|
|
|
|
## Accomplishments
|
|
- Full REST API with 14 endpoints covering stats, keys CRUD, providers, scan/recon triggers, dorks, and config
|
|
- SSE hub with subscribe/unsubscribe/broadcast lifecycle and non-blocking buffered channels
|
|
- 23 passing tests (16 API + 7 SSE) covering happy paths and error cases
|
|
|
|
## Task Commits
|
|
|
|
Each task was committed atomically:
|
|
|
|
1. **Task 1: REST API handlers for /api/v1/*** - `76601b1` (feat)
|
|
2. **Task 2: SSE hub for live scan/recon progress** - `d557c73` (feat)
|
|
|
|
## Files Created/Modified
|
|
- `pkg/web/server.go` - Server struct with ServerConfig dependency injection
|
|
- `pkg/web/api.go` - All 14 REST API handlers with JSON wrapper types
|
|
- `pkg/web/sse.go` - SSEHub with Subscribe/Unsubscribe/Broadcast + HTTP handlers
|
|
- `pkg/web/api_test.go` - 16 tests for all API endpoints
|
|
- `pkg/web/sse_test.go` - 7 tests for SSE hub lifecycle and HTTP streaming
|
|
- `pkg/storage/schema.sql` - Resolved merge conflict (HEAD version kept)
|
|
- `go.mod` / `go.sum` - Added chi v5.2.5
|
|
|
|
## Decisions Made
|
|
- JSON wrapper structs (apiKey, apiProvider, apiDork) with explicit JSON tags since domain structs only have yaml tags -- ensures consistent camelCase JSON API
|
|
- API never exposes raw key values -- KeyValue always empty string in JSON responses for security
|
|
- Single SSEHub shared between scan and recon progress endpoints, events distinguished by Type prefix (scan:*, recon:*)
|
|
- DisallowUnknownFields removed from readJSON to avoid overly strict request parsing
|
|
|
|
## Deviations from Plan
|
|
|
|
### Auto-fixed Issues
|
|
|
|
**1. [Rule 3 - Blocking] Resolved merge conflict in schema.sql**
|
|
- **Found during:** Task 1
|
|
- **Issue:** schema.sql had unresolved git merge conflict markers between two versions of scheduled_jobs table
|
|
- **Fix:** Kept HEAD version (includes subscribers table + scheduled_jobs with scan_command column) and added missing index
|
|
- **Files modified:** pkg/storage/schema.sql
|
|
- **Verification:** All tests pass with resolved schema
|
|
- **Committed in:** 76601b1
|
|
|
|
**2. [Rule 1 - Bug] Added JSON wrapper structs for domain types**
|
|
- **Found during:** Task 1
|
|
- **Issue:** Provider, Dork, and Finding structs only have yaml tags -- json.Marshal would produce PascalCase field names inconsistent with REST API conventions
|
|
- **Fix:** Created apiKey, apiProvider, apiDork structs with explicit JSON tags and converter functions
|
|
- **Files modified:** pkg/web/api.go
|
|
- **Verification:** Tests check exact JSON field names (providerName, name, etc.)
|
|
- **Committed in:** 76601b1
|
|
|
|
---
|
|
|
|
**Total deviations:** 2 auto-fixed (1 blocking, 1 bug)
|
|
**Impact on plan:** Both fixes necessary for correctness. No scope creep.
|
|
|
|
## Issues Encountered
|
|
None beyond the auto-fixed deviations above.
|
|
|
|
## User Setup Required
|
|
None - no external service configuration required.
|
|
|
|
## Known Stubs
|
|
None - all endpoints are fully wired to their backing registries/database.
|
|
|
|
## Next Phase Readiness
|
|
- REST API and SSE infrastructure ready for Plan 18-03 (HTML pages with htmx consuming these endpoints)
|
|
- Server struct ready to be wired into cmd/serve.go
|
|
|
|
---
|
|
*Phase: 18-web-dashboard*
|
|
*Completed: 2026-04-06*
|