pkg/web: chi v5 server with go:embed static assets, HTML templates,
14 REST API endpoints (/api/v1/*), SSE hub for live scan/recon progress,
optional basic/token auth middleware.
cmd/serve.go: keyhunter serve [--telegram] [--port=8080] starts web
dashboard + optional Telegram bot.
- Server struct with chi router, embedded template parsing, static file serving
- AuthMiddleware supports Basic and Bearer token with constant-time comparison
- Overview handler renders stats from providers/recon/storage when available
- Nil-safe: works with zero config (shows zeroes, no DB required)
- All 7 tests pass
- Test overview returns 200 with KeyHunter in body
- Test static asset serving for htmx.min.js
- Test auth returns 401 when configured but no credentials
- Test basic auth and bearer token pass through
- Test overview shows stat cards
- Add chi v5.2.5 to go.mod
- Vendor htmx v2.0.4 minified JS in pkg/web/static/
- Create go:embed directives for static/ and templates/
- Create layout.html with nav bar and Tailwind CDN
- Create overview.html with stat cards and findings table
- cmd/serve.go: starts scheduler, optionally starts Telegram bot with --telegram flag
- cmd/schedule.go: add/list/remove/run subcommands for scheduled scan job CRUD
- pkg/scheduler/: gocron v2 based scheduler with DB-backed jobs and scan execution
- pkg/storage/scheduled_jobs.go: scheduled_jobs table CRUD with tests
- Remove serve and schedule stubs from cmd/stubs.go
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Test extractArg parsing for all command formats
- Test isPrivateChat detection (private vs group vs supergroup)
- Test commandHelp contains all 8 commands with descriptions
- Test storageToEngine conversion fidelity
- Test New constructor wires startedAt correctly
- NotifyNewFindings sends to all subscribers on scan completion with findings
- NotifyFinding sends real-time individual finding notifications (always masked)
- formatNotification/formatErrorNotification/formatFindingNotification helpers
- Zero findings = no notification; errors get separate error format
- Per-subscriber error handling: log and continue on individual send failures
- 6 tests pass: subscribe DB round-trip, no-subscriber no-op, zero-finding skip,
message format validation, error format, masked key enforcement
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- TestSubscribeUnsubscribe: DB round-trip for add/remove subscriber
- TestNotifyNewFindings_NoSubscribers: zero messages with empty table
- TestNotifyNewFindings_ZeroFindings: no notification for 0 findings
- TestFormatNotification: message contains job name, count, duration
- TestFormatFindingNotification: masked key, never full key
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- handleSubscribe checks IsSubscribed, calls AddSubscriber with chat ID and username
- handleUnsubscribe calls RemoveSubscriber, reports rows affected
- Both use storage layer from Plan 17-02
- Removed stub implementations from bot.go
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add telego v1.8.0 as direct dependency for Telegram bot
- Create pkg/bot/bot.go with Bot struct, Config, New, Start, Stop
- Implement isAllowed chat authorization and per-user rate limiting
- Add command dispatch with handler stubs for all 10 commands
- Long polling lifecycle with context cancellation for graceful shutdown
- Scheduler wraps gocron with Start/Stop lifecycle
- Start loads enabled jobs from DB and registers cron schedules
- AddJob/RemoveJob persist to DB and sync with gocron
- RunJob for manual trigger with OnComplete callback
- JobResult struct for notification bridge
- Promote gocron/v2 v2.19.1 to direct dependency
- Storage round-trip test for SaveScheduledJob/ListScheduledJobs
- Subscriber round-trip test for Add/Remove/List/IsSubscribed
- Scheduler Start loads enabled jobs from DB
- Scheduler AddJob/RemoveJob persists and registers
- Scheduler RunJob manual trigger with callback
- Add gocron/v2 v2.19.1 as direct dependency
- Append subscribers and scheduled_jobs CREATE TABLE to schema.sql
- Implement full subscriber CRUD (Add/Remove/List/IsSubscribed)
- Implement full scheduled job CRUD (Save/List/Get/Delete/UpdateLastRun/SetEnabled)
- URLhausSource searches abuse.ch URLhaus API for malicious URLs with API keys
- Credentialless source (Enabled always true, no API key needed)
- Tag lookup with payload endpoint fallback
- ciLogKeyPattern used for content matching
- Tests with httptest mocks for happy path and empty results
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- RapidAPISource searches public API listings for leaked keys
- Scrapes HTML search pages with ciLogKeyPattern matching
- Credentialless, httptest-based tests
- PostmanSource searches public collections via internal search proxy
- SwaggerHubSource searches published API specs for embedded keys
- Both credentialless, use BuildQueries + ciLogKeyPattern
- httptest-based tests for both sources
- VirusTotalSource searches VT Intelligence API for files containing API keys
- IntelligenceXSource searches IX archive with 3-step flow (search/results/read)
- Both credential-gated (Enabled returns false without API key)
- ciLogKeyPattern used for content matching
- Tests with httptest mocks for happy path and empty results
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- APKMirrorSource searches APK metadata pages for key patterns
- CrtShSource discovers subdomains via CT logs and probes config endpoints
- Both credentialless, emit findings on ciLogKeyPattern match
- GrafanaSource: search dashboards via /api/search, fetch detail via /api/dashboards/uid
- SentrySource: search issues via /api/0/issues, fetch events for key detection
- Register all 5 log aggregator sources in RegisterAll (67 sources total)
- Tests use httptest mocks for each API endpoint
- ElasticsearchSource: POST _search API with query_string, parse hits._source
- KibanaSource: GET saved_objects/_find API with kbn-xsrf header
- SplunkSource: GET search/jobs/export API with newline-delimited JSON parsing
- All sources use ciLogKeyPattern for key detection
- Tests use httptest mocks for each API endpoint
- DiscordSource uses dorking approach against configurable search endpoint
- SlackSource uses dorking against slack-archive indexers
- DevToSource searches dev.to API articles list + detail for body_markdown
- RegisterAll extended to include all 6 Phase 15 forum sources
- All credentialless, use ciLogKeyPattern for key detection
- StackOverflowSource searches SE API v2.3 search/excerpts endpoint
- RedditSource searches Reddit JSON API with custom User-Agent
- HackerNewsSource searches Algolia HN API for comments
- All credentialless, use ciLogKeyPattern for key detection
- Tests use httptest mock servers with API key patterns
- ConfluenceSource searches exposed instances via /rest/api/content/search CQL
- GoogleDocsSource uses dorking + /export?format=txt for plain-text scanning
- HTML tag stripping for Confluence storage format
- Both credentialless, tests with httptest mocks confirm findings
- TrelloSource searches public Trello boards via /1/search API
- NotionSource uses dorking to discover and scrape public Notion pages
- Both credentialless, follow established Phase 10 pattern
- Tests with httptest mocks confirm Sweep emits findings
- Add CircleCIToken to SourcesConfig with env/viper lookup in cmd/recon.go
- Register 7 new sources: travisci, ghactions, circleci, jenkins, wayback, commoncrawl, jsbundle
- Update register_test.go expectations from 45 to 52 sources
- Add integration test handlers + registrations for all 12 Phase 14 sources
- Integration test now validates 52 sources end-to-end
- SwaggerSource probes OpenAPI doc endpoints for API keys in example/default fields
- DeployPreviewSource scans Vercel/Netlify preview URLs for __NEXT_DATA__ env leaks
- Both implement ReconSource, credentialless, with httptest-based tests
- GitHubActionsSource: searches GitHub code search for workflow files with provider keywords (token-gated)
- TravisCISource: queries Travis CI v3 API for public build logs (credentialless)
- CircleCISource: queries CircleCI v2 pipeline API for build pipelines (token-gated)
- JenkinsSource: queries open Jenkins /api/json for job build consoles (credentialless)
- GitLabCISource: queries GitLab projects API for CI-enabled projects (token-gated)
- RegisterAll extended to 45 sources (40 Phase 10-13 + 5 Phase 14)
- Integration test updated with fixtures for all 5 new sources
- cmd/recon.go wires CIRCLECI_TOKEN env var
