Commit Graph

375 Commits

Author SHA1 Message Date
salvacybersec
297ad3dc2b feat(16-03): add RapidAPI recon source
- RapidAPISource searches public API listings for leaked keys
- Scrapes HTML search pages with ciLogKeyPattern matching
- Credentialless, httptest-based tests
2026-04-06 16:44:57 +03:00
salvacybersec
edde02f3a2 feat(16-03): add Postman and SwaggerHub recon sources
- PostmanSource searches public collections via internal search proxy
- SwaggerHubSource searches published API specs for embedded keys
- Both credentialless, use BuildQueries + ciLogKeyPattern
- httptest-based tests for both sources
2026-04-06 16:44:47 +03:00
salvacybersec
e02bad69ba feat(16-01): add VirusTotal and IntelligenceX recon sources
- VirusTotalSource searches VT Intelligence API for files containing API keys
- IntelligenceXSource searches IX archive with 3-step flow (search/results/read)
- Both credential-gated (Enabled returns false without API key)
- ciLogKeyPattern used for content matching
- Tests with httptest mocks for happy path and empty results

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 16:44:41 +03:00
salvacybersec
09a8d4cb70 feat(16-02): add APKMirror and crt.sh ReconSource modules
- APKMirrorSource searches APK metadata pages for key patterns
- CrtShSource discovers subdomains via CT logs and probes config endpoints
- Both credentialless, emit findings on ciLogKeyPattern match
2026-04-06 16:44:37 +03:00
salvacybersec
8bcd9ebc18 docs(16): create phase plan 2026-04-06 16:42:15 +03:00
salvacybersec
5216b39826 chore: add .claude/ to gitignore 2026-04-06 16:37:54 +03:00
salvacybersec
af284f56f2 docs(phase-15): complete forums, collaboration, log aggregators 2026-04-06 16:37:48 +03:00
salvacybersec
83a1e83ae5 fix(phase-15): update register tests for 67 total sources (Phase 10-15) 2026-04-06 16:37:48 +03:00
salvacybersec
748efd6691 docs(15-03): complete log aggregator sources plan
- Elasticsearch, Grafana, Sentry, Kibana, Splunk sources implemented
- 5 sources + 5 test files, all passing
- Requirements RECON-LOG-01, RECON-LOG-02, RECON-LOG-03 complete
2026-04-06 16:33:01 +03:00
salvacybersec
d02cdcc7e0 feat(15-03): add Grafana and Sentry ReconSource implementations
- GrafanaSource: search dashboards via /api/search, fetch detail via /api/dashboards/uid
- SentrySource: search issues via /api/0/issues, fetch events for key detection
- Register all 5 log aggregator sources in RegisterAll (67 sources total)
- Tests use httptest mocks for each API endpoint
2026-04-06 16:31:14 +03:00
salvacybersec
bc63ca1f2f feat(15-03): add Elasticsearch, Kibana, and Splunk ReconSource implementations
- ElasticsearchSource: POST _search API with query_string, parse hits._source
- KibanaSource: GET saved_objects/_find API with kbn-xsrf header
- SplunkSource: GET search/jobs/export API with newline-delimited JSON parsing
- All sources use ciLogKeyPattern for key detection
- Tests use httptest mocks for each API endpoint
2026-04-06 16:31:05 +03:00
salvacybersec
77a2a0b531 docs(15-01): complete forum/discussion sources plan
- SUMMARY.md with 6 sources, 2 tasks, 13 files
- STATE.md advanced, ROADMAP.md updated, requirements marked
2026-04-06 16:30:49 +03:00
salvacybersec
fcc1a769c5 feat(15-01): add Discord, Slack, DevTo recon sources and wire all six
- DiscordSource uses dorking approach against configurable search endpoint
- SlackSource uses dorking against slack-archive indexers
- DevToSource searches dev.to API articles list + detail for body_markdown
- RegisterAll extended to include all 6 Phase 15 forum sources
- All credentialless, use ciLogKeyPattern for key detection
2026-04-06 16:29:52 +03:00
salvacybersec
282c145a43 feat(15-01): add StackOverflow, Reddit, HackerNews recon sources
- StackOverflowSource searches SE API v2.3 search/excerpts endpoint
- RedditSource searches Reddit JSON API with custom User-Agent
- HackerNewsSource searches Algolia HN API for comments
- All credentialless, use ciLogKeyPattern for key detection
- Tests use httptest mock servers with API key patterns
2026-04-06 16:28:23 +03:00
salvacybersec
37393a9b5f feat(15-02): wire Trello, Notion, Confluence, GoogleDocs into RegisterAll
- RegisterAll extended to 56 sources (52 Phase 10-14 + 4 Phase 15 collab)
- All four sources credentialless, no new SourcesConfig fields needed
2026-04-06 13:50:56 +03:00
salvacybersec
5d568333c7 feat(15-02): add Confluence and GoogleDocs ReconSource implementations
- ConfluenceSource searches exposed instances via /rest/api/content/search CQL
- GoogleDocsSource uses dorking + /export?format=txt for plain-text scanning
- HTML tag stripping for Confluence storage format
- Both credentialless, tests with httptest mocks confirm findings
2026-04-06 13:50:14 +03:00
salvacybersec
7bb614678d feat(15-02): add Trello and Notion ReconSource implementations
- TrelloSource searches public Trello boards via /1/search API
- NotionSource uses dorking to discover and scrape public Notion pages
- Both credentialless, follow established Phase 10 pattern
- Tests with httptest mocks confirm Sweep emits findings
2026-04-06 13:50:04 +03:00
salvacybersec
1affb0d864 docs(15): create phase plan — forums, collaboration, log aggregators 2026-04-06 13:47:43 +03:00
salvacybersec
554e93435f docs(phase-14): complete CI/CD logs, archives, frontend leaks 2026-04-06 13:42:54 +03:00
salvacybersec
4246db8294 fix: resolve Phase 14 merge conflicts across CI/CD, archive, and frontend sources 2026-04-06 13:42:54 +03:00
salvacybersec
27624e0ec7 merge: phase 14-04 register wiring 2026-04-06 13:39:32 +03:00
salvacybersec
117213aa7e docs(14-04): complete RegisterAll wiring + integration test plan
- 52 total sources across Phases 10-14
- Integration test validates all sources end-to-end
2026-04-06 13:39:16 +03:00
salvacybersec
7ef6c2ac34 feat(14-04): wire all 12 Phase 14 sources in RegisterAll (45 -> 52 total)
- Add CircleCIToken to SourcesConfig with env/viper lookup in cmd/recon.go
- Register 7 new sources: travisci, ghactions, circleci, jenkins, wayback, commoncrawl, jsbundle
- Update register_test.go expectations from 45 to 52 sources
- Add integration test handlers + registrations for all 12 Phase 14 sources
- Integration test now validates 52 sources end-to-end
2026-04-06 13:34:18 +03:00
salvacybersec
169b80b3bc feat(14-04): implement 7 Phase 14 sources (CI/CD, archives, JS bundles)
- TravisCISource: scrapes public Travis CI build logs for API key leaks
- GitHubActionsSource: searches Actions workflow logs (requires GitHub token)
- CircleCISource: scrapes CircleCI pipeline logs (requires CircleCI token)
- JenkinsSource: scrapes public Jenkins console output for leaked secrets
- WaybackMachineSource: searches Wayback Machine CDX for archived key leaks
- CommonCrawlSource: searches Common Crawl index for exposed pages
- JSBundleSource: probes JS bundles for embedded API key literals
2026-04-06 13:34:09 +03:00
salvacybersec
3a4e9c11bf fix: add CircleCIToken to SourcesConfig 2026-04-06 13:22:25 +03:00
salvacybersec
095b90ec07 merge: phase 14-03 frontend leaks 2026-04-06 13:21:39 +03:00
salvacybersec
aeebf37174 merge: phase 14 wave 1 all conflicts resolved 2026-04-06 13:21:32 +03:00
salvacybersec
9079059ab2 Merge branch 'worktree-agent-ad901ba0' 2026-04-06 13:21:21 +03:00
salvacybersec
95ee768266 docs(14-03): complete frontend leak sources plan
- SUMMARY.md with 5 sources, 3 commits, 13 files
- STATE.md updated with position and decisions
- Requirements RECON-JS-01 through RECON-JS-05 marked complete
2026-04-06 13:21:00 +03:00
salvacybersec
0a8be81f0c feat(14-03): wire 5 frontend leak sources in RegisterAll (40 -> 45 sources)
- Register SourceMapSource, WebpackSource, EnvLeakSource, SwaggerSource, DeployPreviewSource
- Update test expectations from 40 to 45 sources
2026-04-06 13:19:34 +03:00
salvacybersec
abfc2f8319 docs(14-01): complete CI/CD log sources plan
- 5 sources: GitHubActions, TravisCI, CircleCI, Jenkins, GitLabCI
- RegisterAll at 45 sources total
2026-04-06 13:18:31 +03:00
salvacybersec
7d8a4182d7 feat(14-03): implement SwaggerSource and DeployPreviewSource with tests
- SwaggerSource probes OpenAPI doc endpoints for API keys in example/default fields
- DeployPreviewSource scans Vercel/Netlify preview URLs for __NEXT_DATA__ env leaks
- Both implement ReconSource, credentialless, with httptest-based tests
2026-04-06 13:18:18 +03:00
salvacybersec
e0f267f7bf feat(14-01): add 5 CI/CD log sources (GitHubActions, TravisCI, CircleCI, Jenkins, GitLabCI)
- GitHubActionsSource: searches GitHub code search for workflow files with provider keywords (token-gated)
- TravisCISource: queries Travis CI v3 API for public build logs (credentialless)
- CircleCISource: queries CircleCI v2 pipeline API for build pipelines (token-gated)
- JenkinsSource: queries open Jenkins /api/json for job build consoles (credentialless)
- GitLabCISource: queries GitLab projects API for CI-enabled projects (token-gated)
- RegisterAll extended to 45 sources (40 Phase 10-13 + 5 Phase 14)
- Integration test updated with fixtures for all 5 new sources
- cmd/recon.go wires CIRCLECI_TOKEN env var
2026-04-06 13:17:31 +03:00
salvacybersec
1013caf843 docs(14-02): complete Wayback Machine + CommonCrawl web archive sources plan 2026-04-06 13:17:13 +03:00
salvacybersec
b57bd5e7d9 feat(14-03): implement SourceMapSource, WebpackSource, EnvLeakSource with tests
- SourceMapSource probes .map files for original source containing API keys
- WebpackSource scans JS bundles for inlined NEXT_PUBLIC_/REACT_APP_/VITE_ env vars
- EnvLeakSource probes common .env paths for exposed environment files
- All three implement ReconSource, credentialless, with httptest-based tests
2026-04-06 13:17:07 +03:00
salvacybersec
c5332454b0 feat(14-02): add WaybackMachine + CommonCrawl recon sources
- WaybackMachineSource queries CDX API for historical snapshots
- CommonCrawlSource queries CC Index API for matching pages
- Both credentialless, rate-limited at 1 req/5s, RespectsRobots=true
- RegisterAll extended to 42 sources (40 Phase 10-13 + 2 Phase 14)
- Full httptest-based test coverage for both sources
2026-04-06 13:16:13 +03:00
salvacybersec
06b0ae0e91 docs(14): create phase plan 2026-04-06 13:12:01 +03:00
salvacybersec
dc90785ab0 docs(phase-13): complete package registries + container/IaC 2026-04-06 13:06:43 +03:00
salvacybersec
6ea7698e31 docs(13-04): complete RegisterAll wiring + integration test plan
- SUMMARY.md with 2 tasks, 3 files modified
- STATE.md advanced to plan 4/4, Phase 13 complete
- ROADMAP.md updated with Phase 13 completion
- REQUIREMENTS.md marked RECON-PKG/INFRA requirements complete
2026-04-06 13:04:51 +03:00
salvacybersec
9b005e78bb test(13-04): add integration test handlers for all 12 Phase 13 sources (40 total)
- Add httptest mux handlers for npm, pypi, crates, rubygems, maven, nuget, goproxy, packagist, dockerhub, k8s, terraform, helm
- Register all 12 Phase 13 sources with BaseURL prefix routing
- Update expected source types and count assertions from 28 to 40
2026-04-06 13:03:27 +03:00
salvacybersec
c16f5feaee feat(13-04): wire all 12 Phase 13 sources into RegisterAll (40 total)
- Add 8 package registry sources (npm, pypi, crates, rubygems, maven, nuget, goproxy, packagist)
- Update register_test to assert 40 sources in sorted list
- Update Phase 12 integration test count from 32 to 40
2026-04-06 12:59:11 +03:00
salvacybersec
a607082131 merge: phase 13 resolve conflicts 2026-04-06 12:57:29 +03:00
salvacybersec
d17f326f62 docs(13-03): complete DockerHub/Kubernetes/Terraform/Helm sources plan
- SUMMARY with 2 tasks, 11 files, 3 commits
- STATE.md advanced to plan 3 of 4
- ROADMAP.md and REQUIREMENTS.md updated
2026-04-06 12:57:16 +03:00
salvacybersec
7e0e401266 feat(13-03): wire 4 Phase 13 sources into RegisterAll (32 total)
- Register DockerHub, Kubernetes, Terraform, Helm as credentialless sources
- Update RegisterAll tests and integration test to expect 32 sources
2026-04-06 12:55:52 +03:00
salvacybersec
c595fef148 docs(13-01): complete package registry sources plan
- SUMMARY.md with 4 sources, 16 tests, 8 files
- STATE.md updated with decisions and metrics
- Requirements RECON-PKG-01, RECON-PKG-02 marked complete
2026-04-06 12:55:06 +03:00
salvacybersec
c2c43dfba9 docs(13-02): complete Maven/NuGet/GoProxy/Packagist plan 2026-04-06 12:54:49 +03:00
salvacybersec
0727b51d79 feat(13-03): implement TerraformSource and HelmSource
- Terraform searches registry.terraform.io v1 modules API with namespace/name/provider URLs
- Helm searches artifacthub.io for charts (kind=0) with repo/chart URL construction
- Both sources: context cancellation, nil registry, httptest-based tests
2026-04-06 12:53:58 +03:00
salvacybersec
9907e2497a feat(13-01): implement CratesIOSource and RubyGemsSource with httptest tests
- CratesIOSource searches crates.io JSON API with custom User-Agent header
- RubyGemsSource searches rubygems.org search.json API for gem matches
- Both credentialless; CratesIO 1 req/s burst 1, RubyGems 1 req/2s burst 2
- Tests verify User-Agent header, Sweep findings, ctx cancellation, metadata
2026-04-06 12:53:41 +03:00
salvacybersec
018bb165fe feat(13-02): implement GoProxySource and PackagistSource with tests
- GoProxySource parses pkg.go.dev HTML search results for module paths
- PackagistSource queries Packagist JSON search API for PHP packages
- GoProxy regex requires domain dot to filter non-module paths
2026-04-06 12:53:37 +03:00
salvacybersec
3a8123edc6 feat(13-03): implement DockerHubSource and KubernetesSource
- DockerHub searches hub.docker.com v2 search API for repos matching provider keywords
- Kubernetes searches Artifact Hub for operators/manifests with kind-aware URL paths
- Both sources: context cancellation, nil registry, httptest-based tests
2026-04-06 12:52:45 +03:00