Commit Graph

350 Commits

Author SHA1 Message Date
salvacybersec
095b90ec07 merge: phase 14-03 frontend leaks 2026-04-06 13:21:39 +03:00
salvacybersec
aeebf37174 merge: phase 14 wave 1 all conflicts resolved 2026-04-06 13:21:32 +03:00
salvacybersec
9079059ab2 Merge branch 'worktree-agent-ad901ba0' 2026-04-06 13:21:21 +03:00
salvacybersec
95ee768266 docs(14-03): complete frontend leak sources plan
- SUMMARY.md with 5 sources, 3 commits, 13 files
- STATE.md updated with position and decisions
- Requirements RECON-JS-01 through RECON-JS-05 marked complete
2026-04-06 13:21:00 +03:00
salvacybersec
0a8be81f0c feat(14-03): wire 5 frontend leak sources in RegisterAll (40 -> 45 sources)
- Register SourceMapSource, WebpackSource, EnvLeakSource, SwaggerSource, DeployPreviewSource
- Update test expectations from 40 to 45 sources
2026-04-06 13:19:34 +03:00
salvacybersec
abfc2f8319 docs(14-01): complete CI/CD log sources plan
- 5 sources: GitHubActions, TravisCI, CircleCI, Jenkins, GitLabCI
- RegisterAll at 45 sources total
2026-04-06 13:18:31 +03:00
salvacybersec
7d8a4182d7 feat(14-03): implement SwaggerSource and DeployPreviewSource with tests
- SwaggerSource probes OpenAPI doc endpoints for API keys in example/default fields
- DeployPreviewSource scans Vercel/Netlify preview URLs for __NEXT_DATA__ env leaks
- Both implement ReconSource, credentialless, with httptest-based tests
2026-04-06 13:18:18 +03:00
salvacybersec
e0f267f7bf feat(14-01): add 5 CI/CD log sources (GitHubActions, TravisCI, CircleCI, Jenkins, GitLabCI)
- GitHubActionsSource: searches GitHub code search for workflow files with provider keywords (token-gated)
- TravisCISource: queries Travis CI v3 API for public build logs (credentialless)
- CircleCISource: queries CircleCI v2 pipeline API for build pipelines (token-gated)
- JenkinsSource: queries open Jenkins /api/json for job build consoles (credentialless)
- GitLabCISource: queries GitLab projects API for CI-enabled projects (token-gated)
- RegisterAll extended to 45 sources (40 Phase 10-13 + 5 Phase 14)
- Integration test updated with fixtures for all 5 new sources
- cmd/recon.go wires CIRCLECI_TOKEN env var
2026-04-06 13:17:31 +03:00
salvacybersec
1013caf843 docs(14-02): complete Wayback Machine + CommonCrawl web archive sources plan 2026-04-06 13:17:13 +03:00
salvacybersec
b57bd5e7d9 feat(14-03): implement SourceMapSource, WebpackSource, EnvLeakSource with tests
- SourceMapSource probes .map files for original source containing API keys
- WebpackSource scans JS bundles for inlined NEXT_PUBLIC_/REACT_APP_/VITE_ env vars
- EnvLeakSource probes common .env paths for exposed environment files
- All three implement ReconSource, credentialless, with httptest-based tests
2026-04-06 13:17:07 +03:00
salvacybersec
c5332454b0 feat(14-02): add WaybackMachine + CommonCrawl recon sources
- WaybackMachineSource queries CDX API for historical snapshots
- CommonCrawlSource queries CC Index API for matching pages
- Both credentialless, rate-limited at 1 req/5s, RespectsRobots=true
- RegisterAll extended to 42 sources (40 Phase 10-13 + 2 Phase 14)
- Full httptest-based test coverage for both sources
2026-04-06 13:16:13 +03:00
salvacybersec
06b0ae0e91 docs(14): create phase plan 2026-04-06 13:12:01 +03:00
salvacybersec
dc90785ab0 docs(phase-13): complete package registries + container/IaC 2026-04-06 13:06:43 +03:00
salvacybersec
6ea7698e31 docs(13-04): complete RegisterAll wiring + integration test plan
- SUMMARY.md with 2 tasks, 3 files modified
- STATE.md advanced to plan 4/4, Phase 13 complete
- ROADMAP.md updated with Phase 13 completion
- REQUIREMENTS.md marked RECON-PKG/INFRA requirements complete
2026-04-06 13:04:51 +03:00
salvacybersec
9b005e78bb test(13-04): add integration test handlers for all 12 Phase 13 sources (40 total)
- Add httptest mux handlers for npm, pypi, crates, rubygems, maven, nuget, goproxy, packagist, dockerhub, k8s, terraform, helm
- Register all 12 Phase 13 sources with BaseURL prefix routing
- Update expected source types and count assertions from 28 to 40
2026-04-06 13:03:27 +03:00
salvacybersec
c16f5feaee feat(13-04): wire all 12 Phase 13 sources into RegisterAll (40 total)
- Add 8 package registry sources (npm, pypi, crates, rubygems, maven, nuget, goproxy, packagist)
- Update register_test to assert 40 sources in sorted list
- Update Phase 12 integration test count from 32 to 40
2026-04-06 12:59:11 +03:00
salvacybersec
a607082131 merge: phase 13 resolve conflicts 2026-04-06 12:57:29 +03:00
salvacybersec
d17f326f62 docs(13-03): complete DockerHub/Kubernetes/Terraform/Helm sources plan
- SUMMARY with 2 tasks, 11 files, 3 commits
- STATE.md advanced to plan 3 of 4
- ROADMAP.md and REQUIREMENTS.md updated
2026-04-06 12:57:16 +03:00
salvacybersec
7e0e401266 feat(13-03): wire 4 Phase 13 sources into RegisterAll (32 total)
- Register DockerHub, Kubernetes, Terraform, Helm as credentialless sources
- Update RegisterAll tests and integration test to expect 32 sources
2026-04-06 12:55:52 +03:00
salvacybersec
c595fef148 docs(13-01): complete package registry sources plan
- SUMMARY.md with 4 sources, 16 tests, 8 files
- STATE.md updated with decisions and metrics
- Requirements RECON-PKG-01, RECON-PKG-02 marked complete
2026-04-06 12:55:06 +03:00
salvacybersec
c2c43dfba9 docs(13-02): complete Maven/NuGet/GoProxy/Packagist plan 2026-04-06 12:54:49 +03:00
salvacybersec
0727b51d79 feat(13-03): implement TerraformSource and HelmSource
- Terraform searches registry.terraform.io v1 modules API with namespace/name/provider URLs
- Helm searches artifacthub.io for charts (kind=0) with repo/chart URL construction
- Both sources: context cancellation, nil registry, httptest-based tests
2026-04-06 12:53:58 +03:00
salvacybersec
9907e2497a feat(13-01): implement CratesIOSource and RubyGemsSource with httptest tests
- CratesIOSource searches crates.io JSON API with custom User-Agent header
- RubyGemsSource searches rubygems.org search.json API for gem matches
- Both credentialless; CratesIO 1 req/s burst 1, RubyGems 1 req/2s burst 2
- Tests verify User-Agent header, Sweep findings, ctx cancellation, metadata
2026-04-06 12:53:41 +03:00
salvacybersec
018bb165fe feat(13-02): implement GoProxySource and PackagistSource with tests
- GoProxySource parses pkg.go.dev HTML search results for module paths
- PackagistSource queries Packagist JSON search API for PHP packages
- GoProxy regex requires domain dot to filter non-module paths
2026-04-06 12:53:37 +03:00
salvacybersec
3a8123edc6 feat(13-03): implement DockerHubSource and KubernetesSource
- DockerHub searches hub.docker.com v2 search API for repos matching provider keywords
- Kubernetes searches Artifact Hub for operators/manifests with kind-aware URL paths
- Both sources: context cancellation, nil registry, httptest-based tests
2026-04-06 12:52:45 +03:00
salvacybersec
4b268d109f feat(13-01): implement NpmSource and PyPISource with httptest tests
- NpmSource searches npm registry JSON API for provider keywords
- PyPISource scrapes pypi.org search HTML for project links
- Both credentialless, rate-limited at 1 req/2s, burst 2
- httptest-based tests verify Sweep, ctx cancellation, Name/Rate/Burst
2026-04-06 12:52:31 +03:00
salvacybersec
23613150f6 feat(13-02): implement MavenSource and NuGetSource with tests
- MavenSource queries Maven Central Solr API for provider keyword matches
- NuGetSource queries NuGet gallery search API with projectUrl fallback
- Both sources: httptest fixtures, ctx cancellation, metadata tests
2026-04-06 12:52:27 +03:00
salvacybersec
877ae8c743 docs(13): create phase plan — 4 plans for package registries + container/IaC sources 2026-04-06 12:50:38 +03:00
salvacybersec
a5253cf9dd docs(phase-12): complete OSINT IoT + cloud storage 2026-04-06 12:45:39 +03:00
salvacybersec
a2347f150a docs(12-04): complete RegisterAll wiring + integration test plan
- SUMMARY.md with 28-source RegisterAll wiring and integration test
- STATE.md, ROADMAP.md, REQUIREMENTS.md updated
2026-04-06 12:43:34 +03:00
salvacybersec
f0f22191ef test(12-04): add end-to-end SweepAll integration test across all 28 sources
- Extend integration test with Phase 12 IoT scanner fixtures (shodan, censys, zoomeye, fofa, netlas, binaryedge)
- Add cloud storage fixtures with correct formats (S3 XML, GCS JSON, Azure EnumerationResults XML)
- Add TestRegisterAll_Phase12 verifying 28 sources, enabled/disabled states
- Add TestRegisterAll_Phase12_SweepAllNoPanic for timeout resilience
- Update existing register_test.go from 18 to 28 expected sources
2026-04-06 12:41:59 +03:00
salvacybersec
870431658d feat(12-04): wire all 10 Phase 12 sources into RegisterAll + cmd/recon.go credentials
- Add 8 Phase 12 credential fields to SourcesConfig (Shodan, Censys, ZoomEye, FOFA, Netlas, BinaryEdge)
- Register all 10 Phase 12 sources in RegisterAll (6 IoT + 4 cloud storage)
- Wire env/viper credential lookup in cmd/recon.go buildReconEngine
- Update reconCmd Long description to mention Phase 12 sources
2026-04-06 12:31:57 +03:00
salvacybersec
ade609d562 merge: phase 12 resolve conflicts 2026-04-06 12:27:23 +03:00
salvacybersec
c54e9c73ca merge: phase 12 resolve conflicts 2026-04-06 12:27:23 +03:00
salvacybersec
0afb19cc83 docs(12-03): complete cloud storage scanners plan
- SUMMARY.md with 4 cloud scanner sources (S3, GCS, Azure Blob, DO Spaces)
- STATE.md, ROADMAP.md, REQUIREMENTS.md updated
2026-04-06 12:27:05 +03:00
salvacybersec
13905eb5ee feat(12-03): implement AzureBlobScanner, DOSpacesScanner, and all cloud scanner tests
- AzureBlobScanner enumerates public Azure Blob containers with XML listing
- DOSpacesScanner enumerates public DO Spaces across 5 regions (S3-compatible XML)
- httptest-based tests for all four scanners: sweep, empty registry, ctx cancel, metadata
- All sources credentialless, compile-time interface assertions
2026-04-06 12:26:01 +03:00
salvacybersec
47d542b9de feat(12-03): implement S3Scanner and GCSScanner cloud storage recon sources
- S3Scanner enumerates public AWS S3 buckets by provider keyword + suffix pattern
- GCSScanner enumerates public GCS buckets with JSON listing format
- Shared bucketNames() helper and isConfigFile() filter for config-pattern files
- Both credentialless (anonymous HTTP), always Enabled, BaseURL override for tests
2026-04-06 12:25:55 +03:00
salvacybersec
8d97b263ec docs(12-01): complete Shodan/Censys/ZoomEye IoT scanner sources plan
- SUMMARY.md with 2 task commits, 6 files, 12 passing tests
- STATE.md, ROADMAP.md, REQUIREMENTS.md updated
2026-04-06 12:25:18 +03:00
salvacybersec
6ab411cda2 docs(12-02): complete FOFA, Netlas, BinaryEdge plan
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 12:25:06 +03:00
salvacybersec
6443e63b9a test(12-01): add httptest tests for Shodan, Censys, ZoomEye sources
- Each source tested with mock API server returning 2 results
- Verifies Enabled() disabled when credentials empty
- Verifies Sweep emits correct findings with proper SourceType
- Verifies context cancellation propagation
2026-04-06 12:24:18 +03:00
salvacybersec
d6c35f4f14 test(12-02): add httptest tests for FOFA, Netlas, BinaryEdge sources
- FOFA: mock JSON with 2 results, credential validation, context cancellation
- Netlas: mock JSON with 2 items, X-API-Key header check, context cancellation
- BinaryEdge: mock JSON with 2 events, X-Key header check, context cancellation
- All verify correct finding count, source type, and disabled state

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 12:24:11 +03:00
salvacybersec
270bbbfb49 feat(12-02): implement FOFA, Netlas, BinaryEdge recon sources
- FOFASource searches FOFA API with base64-encoded queries (email+key auth)
- NetlasSource searches Netlas API with X-API-Key header auth
- BinaryEdgeSource searches BinaryEdge API with X-Key header auth
- All three implement recon.ReconSource with shared Client retry/backoff

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 12:24:04 +03:00
salvacybersec
f5d8470aab feat(12-01): implement Shodan, Censys, ZoomEye recon sources
- ShodanSource searches /shodan/host/search with API key auth
- CensysSource POSTs to /v2/hosts/search with Basic Auth
- ZoomEyeSource searches /host/search with API-KEY header
- All use shared Client for retry/backoff, LimiterRegistry for rate limiting
2026-04-06 12:23:06 +03:00
salvacybersec
4b39c0828a Merge branch 'worktree-agent-a6700ee2' 2026-04-06 12:21:07 +03:00
salvacybersec
d8a54f2c16 docs: update README to reflect current implementation state (phases 1-11)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 12:20:42 +03:00
salvacybersec
e12b4bd2b5 docs(12): create phase plan — IoT scanners + cloud storage sources 2026-04-06 12:14:06 +03:00
salvacybersec
6f392b0b17 docs(phase-11): complete OSINT search & paste 2026-04-06 12:09:48 +03:00
salvacybersec
90d188fe9e docs(11-03): complete RegisterAll wiring + integration test plan
- SUMMARY.md with 18-source wiring details
- STATE.md updated with Phase 11 completion
- ROADMAP.md phase progress updated
- REQUIREMENTS.md: RECON-PASTE-01 marked complete
2026-04-06 12:07:56 +03:00
salvacybersec
bebc3e7a0b test(11-03): add end-to-end SweepAll integration test across all 18 sources
- Extend httptest mux with fixtures for Google, Bing, DuckDuckGo, Yandex, Brave
- Add Pastebin (routed /pb/), GistPaste (/gp/), PasteSites (injected platform)
- Assert all 18 SourceTypes emit at least one finding via SweepAll
2026-04-06 12:06:27 +03:00
salvacybersec
3250408f23 feat(11-03): wire 18 sources into RegisterAll + credential wiring in cmd/recon.go
- Extend SourcesConfig with GoogleAPIKey, GoogleCX, BingAPIKey, YandexUser, YandexAPIKey, BraveAPIKey
- RegisterAll registers 8 Phase 11 sources alongside 10 Phase 10 sources (18 total)
- cmd/recon.go reads search engine API keys from env vars and viper config
- Guardrail tests updated to assert 18 sources
2026-04-06 12:02:11 +03:00