diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index ff5647d..fd147d0 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -93,9 +93,9 @@ Requirements for initial release. Each maps to roadmap phases. ### OSINT/Recon — IoT & Internet Scanners -- [ ] **RECON-IOT-01**: Shodan API search and dorking -- [ ] **RECON-IOT-02**: Censys API search -- [ ] **RECON-IOT-03**: ZoomEye API search +- [x] **RECON-IOT-01**: Shodan API search and dorking +- [x] **RECON-IOT-02**: Censys API search +- [x] **RECON-IOT-03**: ZoomEye API search - [ ] **RECON-IOT-04**: FOFA API search - [ ] **RECON-IOT-05**: Netlas API search - [ ] **RECON-IOT-06**: BinaryEdge API search diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 99468e1..987c26f 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -255,7 +255,7 @@ Plans: **Plans**: 4 plans Plans: -- [ ] 12-01-PLAN.md — ShodanSource + CensysSource + ZoomEyeSource (RECON-IOT-01, RECON-IOT-02, RECON-IOT-03) +- [x] 12-01-PLAN.md — ShodanSource + CensysSource + ZoomEyeSource (RECON-IOT-01, RECON-IOT-02, RECON-IOT-03) - [ ] 12-02-PLAN.md — FOFASource + NetlasSource + BinaryEdgeSource (RECON-IOT-04, RECON-IOT-05, RECON-IOT-06) - [ ] 12-03-PLAN.md — S3Scanner + GCSScanner + AzureBlobScanner + DOSpacesScanner (RECON-CLOUD-01, RECON-CLOUD-02, RECON-CLOUD-03, RECON-CLOUD-04) - [ ] 12-04-PLAN.md — RegisterAll wiring + cmd/recon.go credentials + integration test (all Phase 12 reqs) @@ -349,7 +349,7 @@ Phases execute in numeric order: 1 → 2 → 3 → ... → 18 | 9. OSINT Infrastructure | 2/6 | In Progress| | | 10. OSINT Code Hosting | 9/9 | Complete | 2026-04-06 | | 11. OSINT Search & Paste | 3/3 | Complete | 2026-04-06 | -| 12. OSINT IoT & Cloud Storage | 0/? | Not started | - | +| 12. OSINT IoT & Cloud Storage | 1/4 | In Progress| | | 13. OSINT Package Registries & Container/IaC | 0/? | Not started | - | | 14. OSINT CI/CD Logs, Web Archives & Frontend Leaks | 0/? | Not started | - | | 15. OSINT Forums, Collaboration & Log Aggregators | 0/? | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index fe584be..3be47e6 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -3,14 +3,14 @@ gsd_state_version: 1.0 milestone: v1.0 milestone_name: milestone status: completed -stopped_at: Completed 11-03-PLAN.md -last_updated: "2026-04-06T09:09:48.100Z" +stopped_at: Completed 12-01-PLAN.md +last_updated: "2026-04-06T09:25:12.004Z" last_activity: 2026-04-06 progress: total_phases: 18 - completed_phases: 11 - total_plans: 65 - completed_plans: 66 + completed_phases: 10 + total_plans: 64 + completed_plans: 67 percent: 20 --- @@ -91,6 +91,7 @@ Progress: [██░░░░░░░░] 20% | Phase 10 P09 | 12min | 2 tasks | 5 files | | Phase 11 P03 | 6min | 2 tasks | 4 files | | Phase 11 P01 | 3min | 2 tasks | 11 files | +| Phase 12 P01 | 3min | 2 tasks | 6 files | ## Accumulated Context @@ -131,6 +132,7 @@ Recent decisions affecting current work: - [Phase 11]: RegisterAll extended to 18 sources (10 Phase 10 + 8 Phase 11); paste sources use BaseURL prefix in integration test to avoid /search path collision - [Phase 11]: Integration test uses injected test platforms for PasteSites (same pattern as SandboxesSource) - [Phase 11]: All five search sources use dork query format to focus on paste/code hosting leak sites +- [Phase 12]: Shodan/Censys/ZoomEye use bare keyword queries; Censys POST+BasicAuth, Shodan key param, ZoomEye API-KEY header ### Pending Todos @@ -145,6 +147,6 @@ None yet. ## Session Continuity -Last session: 2026-04-06T09:07:51.980Z -Stopped at: Completed 11-03-PLAN.md +Last session: 2026-04-06T09:25:12.000Z +Stopped at: Completed 12-01-PLAN.md Resume file: None diff --git a/.planning/phases/12-osint_iot_cloud_storage/12-01-SUMMARY.md b/.planning/phases/12-osint_iot_cloud_storage/12-01-SUMMARY.md new file mode 100644 index 0000000..11f943b --- /dev/null +++ b/.planning/phases/12-osint_iot_cloud_storage/12-01-SUMMARY.md @@ -0,0 +1,99 @@ +--- +phase: 12-osint_iot_cloud_storage +plan: 01 +subsystem: recon +tags: [shodan, censys, zoomeye, iot, device-search, osint] + +# Dependency graph +requires: + - phase: 10-osint-code-hosting + provides: ReconSource interface, shared Client, BuildQueries, LimiterRegistry +provides: + - ShodanSource implementing recon.ReconSource + - CensysSource implementing recon.ReconSource + - ZoomEyeSource implementing recon.ReconSource +affects: [12-osint_iot_cloud_storage, recon-registration] + +# Tech tracking +tech-stack: + added: [] + patterns: [IoT device scanner source pattern with API key/header auth] + +key-files: + created: + - pkg/recon/sources/shodan.go + - pkg/recon/sources/censys.go + - pkg/recon/sources/zoomeye.go + - pkg/recon/sources/shodan_test.go + - pkg/recon/sources/censys_test.go + - pkg/recon/sources/zoomeye_test.go + modified: [] + +key-decisions: + - "Shodan, Censys, ZoomEye use bare keyword queries (default formatQuery case) -- no special syntax needed" + - "Censys uses POST with JSON body + Basic Auth; Shodan/ZoomEye use GET with key param/header" + +patterns-established: + - "IoT scanner source pattern: GET/POST to device search API, parse JSON matches, emit Finding per hit" + +requirements-completed: [RECON-IOT-01, RECON-IOT-02, RECON-IOT-03] + +# Metrics +duration: 3min +completed: 2026-04-06 +--- + +# Phase 12 Plan 01: Shodan, Censys, ZoomEye IoT Scanner Sources Summary + +**Three IoT device scanner recon sources searching Shodan host/search, Censys v2 hosts/search, and ZoomEye host/search for exposed LLM endpoints** + +## Performance + +- **Duration:** 3 min +- **Started:** 2026-04-06T09:21:40Z +- **Completed:** 2026-04-06T09:24:28Z +- **Tasks:** 2 +- **Files modified:** 6 + +## Accomplishments +- ShodanSource queries /shodan/host/search with API key param, emits findings per IP:port match +- CensysSource POSTs to /v2/hosts/search with Basic Auth (APIId:APISecret), emits findings per host hit +- ZoomEyeSource queries /host/search with API-KEY header, emits findings per IP:port match +- All three sources disabled when credentials empty, use shared retry Client, respect LimiterRegistry + +## Task Commits + +Each task was committed atomically: + +1. **Task 1: Implement ShodanSource, CensysSource, ZoomEyeSource** - `f5d8470` (feat) +2. **Task 2: Unit tests for Shodan, Censys, ZoomEye sources** - `6443e63` (test) + +## Files Created/Modified +- `pkg/recon/sources/shodan.go` - ShodanSource with /shodan/host/search API integration +- `pkg/recon/sources/censys.go` - CensysSource with POST /v2/hosts/search + Basic Auth +- `pkg/recon/sources/zoomeye.go` - ZoomEyeSource with /host/search + API-KEY header +- `pkg/recon/sources/shodan_test.go` - 4 tests: enabled, empty key, sweep findings, ctx cancel +- `pkg/recon/sources/censys_test.go` - 4 tests: enabled, empty creds, sweep findings, ctx cancel +- `pkg/recon/sources/zoomeye_test.go` - 4 tests: enabled, empty key, sweep findings, ctx cancel + +## Decisions Made +- Shodan, Censys, ZoomEye use bare keyword queries (default formatQuery case) -- no queries.go changes needed +- Censys uses POST with JSON body and Basic Auth; Shodan uses API key as query param; ZoomEye uses API-KEY header + +## Deviations from Plan + +None - plan executed exactly as written. + +## Issues Encountered +None + +## User Setup Required +None - no external service configuration required. + +## Next Phase Readiness +- Three IoT scanner sources ready for RegisterAll wiring in Plan 12-04 +- Same pattern applies to remaining Phase 12 sources (FOFA, Netlas, BinaryEdge) + +--- +*Phase: 12-osint_iot_cloud_storage* +*Completed: 2026-04-06*