docs(12-01): complete Shodan/Censys/ZoomEye IoT scanner sources plan

- SUMMARY.md with 2 task commits, 6 files, 12 passing tests
- STATE.md, ROADMAP.md, REQUIREMENTS.md updated
This commit is contained in:
salvacybersec
2026-04-06 12:25:18 +03:00
parent 6443e63b9a
commit 8d97b263ec
4 changed files with 113 additions and 12 deletions

View File

@@ -93,9 +93,9 @@ Requirements for initial release. Each maps to roadmap phases.
### OSINT/Recon — IoT & Internet Scanners ### OSINT/Recon — IoT & Internet Scanners
- [ ] **RECON-IOT-01**: Shodan API search and dorking - [x] **RECON-IOT-01**: Shodan API search and dorking
- [ ] **RECON-IOT-02**: Censys API search - [x] **RECON-IOT-02**: Censys API search
- [ ] **RECON-IOT-03**: ZoomEye API search - [x] **RECON-IOT-03**: ZoomEye API search
- [ ] **RECON-IOT-04**: FOFA API search - [ ] **RECON-IOT-04**: FOFA API search
- [ ] **RECON-IOT-05**: Netlas API search - [ ] **RECON-IOT-05**: Netlas API search
- [ ] **RECON-IOT-06**: BinaryEdge API search - [ ] **RECON-IOT-06**: BinaryEdge API search

View File

@@ -255,7 +255,7 @@ Plans:
**Plans**: 4 plans **Plans**: 4 plans
Plans: Plans:
- [ ] 12-01-PLAN.md — ShodanSource + CensysSource + ZoomEyeSource (RECON-IOT-01, RECON-IOT-02, RECON-IOT-03) - [x] 12-01-PLAN.md — ShodanSource + CensysSource + ZoomEyeSource (RECON-IOT-01, RECON-IOT-02, RECON-IOT-03)
- [ ] 12-02-PLAN.md — FOFASource + NetlasSource + BinaryEdgeSource (RECON-IOT-04, RECON-IOT-05, RECON-IOT-06) - [ ] 12-02-PLAN.md — FOFASource + NetlasSource + BinaryEdgeSource (RECON-IOT-04, RECON-IOT-05, RECON-IOT-06)
- [ ] 12-03-PLAN.md — S3Scanner + GCSScanner + AzureBlobScanner + DOSpacesScanner (RECON-CLOUD-01, RECON-CLOUD-02, RECON-CLOUD-03, RECON-CLOUD-04) - [ ] 12-03-PLAN.md — S3Scanner + GCSScanner + AzureBlobScanner + DOSpacesScanner (RECON-CLOUD-01, RECON-CLOUD-02, RECON-CLOUD-03, RECON-CLOUD-04)
- [ ] 12-04-PLAN.md — RegisterAll wiring + cmd/recon.go credentials + integration test (all Phase 12 reqs) - [ ] 12-04-PLAN.md — RegisterAll wiring + cmd/recon.go credentials + integration test (all Phase 12 reqs)
@@ -349,7 +349,7 @@ Phases execute in numeric order: 1 → 2 → 3 → ... → 18
| 9. OSINT Infrastructure | 2/6 | In Progress| | | 9. OSINT Infrastructure | 2/6 | In Progress| |
| 10. OSINT Code Hosting | 9/9 | Complete | 2026-04-06 | | 10. OSINT Code Hosting | 9/9 | Complete | 2026-04-06 |
| 11. OSINT Search & Paste | 3/3 | Complete | 2026-04-06 | | 11. OSINT Search & Paste | 3/3 | Complete | 2026-04-06 |
| 12. OSINT IoT & Cloud Storage | 0/? | Not started | - | | 12. OSINT IoT & Cloud Storage | 1/4 | In Progress| |
| 13. OSINT Package Registries & Container/IaC | 0/? | Not started | - | | 13. OSINT Package Registries & Container/IaC | 0/? | Not started | - |
| 14. OSINT CI/CD Logs, Web Archives & Frontend Leaks | 0/? | Not started | - | | 14. OSINT CI/CD Logs, Web Archives & Frontend Leaks | 0/? | Not started | - |
| 15. OSINT Forums, Collaboration & Log Aggregators | 0/? | Not started | - | | 15. OSINT Forums, Collaboration & Log Aggregators | 0/? | Not started | - |

View File

@@ -3,14 +3,14 @@ gsd_state_version: 1.0
milestone: v1.0 milestone: v1.0
milestone_name: milestone milestone_name: milestone
status: completed status: completed
stopped_at: Completed 11-03-PLAN.md stopped_at: Completed 12-01-PLAN.md
last_updated: "2026-04-06T09:09:48.100Z" last_updated: "2026-04-06T09:25:12.004Z"
last_activity: 2026-04-06 last_activity: 2026-04-06
progress: progress:
total_phases: 18 total_phases: 18
completed_phases: 11 completed_phases: 10
total_plans: 65 total_plans: 64
completed_plans: 66 completed_plans: 67
percent: 20 percent: 20
--- ---
@@ -91,6 +91,7 @@ Progress: [██░░░░░░░░] 20%
| Phase 10 P09 | 12min | 2 tasks | 5 files | | Phase 10 P09 | 12min | 2 tasks | 5 files |
| Phase 11 P03 | 6min | 2 tasks | 4 files | | Phase 11 P03 | 6min | 2 tasks | 4 files |
| Phase 11 P01 | 3min | 2 tasks | 11 files | | Phase 11 P01 | 3min | 2 tasks | 11 files |
| Phase 12 P01 | 3min | 2 tasks | 6 files |
## Accumulated Context ## Accumulated Context
@@ -131,6 +132,7 @@ Recent decisions affecting current work:
- [Phase 11]: RegisterAll extended to 18 sources (10 Phase 10 + 8 Phase 11); paste sources use BaseURL prefix in integration test to avoid /search path collision - [Phase 11]: RegisterAll extended to 18 sources (10 Phase 10 + 8 Phase 11); paste sources use BaseURL prefix in integration test to avoid /search path collision
- [Phase 11]: Integration test uses injected test platforms for PasteSites (same pattern as SandboxesSource) - [Phase 11]: Integration test uses injected test platforms for PasteSites (same pattern as SandboxesSource)
- [Phase 11]: All five search sources use dork query format to focus on paste/code hosting leak sites - [Phase 11]: All five search sources use dork query format to focus on paste/code hosting leak sites
- [Phase 12]: Shodan/Censys/ZoomEye use bare keyword queries; Censys POST+BasicAuth, Shodan key param, ZoomEye API-KEY header
### Pending Todos ### Pending Todos
@@ -145,6 +147,6 @@ None yet.
## Session Continuity ## Session Continuity
Last session: 2026-04-06T09:07:51.980Z Last session: 2026-04-06T09:25:12.000Z
Stopped at: Completed 11-03-PLAN.md Stopped at: Completed 12-01-PLAN.md
Resume file: None Resume file: None

View File

@@ -0,0 +1,99 @@
---
phase: 12-osint_iot_cloud_storage
plan: 01
subsystem: recon
tags: [shodan, censys, zoomeye, iot, device-search, osint]
# Dependency graph
requires:
- phase: 10-osint-code-hosting
provides: ReconSource interface, shared Client, BuildQueries, LimiterRegistry
provides:
- ShodanSource implementing recon.ReconSource
- CensysSource implementing recon.ReconSource
- ZoomEyeSource implementing recon.ReconSource
affects: [12-osint_iot_cloud_storage, recon-registration]
# Tech tracking
tech-stack:
added: []
patterns: [IoT device scanner source pattern with API key/header auth]
key-files:
created:
- pkg/recon/sources/shodan.go
- pkg/recon/sources/censys.go
- pkg/recon/sources/zoomeye.go
- pkg/recon/sources/shodan_test.go
- pkg/recon/sources/censys_test.go
- pkg/recon/sources/zoomeye_test.go
modified: []
key-decisions:
- "Shodan, Censys, ZoomEye use bare keyword queries (default formatQuery case) -- no special syntax needed"
- "Censys uses POST with JSON body + Basic Auth; Shodan/ZoomEye use GET with key param/header"
patterns-established:
- "IoT scanner source pattern: GET/POST to device search API, parse JSON matches, emit Finding per hit"
requirements-completed: [RECON-IOT-01, RECON-IOT-02, RECON-IOT-03]
# Metrics
duration: 3min
completed: 2026-04-06
---
# Phase 12 Plan 01: Shodan, Censys, ZoomEye IoT Scanner Sources Summary
**Three IoT device scanner recon sources searching Shodan host/search, Censys v2 hosts/search, and ZoomEye host/search for exposed LLM endpoints**
## Performance
- **Duration:** 3 min
- **Started:** 2026-04-06T09:21:40Z
- **Completed:** 2026-04-06T09:24:28Z
- **Tasks:** 2
- **Files modified:** 6
## Accomplishments
- ShodanSource queries /shodan/host/search with API key param, emits findings per IP:port match
- CensysSource POSTs to /v2/hosts/search with Basic Auth (APIId:APISecret), emits findings per host hit
- ZoomEyeSource queries /host/search with API-KEY header, emits findings per IP:port match
- All three sources disabled when credentials empty, use shared retry Client, respect LimiterRegistry
## Task Commits
Each task was committed atomically:
1. **Task 1: Implement ShodanSource, CensysSource, ZoomEyeSource** - `f5d8470` (feat)
2. **Task 2: Unit tests for Shodan, Censys, ZoomEye sources** - `6443e63` (test)
## Files Created/Modified
- `pkg/recon/sources/shodan.go` - ShodanSource with /shodan/host/search API integration
- `pkg/recon/sources/censys.go` - CensysSource with POST /v2/hosts/search + Basic Auth
- `pkg/recon/sources/zoomeye.go` - ZoomEyeSource with /host/search + API-KEY header
- `pkg/recon/sources/shodan_test.go` - 4 tests: enabled, empty key, sweep findings, ctx cancel
- `pkg/recon/sources/censys_test.go` - 4 tests: enabled, empty creds, sweep findings, ctx cancel
- `pkg/recon/sources/zoomeye_test.go` - 4 tests: enabled, empty key, sweep findings, ctx cancel
## Decisions Made
- Shodan, Censys, ZoomEye use bare keyword queries (default formatQuery case) -- no queries.go changes needed
- Censys uses POST with JSON body and Basic Auth; Shodan uses API key as query param; ZoomEye uses API-KEY header
## Deviations from Plan
None - plan executed exactly as written.
## Issues Encountered
None
## User Setup Required
None - no external service configuration required.
## Next Phase Readiness
- Three IoT scanner sources ready for RegisterAll wiring in Plan 12-04
- Same pattern applies to remaining Phase 12 sources (FOFA, Netlas, BinaryEdge)
---
*Phase: 12-osint_iot_cloud_storage*
*Completed: 2026-04-06*