docs(12): create phase plan — IoT scanners + cloud storage sources
This commit is contained in:
salvacybersec
217
.planning/phases/12-osint_iot_cloud_storage/12-04-PLAN.md
Normal file
217
.planning/phases/12-osint_iot_cloud_storage/12-04-PLAN.md
Normal file
@@ -0,0 +1,217 @@
|
||||
---
|
||||
phase: 12-osint_iot_cloud_storage
|
||||
plan: 04
|
||||
type: execute
|
||||
wave: 2
|
||||
depends_on: [12-01, 12-02, 12-03]
|
||||
files_modified:
|
||||
- pkg/recon/sources/register.go
|
||||
- cmd/recon.go
|
||||
- pkg/recon/sources/integration_test.go
|
||||
autonomous: true
|
||||
requirements: [RECON-IOT-01, RECON-IOT-02, RECON-IOT-03, RECON-IOT-04, RECON-IOT-05, RECON-IOT-06, RECON-CLOUD-01, RECON-CLOUD-02, RECON-CLOUD-03, RECON-CLOUD-04]
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "RegisterAll registers all 28 sources (18 Phase 10-11 + 10 Phase 12)"
|
||||
- "cmd/recon.go populates SourcesConfig with all Phase 12 credential fields from env/viper"
|
||||
- "Integration test proves all 10 new sources are registered and discoverable by name"
|
||||
artifacts:
|
||||
- path: "pkg/recon/sources/register.go"
|
||||
provides: "RegisterAll with all Phase 12 sources added"
|
||||
contains: "Phase 12"
|
||||
- path: "cmd/recon.go"
|
||||
provides: "buildReconEngine with Phase 12 credential wiring"
|
||||
contains: "ShodanAPIKey"
|
||||
- path: "pkg/recon/sources/integration_test.go"
|
||||
provides: "Integration test covering all 28 registered sources"
|
||||
contains: "28"
|
||||
key_links:
|
||||
- from: "pkg/recon/sources/register.go"
|
||||
to: "pkg/recon/sources/shodan.go"
|
||||
via: "engine.Register(&ShodanSource{...})"
|
||||
pattern: "ShodanSource"
|
||||
- from: "cmd/recon.go"
|
||||
to: "pkg/recon/sources/register.go"
|
||||
via: "sources.RegisterAll(e, cfg)"
|
||||
pattern: "RegisterAll"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Wire all 10 Phase 12 sources into RegisterAll and cmd/recon.go, plus integration test.
|
||||
|
||||
Purpose: Make all IoT and cloud storage sources available via `keyhunter recon list` and `keyhunter recon full`.
|
||||
Output: Updated RegisterAll (28 sources total), updated cmd/recon.go with credential wiring, integration test.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/PROJECT.md
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/STATE.md
|
||||
@pkg/recon/sources/register.go
|
||||
@cmd/recon.go
|
||||
@pkg/recon/sources/integration_test.go
|
||||
|
||||
<interfaces>
|
||||
From pkg/recon/sources/register.go:
|
||||
```go
|
||||
type SourcesConfig struct {
|
||||
GitHubToken string
|
||||
// ... existing Phase 10-11 fields ...
|
||||
Registry *providers.Registry
|
||||
Limiters *recon.LimiterRegistry
|
||||
}
|
||||
func RegisterAll(engine *recon.Engine, cfg SourcesConfig)
|
||||
```
|
||||
|
||||
From cmd/recon.go:
|
||||
```go
|
||||
func buildReconEngine() *recon.Engine // constructs engine with all sources
|
||||
func firstNonEmpty(a, b string) string // env -> viper precedence
|
||||
```
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Extend SourcesConfig, RegisterAll, and cmd/recon.go</name>
|
||||
<files>pkg/recon/sources/register.go, cmd/recon.go</files>
|
||||
<action>
|
||||
**SourcesConfig** (register.go) — add these fields after the existing Phase 11 fields:
|
||||
|
||||
```go
|
||||
// Phase 12: IoT scanner API keys.
|
||||
ShodanAPIKey string
|
||||
CensysAPIId string
|
||||
CensysAPISecret string
|
||||
ZoomEyeAPIKey string
|
||||
FOFAEmail string
|
||||
FOFAAPIKey string
|
||||
NetlasAPIKey string
|
||||
BinaryEdgeAPIKey string
|
||||
```
|
||||
|
||||
**RegisterAll** (register.go) — add after the Phase 11 paste site registrations:
|
||||
|
||||
```go
|
||||
// Phase 12: IoT scanner sources.
|
||||
engine.Register(&ShodanSource{
|
||||
APIKey: cfg.ShodanAPIKey,
|
||||
Registry: reg,
|
||||
Limiters: lim,
|
||||
})
|
||||
engine.Register(&CensysSource{
|
||||
APIId: cfg.CensysAPIId,
|
||||
APISecret: cfg.CensysAPISecret,
|
||||
Registry: reg,
|
||||
Limiters: lim,
|
||||
})
|
||||
engine.Register(&ZoomEyeSource{
|
||||
APIKey: cfg.ZoomEyeAPIKey,
|
||||
Registry: reg,
|
||||
Limiters: lim,
|
||||
})
|
||||
engine.Register(&FOFASource{
|
||||
Email: cfg.FOFAEmail,
|
||||
APIKey: cfg.FOFAAPIKey,
|
||||
Registry: reg,
|
||||
Limiters: lim,
|
||||
})
|
||||
engine.Register(&NetlasSource{
|
||||
APIKey: cfg.NetlasAPIKey,
|
||||
Registry: reg,
|
||||
Limiters: lim,
|
||||
})
|
||||
engine.Register(&BinaryEdgeSource{
|
||||
APIKey: cfg.BinaryEdgeAPIKey,
|
||||
Registry: reg,
|
||||
Limiters: lim,
|
||||
})
|
||||
|
||||
// Phase 12: Cloud storage sources (credentialless).
|
||||
engine.Register(&S3Scanner{
|
||||
Registry: reg,
|
||||
Limiters: lim,
|
||||
})
|
||||
engine.Register(&GCSScanner{
|
||||
Registry: reg,
|
||||
Limiters: lim,
|
||||
})
|
||||
engine.Register(&AzureBlobScanner{
|
||||
Registry: reg,
|
||||
Limiters: lim,
|
||||
})
|
||||
engine.Register(&DOSpacesScanner{
|
||||
Registry: reg,
|
||||
Limiters: lim,
|
||||
})
|
||||
```
|
||||
|
||||
Update the RegisterAll doc comment to say "28 sources total" (18 Phase 10-11 + 10 Phase 12).
|
||||
|
||||
**cmd/recon.go** — in buildReconEngine(), add to the SourcesConfig literal:
|
||||
|
||||
```go
|
||||
ShodanAPIKey: firstNonEmpty(os.Getenv("SHODAN_API_KEY"), viper.GetString("recon.shodan.api_key")),
|
||||
CensysAPIId: firstNonEmpty(os.Getenv("CENSYS_API_ID"), viper.GetString("recon.censys.api_id")),
|
||||
CensysAPISecret: firstNonEmpty(os.Getenv("CENSYS_API_SECRET"), viper.GetString("recon.censys.api_secret")),
|
||||
ZoomEyeAPIKey: firstNonEmpty(os.Getenv("ZOOMEYE_API_KEY"), viper.GetString("recon.zoomeye.api_key")),
|
||||
FOFAEmail: firstNonEmpty(os.Getenv("FOFA_EMAIL"), viper.GetString("recon.fofa.email")),
|
||||
FOFAAPIKey: firstNonEmpty(os.Getenv("FOFA_API_KEY"), viper.GetString("recon.fofa.api_key")),
|
||||
NetlasAPIKey: firstNonEmpty(os.Getenv("NETLAS_API_KEY"), viper.GetString("recon.netlas.api_key")),
|
||||
BinaryEdgeAPIKey: firstNonEmpty(os.Getenv("BINARYEDGE_API_KEY"), viper.GetString("recon.binaryedge.api_key")),
|
||||
```
|
||||
|
||||
Update the reconCmd Long description to mention Phase 12 sources.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /home/salva/Documents/apikey/.claude/worktrees/agent-a6700ee2 && go build ./cmd/...</automated>
|
||||
</verify>
|
||||
<done>RegisterAll registers 28 sources; cmd/recon.go wires all Phase 12 credentials from env/viper</done>
|
||||
</task>
|
||||
|
||||
<task type="auto" tdd="true">
|
||||
<name>Task 2: Integration test for all 28 registered sources</name>
|
||||
<files>pkg/recon/sources/integration_test.go</files>
|
||||
<behavior>
|
||||
- TestRegisterAll_Phase12 registers all sources, asserts 28 total
|
||||
- All 10 new source names are present: shodan, censys, zoomeye, fofa, netlas, binaryedge, s3, gcs, azureblob, spaces
|
||||
- IoT sources with empty credentials report Enabled()==false
|
||||
- Cloud storage sources (credentialless) report Enabled()==true
|
||||
- SweepAll with short context timeout completes without panic
|
||||
</behavior>
|
||||
<action>
|
||||
Extend the existing integration_test.go (which currently tests 18 Phase 10-11 sources):
|
||||
- Update the expected source count from 18 to 28
|
||||
- Add all 10 new source names to the expected names list
|
||||
- Add assertions that IoT sources (shodan, censys, zoomeye, fofa, netlas, binaryedge) are Enabled()==false when credentials are empty
|
||||
- Add assertions that cloud sources (s3, gcs, azureblob, spaces) are Enabled()==true (credentialless)
|
||||
- Keep the existing SweepAll test with short context timeout, verify no panics
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /home/salva/Documents/apikey/.claude/worktrees/agent-a6700ee2 && go test ./pkg/recon/sources/ -run "TestRegisterAll" -v -count=1</automated>
|
||||
</verify>
|
||||
<done>Integration test passes with 28 registered sources; all Phase 12 source names are discoverable</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
- `go build ./cmd/...` compiles without errors
|
||||
- `go test ./pkg/recon/sources/ -run "TestRegisterAll" -v` passes with 28 sources
|
||||
- `go test ./pkg/recon/sources/ -v -count=1` all tests pass (existing + new)
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
All 10 Phase 12 sources are wired into RegisterAll and discoverable via the recon engine. cmd/recon.go reads credentials from env vars and viper config. Integration test confirms 28 total sources registered.
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/phases/12-osint_iot_cloud_storage/12-04-SUMMARY.md`
|
||||
</output>
|
||||
Reference in New Issue
Block a user