keyhunter/.planning/phases/12-osint_iot_cloud_storage/12-02-PLAN.md

phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves

phase	plan	type	wave	depends_on	files_modified	autonomous	requirements	must_haves
12-osint_iot_cloud_storage	02	execute	1		pkg/recon/sources/fofa.go pkg/recon/sources/fofa_test.go pkg/recon/sources/netlas.go pkg/recon/sources/netlas_test.go pkg/recon/sources/binaryedge.go pkg/recon/sources/binaryedge_test.go	true	RECON-IOT-04 RECON-IOT-05 RECON-IOT-06	truths artifacts key_links FOFASource searches FOFA API for exposed endpoints and emits findings NetlasSource searches Netlas API for internet-wide scan results and emits findings BinaryEdgeSource searches BinaryEdge API for exposed services and emits findings Each source is disabled when its API key/credentials are empty path provides exports pkg/recon/sources/fofa.go FOFASource implementing recon.ReconSource FOFASource path provides exports pkg/recon/sources/netlas.go NetlasSource implementing recon.ReconSource NetlasSource path provides exports pkg/recon/sources/binaryedge.go BinaryEdgeSource implementing recon.ReconSource BinaryEdgeSource from to via pattern pkg/recon/sources/fofa.go pkg/recon/sources/httpclient.go sources.Client for retry/backoff HTTP s.client.Do from to via pattern pkg/recon/sources/netlas.go pkg/recon/sources/httpclient.go sources.Client for retry/backoff HTTP s.client.Do from to via pattern pkg/recon/sources/binaryedge.go pkg/recon/sources/httpclient.go sources.Client for retry/backoff HTTP s.client.Do

Implement three IoT scanner recon sources: FOFA, Netlas, and BinaryEdge.

Purpose: Complete the IoT/device scanner coverage with Chinese (FOFA) and alternative (Netlas, BinaryEdge) internet search engines. Output: Three source files + tests following the established Phase 10 pattern.

<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @pkg/recon/source.go @pkg/recon/sources/httpclient.go @pkg/recon/sources/bing.go @pkg/recon/sources/queries.go @pkg/recon/sources/register.go From pkg/recon/source.go: ```go type ReconSource interface { Name() string RateLimit() rate.Limit Burst() int RespectsRobots() bool Enabled(cfg Config) bool Sweep(ctx context.Context, query string, out chan<- Finding) error } ```

From pkg/recon/sources/httpclient.go:

type Client struct { HTTP *http.Client; MaxRetries int; UserAgent string }
func NewClient() *Client
func (c *Client) Do(ctx context.Context, req *http.Request) (*http.Response, error)
var ErrUnauthorized = errors.New("sources: unauthorized (check credentials)")

Task 1: Implement FOFASource, NetlasSource, BinaryEdgeSource pkg/recon/sources/fofa.go, pkg/recon/sources/netlas.go, pkg/recon/sources/binaryedge.go Create three source files following the BingDorkSource pattern:

FOFASource (fofa.go):

• Struct: FOFASource with fields Email string, APIKey string, BaseURL string, Registry *providers.Registry, Limiters *recon.LimiterRegistry, client *Client
• Compile-time assertion: var _ recon.ReconSource = (*FOFASource)(nil)
• Name(): "fofa"
• RateLimit(): rate.Every(1 * time.Second) — FOFA allows ~1 req/s
• Burst(): 1
• RespectsRobots(): false
• Enabled(): returns s.Email != "" && s.APIKey != ""
• BaseURL default: "https://fofa.info"
• Sweep(): For each query from BuildQueries, base64-encode the query, then GET {base}/api/v1/search/all?email={email}&key={apikey}&qbase64={base64query}&size=100. Parse JSON response {"results":[["ip","port","protocol","host"],...],"size":N}. Emit Finding per result with Source=fmt.Sprintf("fofa://%s:%s", result[0], result[1]), SourceType="recon:fofa".
• Note: FOFA results array contains string arrays, not objects. Each inner array is [host, ip, port].
• Add fofaKeywordIndex helper.

NetlasSource (netlas.go):

• Struct: NetlasSource with fields APIKey string, BaseURL string, Registry *providers.Registry, Limiters *recon.LimiterRegistry, client *Client
• Name(): "netlas"
• RateLimit(): rate.Every(1 * time.Second)
• Burst(): 1
• RespectsRobots(): false
• Enabled(): returns s.APIKey != ""
• BaseURL default: "https://app.netlas.io"
• Sweep(): For each query, GET {base}/api/responses/?q={url.QueryEscape(q)}&start=0&indices=. Set header X-API-Key: {apikey}. Parse JSON response {"items":[{"data":{"ip":"...","port":N}},...]}. Emit Finding per item with Source=fmt.Sprintf("netlas://%s:%d", item.Data.IP, item.Data.Port).
• Add netlasKeywordIndex helper.

BinaryEdgeSource (binaryedge.go):

• Struct: BinaryEdgeSource with fields APIKey string, BaseURL string, Registry *providers.Registry, Limiters *recon.LimiterRegistry, client *Client
• Name(): "binaryedge"
• RateLimit(): rate.Every(2 * time.Second) — BinaryEdge free tier is conservative
• Burst(): 1
• RespectsRobots(): false
• Enabled(): returns s.APIKey != ""
• BaseURL default: "https://api.binaryedge.io"
• Sweep(): For each query, GET {base}/v2/query/search?query={url.QueryEscape(q)}&page=1. Set header X-Key: {apikey}. Parse JSON response {"events":[{"target":{"ip":"...","port":N}},...]}. Emit Finding per event with Source=fmt.Sprintf("binaryedge://%s:%d", event.Target.IP, event.Target.Port).
• Add binaryedgeKeywordIndex helper.

Update formatQuery in queries.go to add cases for "fofa", "netlas", "binaryedge" — all use bare keyword (same as default).

Same patterns as Plan 12-01: use sources.NewClient(), s.Limiters.Wait before requests, standard error handling. cd /home/salva/Documents/apikey/.claude/worktrees/agent-a6700ee2 && go build ./pkg/recon/sources/ Three source files compile, each implements recon.ReconSource interface

Task 2: Unit tests for FOFA, Netlas, BinaryEdge sources pkg/recon/sources/fofa_test.go, pkg/recon/sources/netlas_test.go, pkg/recon/sources/binaryedge_test.go - FOFA: httptest server returns mock JSON with 2 results; Sweep emits 2 findings with "recon:fofa" source type - FOFA: empty Email or APIKey => Enabled()==false - Netlas: httptest server returns mock JSON with 2 items; Sweep emits 2 findings with "recon:netlas" source type - Netlas: empty APIKey => Enabled()==false - BinaryEdge: httptest server returns mock JSON with 2 events; Sweep emits 2 findings with "recon:binaryedge" source type - BinaryEdge: empty APIKey => Enabled()==false - All: cancelled context returns context error Create test files following the same httptest pattern used in Plan 12-01: - Use httptest.NewServer to mock API responses matching each source's expected JSON shape - Set BaseURL to test server URL - Create a minimal providers.Registry with 1-2 test providers - Verify Finding count, SourceType, and Source URL format - Test disabled state (empty credentials) - Test context cancellation cd /home/salva/Documents/apikey/.claude/worktrees/agent-a6700ee2 && go test ./pkg/recon/sources/ -run "TestFOFA|TestNetlas|TestBinaryEdge" -v -count=1 All FOFA, Netlas, BinaryEdge tests pass; each source emits correct findings from mock API responses - `go build ./pkg/recon/sources/` compiles without errors - `go test ./pkg/recon/sources/ -run "TestFOFA|TestNetlas|TestBinaryEdge" -v` all pass - Each source file has compile-time assertion `var _ recon.ReconSource = (*XxxSource)(nil)`

<success_criteria> Three IoT scanner sources (FOFA, Netlas, BinaryEdge) implement recon.ReconSource, use shared Client for HTTP, respect rate limiting via LimiterRegistry, and pass unit tests with mock API responses. </success_criteria>

After completion, create `.planning/phases/12-osint_iot_cloud_storage/12-02-SUMMARY.md`