docs(13): create phase plan — 4 plans for package registries + container/IaC sources
This commit is contained in:
salvacybersec
@@ -0,0 +1,224 @@
|
||||
---
|
||||
phase: 13-osint_package_registries_container_iac
|
||||
plan: 03
|
||||
type: execute
|
||||
wave: 1
|
||||
depends_on: []
|
||||
files_modified:
|
||||
- pkg/recon/sources/dockerhub.go
|
||||
- pkg/recon/sources/dockerhub_test.go
|
||||
- pkg/recon/sources/kubernetes.go
|
||||
- pkg/recon/sources/kubernetes_test.go
|
||||
- pkg/recon/sources/terraform.go
|
||||
- pkg/recon/sources/terraform_test.go
|
||||
- pkg/recon/sources/helm.go
|
||||
- pkg/recon/sources/helm_test.go
|
||||
autonomous: true
|
||||
requirements:
|
||||
- RECON-INFRA-01
|
||||
- RECON-INFRA-02
|
||||
- RECON-INFRA-03
|
||||
- RECON-INFRA-04
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "DockerHubSource searches Docker Hub for images matching provider keywords and emits findings"
|
||||
- "KubernetesSource searches for publicly exposed Kubernetes configs via search/dorking and emits findings"
|
||||
- "TerraformSource searches Terraform Registry for modules matching provider keywords and emits findings"
|
||||
- "HelmSource searches Artifact Hub for Helm charts matching provider keywords and emits findings"
|
||||
- "All four sources handle context cancellation, empty registries, and HTTP errors gracefully"
|
||||
artifacts:
|
||||
- path: "pkg/recon/sources/dockerhub.go"
|
||||
provides: "DockerHubSource implementing recon.ReconSource"
|
||||
contains: "func (s *DockerHubSource) Sweep"
|
||||
- path: "pkg/recon/sources/kubernetes.go"
|
||||
provides: "KubernetesSource implementing recon.ReconSource"
|
||||
contains: "func (s *KubernetesSource) Sweep"
|
||||
- path: "pkg/recon/sources/terraform.go"
|
||||
provides: "TerraformSource implementing recon.ReconSource"
|
||||
contains: "func (s *TerraformSource) Sweep"
|
||||
- path: "pkg/recon/sources/helm.go"
|
||||
provides: "HelmSource implementing recon.ReconSource"
|
||||
contains: "func (s *HelmSource) Sweep"
|
||||
key_links:
|
||||
- from: "pkg/recon/sources/dockerhub.go"
|
||||
to: "pkg/recon/source.go"
|
||||
via: "implements ReconSource interface"
|
||||
pattern: "var _ recon\\.ReconSource"
|
||||
- from: "pkg/recon/sources/terraform.go"
|
||||
to: "pkg/recon/source.go"
|
||||
via: "implements ReconSource interface"
|
||||
pattern: "var _ recon\\.ReconSource"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Implement four container and infrastructure-as-code ReconSource modules: Docker Hub, Kubernetes, Terraform Registry, and Helm (via Artifact Hub).
|
||||
|
||||
Purpose: Enables KeyHunter to scan container images, Kubernetes configs, Terraform modules, and Helm charts for leaked API keys embedded in infrastructure definitions.
|
||||
Output: 4 source files + 4 test files in pkg/recon/sources/
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/PROJECT.md
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/STATE.md
|
||||
@pkg/recon/source.go
|
||||
@pkg/recon/sources/httpclient.go
|
||||
@pkg/recon/sources/queries.go
|
||||
@pkg/recon/sources/replit.go (pattern reference)
|
||||
@pkg/recon/sources/shodan.go (pattern reference — search API source)
|
||||
@pkg/recon/sources/replit_test.go (test pattern reference)
|
||||
|
||||
<interfaces>
|
||||
From pkg/recon/source.go:
|
||||
```go
|
||||
type ReconSource interface {
|
||||
Name() string
|
||||
RateLimit() rate.Limit
|
||||
Burst() int
|
||||
RespectsRobots() bool
|
||||
Enabled(cfg Config) bool
|
||||
Sweep(ctx context.Context, query string, out chan<- Finding) error
|
||||
}
|
||||
```
|
||||
|
||||
From pkg/recon/sources/httpclient.go:
|
||||
```go
|
||||
func NewClient() *Client
|
||||
func (c *Client) Do(ctx context.Context, req *http.Request) (*http.Response, error)
|
||||
```
|
||||
|
||||
From pkg/recon/sources/queries.go:
|
||||
```go
|
||||
func BuildQueries(reg *providers.Registry, source string) []string
|
||||
```
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Implement DockerHubSource and KubernetesSource</name>
|
||||
<files>pkg/recon/sources/dockerhub.go, pkg/recon/sources/dockerhub_test.go, pkg/recon/sources/kubernetes.go, pkg/recon/sources/kubernetes_test.go</files>
|
||||
<action>
|
||||
**DockerHubSource** (dockerhub.go):
|
||||
- Struct: `DockerHubSource` with `BaseURL`, `Registry`, `Limiters`, `Client`
|
||||
- Compile-time assertion: `var _ recon.ReconSource = (*DockerHubSource)(nil)`
|
||||
- Name() returns "dockerhub"
|
||||
- RateLimit() returns rate.Every(2 * time.Second) — Docker Hub rate limits unauthenticated at ~100 pulls/6h, search is more lenient
|
||||
- Burst() returns 2
|
||||
- RespectsRobots() returns false (JSON API)
|
||||
- Enabled() always true (Docker Hub search is unauthenticated)
|
||||
- BaseURL defaults to "https://hub.docker.com"
|
||||
- Sweep() logic:
|
||||
1. BuildQueries(s.Registry, "dockerhub")
|
||||
2. For each keyword, GET `{BaseURL}/v2/search/repositories/?query={keyword}&page_size=20`
|
||||
3. Parse JSON: `{"results": [{"repo_name": "...", "description": "...", "is_official": false}]}`
|
||||
4. Define response structs: `dockerHubSearchResponse`, `dockerHubRepo`
|
||||
5. Emit Finding per result: Source="https://hub.docker.com/r/{repo_name}", SourceType="recon:dockerhub"
|
||||
6. Description in finding can hint at build-arg or env-var exposure
|
||||
|
||||
**KubernetesSource** (kubernetes.go):
|
||||
- Struct: `KubernetesSource` with `BaseURL`, `Registry`, `Limiters`, `Client`
|
||||
- Compile-time assertion: `var _ recon.ReconSource = (*KubernetesSource)(nil)`
|
||||
- Name() returns "k8s"
|
||||
- RateLimit() returns rate.Every(3 * time.Second)
|
||||
- Burst() returns 1
|
||||
- RespectsRobots() returns true (searches public web for exposed K8s dashboards/configs)
|
||||
- Enabled() always true
|
||||
- BaseURL defaults to "https://search.censys.io" — uses Censys-style search for exposed K8s dashboards
|
||||
- ALTERNATIVE simpler approach: Search GitHub for exposed Kubernetes manifests containing secrets.
|
||||
Use BaseURL "https://api.github.com" and search for `kind: Secret` or `apiVersion: v1 kind: ConfigMap` with provider keywords.
|
||||
BUT this duplicates GitHubSource.
|
||||
- BEST approach: Use a dedicated search via pkg.go.dev-style HTML scraping but for Kubernetes YAML files on public artifact hubs.
|
||||
Actually, the simplest approach that aligns with RECON-INFRA-02 ("discovers publicly exposed Kubernetes dashboards and scans publicly readable Secret/ConfigMap objects"):
|
||||
Use Shodan/Censys-style dork queries. But those sources already exist.
|
||||
- FINAL approach: KubernetesSource searches Artifact Hub (artifacthub.io) for Kubernetes manifests/operators that may embed secrets. ArtifactHub has a JSON API.
|
||||
GET `{BaseURL}/api/v1/packages/search?ts_query_web={keyword}&kind=0&limit=20` (kind=0 = Helm charts, but also covers operators)
|
||||
Actually, use kind=6 for "Kube Operator" or leave blank for all kinds.
|
||||
BaseURL defaults to "https://artifacthub.io"
|
||||
Parse JSON: `{"packages": [{"name": "...", "normalized_name": "...", "repository": {"name": "...", "url": "..."}}]}`
|
||||
Emit Finding: Source="https://artifacthub.io/packages/{repository.kind}/{repository.name}/{package.name}", SourceType="recon:k8s"
|
||||
|
||||
**Tests** — httptest pattern:
|
||||
- dockerhub_test.go: httptest serving canned Docker Hub search JSON. Verify findings have correct SourceType and Source URL format.
|
||||
- kubernetes_test.go: httptest serving canned Artifact Hub search JSON. Standard test categories.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /home/salva/Documents/apikey && go test ./pkg/recon/sources/ -run "TestDockerHub|TestKubernetes" -v -count=1</automated>
|
||||
</verify>
|
||||
<done>DockerHubSource and KubernetesSource pass all tests: Docker Hub search returns repo findings, K8s source finds Artifact Hub packages</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Implement TerraformSource and HelmSource</name>
|
||||
<files>pkg/recon/sources/terraform.go, pkg/recon/sources/terraform_test.go, pkg/recon/sources/helm.go, pkg/recon/sources/helm_test.go</files>
|
||||
<action>
|
||||
**TerraformSource** (terraform.go):
|
||||
- Struct: `TerraformSource` with `BaseURL`, `Registry`, `Limiters`, `Client`
|
||||
- Compile-time assertion: `var _ recon.ReconSource = (*TerraformSource)(nil)`
|
||||
- Name() returns "terraform"
|
||||
- RateLimit() returns rate.Every(2 * time.Second)
|
||||
- Burst() returns 2
|
||||
- RespectsRobots() returns false (JSON API)
|
||||
- Enabled() always true
|
||||
- BaseURL defaults to "https://registry.terraform.io"
|
||||
- Sweep() logic:
|
||||
1. BuildQueries(s.Registry, "terraform")
|
||||
2. For each keyword, GET `{BaseURL}/v1/modules?q={keyword}&limit=20`
|
||||
3. Parse JSON: `{"modules": [{"id": "namespace/name/provider", "namespace": "...", "name": "...", "provider": "...", "description": "..."}]}`
|
||||
4. Define response structs: `terraformSearchResponse`, `terraformModule`
|
||||
5. Emit Finding per module: Source="https://registry.terraform.io/modules/{namespace}/{name}/{provider}", SourceType="recon:terraform"
|
||||
|
||||
**HelmSource** (helm.go):
|
||||
- Struct: `HelmSource` with `BaseURL`, `Registry`, `Limiters`, `Client`
|
||||
- Compile-time assertion: `var _ recon.ReconSource = (*HelmSource)(nil)`
|
||||
- Name() returns "helm"
|
||||
- RateLimit() returns rate.Every(2 * time.Second)
|
||||
- Burst() returns 2
|
||||
- RespectsRobots() returns false (JSON API)
|
||||
- Enabled() always true
|
||||
- BaseURL defaults to "https://artifacthub.io"
|
||||
- Sweep() logic:
|
||||
1. BuildQueries(s.Registry, "helm")
|
||||
2. For each keyword, GET `{BaseURL}/api/v1/packages/search?ts_query_web={keyword}&kind=0&limit=20` (kind=0 = Helm charts)
|
||||
3. Parse JSON: `{"packages": [{"package_id": "...", "name": "...", "normalized_name": "...", "repository": {"name": "...", "kind": 0}}]}`
|
||||
4. Define response structs: `artifactHubSearchResponse`, `artifactHubPackage`, `artifactHubRepo`
|
||||
5. Emit Finding per package: Source="https://artifacthub.io/packages/helm/{repo.name}/{package.name}", SourceType="recon:helm"
|
||||
6. Note: HelmSource and KubernetesSource both use Artifact Hub but with different `kind` parameters and different SourceType tags. Keep them separate — different concerns.
|
||||
|
||||
**Tests** — httptest pattern:
|
||||
- terraform_test.go: httptest serving canned Terraform registry JSON. Verify module URL construction from namespace/name/provider.
|
||||
- helm_test.go: httptest serving canned Artifact Hub JSON for Helm charts. Standard test categories.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /home/salva/Documents/apikey && go test ./pkg/recon/sources/ -run "TestTerraform|TestHelm" -v -count=1</automated>
|
||||
</verify>
|
||||
<done>TerraformSource and HelmSource pass all tests. Terraform constructs correct module URLs. Helm extracts Artifact Hub packages correctly.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
All 8 new files compile and pass tests:
|
||||
```bash
|
||||
go test ./pkg/recon/sources/ -run "TestDockerHub|TestKubernetes|TestTerraform|TestHelm" -v -count=1
|
||||
go vet ./pkg/recon/sources/
|
||||
```
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- 4 new source files implement recon.ReconSource interface
|
||||
- 4 test files use httptest with canned fixtures
|
||||
- All tests pass
|
||||
- No compilation errors across the package
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/phases/13-osint_package_registries_container_iac/13-03-SUMMARY.md`
|
||||
</output>
|
||||
Reference in New Issue
Block a user