docs(10-osint-code-hosting): create phase 10 plans (9 plans across 3 waves)
This commit is contained in:
120
.planning/phases/10-osint-code-hosting/10-03-PLAN.md
Normal file
120
.planning/phases/10-osint-code-hosting/10-03-PLAN.md
Normal file
@@ -0,0 +1,120 @@
|
||||
---
|
||||
phase: 10-osint-code-hosting
|
||||
plan: 03
|
||||
type: execute
|
||||
wave: 2
|
||||
depends_on: [10-01]
|
||||
files_modified:
|
||||
- pkg/recon/sources/gitlab.go
|
||||
- pkg/recon/sources/gitlab_test.go
|
||||
autonomous: true
|
||||
requirements: [RECON-CODE-02]
|
||||
must_haves:
|
||||
truths:
|
||||
- "GitLabSource.Sweep queries GitLab /api/v4/search?scope=blobs and emits Findings"
|
||||
- "Disabled when token empty; enabled otherwise"
|
||||
- "Findings have SourceType=\"recon:gitlab\" and Source = web_url of blob"
|
||||
artifacts:
|
||||
- path: "pkg/recon/sources/gitlab.go"
|
||||
provides: "GitLabSource implementing recon.ReconSource"
|
||||
- path: "pkg/recon/sources/gitlab_test.go"
|
||||
provides: "httptest tests"
|
||||
key_links:
|
||||
- from: "pkg/recon/sources/gitlab.go"
|
||||
to: "pkg/recon/sources/httpclient.go"
|
||||
via: "c.client.Do(ctx, req)"
|
||||
pattern: "client\\.Do"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Implement GitLabSource against GitLab's Search API (/api/v4/search?scope=blobs).
|
||||
Honors PRIVATE-TOKEN header auth, 2000 req/min rate limit.
|
||||
|
||||
Purpose: RECON-CODE-02.
|
||||
Output: pkg/recon/sources/gitlab.go + tests.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/phases/10-osint-code-hosting/10-CONTEXT.md
|
||||
@.planning/phases/10-osint-code-hosting/10-01-SUMMARY.md
|
||||
@pkg/recon/source.go
|
||||
@pkg/recon/sources/httpclient.go
|
||||
@pkg/recon/sources/queries.go
|
||||
|
||||
<interfaces>
|
||||
GitLab Search API (docs: https://docs.gitlab.com/ee/api/search.html):
|
||||
GET /api/v4/search?scope=blobs&search=<query>&per_page=20
|
||||
Header: PRIVATE-TOKEN: <token>
|
||||
Response (array of blob objects):
|
||||
[{ "basename": "...", "data": "matched snippet", "path": "...", "project_id": 123,
|
||||
"ref": "main", "startline": 42 }, ...]
|
||||
Project web_url must be constructed from project_id → fetch /api/v4/projects/<id> (or
|
||||
just use basename+path with a placeholder Source — keep it minimal: Source =
|
||||
"https://gitlab.com/projects/<project_id>/-/blob/<ref>/<path>").
|
||||
|
||||
Rate limit: 2000 req/min → rate.Every(30 * time.Millisecond) ≈ 2000/min, burst 5.
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto" tdd="true">
|
||||
<name>Task 1: GitLabSource implementation + tests</name>
|
||||
<files>pkg/recon/sources/gitlab.go, pkg/recon/sources/gitlab_test.go</files>
|
||||
<behavior>
|
||||
- Test A: Enabled false when token empty
|
||||
- Test B: Sweep queries /api/v4/search with scope=blobs, PRIVATE-TOKEN header set
|
||||
- Test C: Decodes array response, emits one Finding per blob with Source containing project_id + path + ref
|
||||
- Test D: 401 returns wrapped ErrUnauthorized
|
||||
- Test E: Ctx cancellation respected
|
||||
- Test F: Empty token → Sweep returns nil with no calls
|
||||
</behavior>
|
||||
<action>
|
||||
Create `pkg/recon/sources/gitlab.go` with struct `GitLabSource { Token, BaseURL string; Registry *providers.Registry; Limiters *recon.LimiterRegistry; client *Client }`.
|
||||
|
||||
Default BaseURL: `https://gitlab.com`.
|
||||
Name: "gitlab". RateLimit: `rate.Every(30 * time.Millisecond)`. Burst: 5. RespectsRobots: false.
|
||||
|
||||
Sweep loop:
|
||||
- For each query from BuildQueries(reg, "gitlab"):
|
||||
- Build `base + /api/v4/search?scope=blobs&search=<url-escaped>&per_page=20`
|
||||
- Set header `PRIVATE-TOKEN: <token>`
|
||||
- limiters.Wait, then client.Do
|
||||
- Decode `[]glBlob` where glBlob has ProjectID int, Path, Ref, Data, Startline
|
||||
- Emit Finding with Source = fmt.Sprintf("%s/projects/%d/-/blob/%s/%s", base, b.ProjectID, b.Ref, b.Path), SourceType="recon:gitlab", Confidence="low", ProviderName derived via keywordIndex(reg)
|
||||
- Respect ctx.Done on send
|
||||
|
||||
Add compile-time assert: `var _ recon.ReconSource = (*GitLabSource)(nil)`.
|
||||
|
||||
Create `pkg/recon/sources/gitlab_test.go` with httptest server returning a JSON
|
||||
array of two blob objects. Assert both Findings received, Source URLs contain
|
||||
project IDs, ctx cancellation test, 401 test, empty-token test. Use synthetic
|
||||
registry with 2 providers.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /home/salva/Documents/apikey && go test ./pkg/recon/sources/ -run TestGitLab -v -timeout 30s</automated>
|
||||
</verify>
|
||||
<done>
|
||||
GitLabSource compiles, implements ReconSource, all test behaviors covered.
|
||||
</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
- `go build ./...`
|
||||
- `go test ./pkg/recon/sources/ -run TestGitLab -v`
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
RECON-CODE-02 satisfied.
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/phases/10-osint-code-hosting/10-03-SUMMARY.md`.
|
||||
</output>
|
||||
Reference in New Issue
Block a user