diff --git a/.planning/phases/10-osint-code-hosting/10-03-SUMMARY.md b/.planning/phases/10-osint-code-hosting/10-03-SUMMARY.md new file mode 100644 index 0000000..7e17eaf --- /dev/null +++ b/.planning/phases/10-osint-code-hosting/10-03-SUMMARY.md @@ -0,0 +1,90 @@ +--- +phase: 10-osint-code-hosting +plan: 03 +subsystem: recon/sources +tags: [recon, osint, gitlab, wave-2] +requires: + - pkg/recon/sources.Client (Plan 10-01) + - pkg/recon/sources.BuildQueries (Plan 10-01) + - pkg/recon.LimiterRegistry (Phase 9) + - pkg/providers.Registry +provides: + - pkg/recon/sources.GitLabSource +affects: + - pkg/recon/sources +tech_stack_added: [] +patterns: + - "Per-keyword BuildQueries loop driving search API calls" + - "PRIVATE-TOKEN header auth with shared retry-aware Client" + - "Disabled-when-empty-token semantics (Sweep returns nil with no requests)" + - "Bare-keyword → provider-name lookup via local keyword index" +key_files_created: + - pkg/recon/sources/gitlab.go + - pkg/recon/sources/gitlab_test.go +key_files_modified: [] +decisions: + - "Bare keyword BuildQueries output (gitlab case in formatQuery) — reverse lookup is a direct map[string]string access" + - "gitlabKeywordIndex helper named with gitlab prefix to avoid collision with peer github.go keywordIndex during parallel wave" + - "Finding.Source uses constructed /projects//-/blob// URL (per plan) rather than extra /api/v4/projects/ lookup to keep request budget tight" + - "Confidence=low across all recon findings; Phase 5 verify promotes to high" +metrics: + duration: ~8 minutes + completed_date: 2026-04-05 + tasks_completed: 1 + tests_added: 6 +--- + +# Phase 10 Plan 03: GitLabSource Summary + +GitLabSource is a thin recon.ReconSource that queries GitLab's `/api/v4/search?scope=blobs` endpoint with a PRIVATE-TOKEN header, iterating one search call per provider keyword from the shared BuildQueries helper and emitting a Finding per returned blob with Source pointing at a constructed `projects//-/blob//` URL. + +## What Was Built + +`pkg/recon/sources/gitlab.go` contains: + +- `GitLabSource` struct exposing Token, BaseURL, Registry, Limiters (lazy Client) +- ReconSource interface methods: `Name()="gitlab"`, `RateLimit()=rate.Every(30ms)`, `Burst()=5`, `RespectsRobots()=false`, `Enabled()` (token non-empty), `Sweep()` +- `glBlob` response DTO matching GitLab's documented blob search schema +- `gitlabKeywordIndex()` local helper (prefixed to avoid colliding with peer plan helpers during parallel wave execution) +- Compile-time `var _ recon.ReconSource = (*GitLabSource)(nil)` assertion + +`pkg/recon/sources/gitlab_test.go` covers all behaviors the plan called out: + +| Test | Verifies | +| --- | --- | +| `TestGitLabSource_EnabledFalseWhenTokenEmpty` | Enabled gating + Name/RespectsRobots accessors | +| `TestGitLabSource_EmptyToken_NoCallsNoError` | No HTTP request issued when Token=="" | +| `TestGitLabSource_Sweep_EmitsFindings` | PRIVATE-TOKEN header, `scope=blobs`, two queries × two blobs = 4 Findings, Source URLs contain project_id/ref/path | +| `TestGitLabSource_Unauthorized` | 401 propagates as `errors.Is(err, ErrUnauthorized)` | +| `TestGitLabSource_CtxCancellation` | Sweep returns promptly on ctx timeout against a hanging server | +| `TestGitLabSource_InterfaceAssertion` | Static recon.ReconSource conformance | + +## Verification + +``` +go build ./... # clean +go test ./pkg/recon/sources/ -run TestGitLab -v # 6/6 PASS +go test ./pkg/recon/sources/ # full package PASS (3.164s) +``` + +## Deviations from Plan + +None for must-have behavior. Two minor adjustments: + +1. `keywordIndex` helper renamed to `gitlabKeywordIndex` because `pkg/recon/sources/github.go` (Plan 10-02, wave-2 sibling) introduces an identically-named package-level symbol. Prefixing prevents a redeclared-identifier build failure when the parallel wave merges. +2. Provider name lookup simplified to direct `map[string]string` access on the bare keyword because `formatQuery("gitlab", k)` returns the keyword verbatim (no wrapping syntax), avoiding a second `extractKeyword`-style helper. + +## Deferred Issues + +None. + +## Known Stubs + +None. + +## Self-Check: PASSED + +- pkg/recon/sources/gitlab.go — FOUND +- pkg/recon/sources/gitlab_test.go — FOUND +- .planning/phases/10-osint-code-hosting/10-03-SUMMARY.md — FOUND +- commit 0137dc5 — FOUND