91 lines
4.0 KiB
Markdown
91 lines
4.0 KiB
Markdown
---
|
||
phase: 10-osint-code-hosting
|
||
plan: 03
|
||
subsystem: recon/sources
|
||
tags: [recon, osint, gitlab, wave-2]
|
||
requires:
|
||
- pkg/recon/sources.Client (Plan 10-01)
|
||
- pkg/recon/sources.BuildQueries (Plan 10-01)
|
||
- pkg/recon.LimiterRegistry (Phase 9)
|
||
- pkg/providers.Registry
|
||
provides:
|
||
- pkg/recon/sources.GitLabSource
|
||
affects:
|
||
- pkg/recon/sources
|
||
tech_stack_added: []
|
||
patterns:
|
||
- "Per-keyword BuildQueries loop driving search API calls"
|
||
- "PRIVATE-TOKEN header auth with shared retry-aware Client"
|
||
- "Disabled-when-empty-token semantics (Sweep returns nil with no requests)"
|
||
- "Bare-keyword → provider-name lookup via local keyword index"
|
||
key_files_created:
|
||
- pkg/recon/sources/gitlab.go
|
||
- pkg/recon/sources/gitlab_test.go
|
||
key_files_modified: []
|
||
decisions:
|
||
- "Bare keyword BuildQueries output (gitlab case in formatQuery) — reverse lookup is a direct map[string]string access"
|
||
- "gitlabKeywordIndex helper named with gitlab prefix to avoid collision with peer github.go keywordIndex during parallel wave"
|
||
- "Finding.Source uses constructed /projects/<id>/-/blob/<ref>/<path> URL (per plan) rather than extra /api/v4/projects/<id> lookup to keep request budget tight"
|
||
- "Confidence=low across all recon findings; Phase 5 verify promotes to high"
|
||
metrics:
|
||
duration: ~8 minutes
|
||
completed_date: 2026-04-05
|
||
tasks_completed: 1
|
||
tests_added: 6
|
||
---
|
||
|
||
# Phase 10 Plan 03: GitLabSource Summary
|
||
|
||
GitLabSource is a thin recon.ReconSource that queries GitLab's `/api/v4/search?scope=blobs` endpoint with a PRIVATE-TOKEN header, iterating one search call per provider keyword from the shared BuildQueries helper and emitting a Finding per returned blob with Source pointing at a constructed `projects/<id>/-/blob/<ref>/<path>` URL.
|
||
|
||
## What Was Built
|
||
|
||
`pkg/recon/sources/gitlab.go` contains:
|
||
|
||
- `GitLabSource` struct exposing Token, BaseURL, Registry, Limiters (lazy Client)
|
||
- ReconSource interface methods: `Name()="gitlab"`, `RateLimit()=rate.Every(30ms)`, `Burst()=5`, `RespectsRobots()=false`, `Enabled()` (token non-empty), `Sweep()`
|
||
- `glBlob` response DTO matching GitLab's documented blob search schema
|
||
- `gitlabKeywordIndex()` local helper (prefixed to avoid colliding with peer plan helpers during parallel wave execution)
|
||
- Compile-time `var _ recon.ReconSource = (*GitLabSource)(nil)` assertion
|
||
|
||
`pkg/recon/sources/gitlab_test.go` covers all behaviors the plan called out:
|
||
|
||
| Test | Verifies |
|
||
| --- | --- |
|
||
| `TestGitLabSource_EnabledFalseWhenTokenEmpty` | Enabled gating + Name/RespectsRobots accessors |
|
||
| `TestGitLabSource_EmptyToken_NoCallsNoError` | No HTTP request issued when Token=="" |
|
||
| `TestGitLabSource_Sweep_EmitsFindings` | PRIVATE-TOKEN header, `scope=blobs`, two queries × two blobs = 4 Findings, Source URLs contain project_id/ref/path |
|
||
| `TestGitLabSource_Unauthorized` | 401 propagates as `errors.Is(err, ErrUnauthorized)` |
|
||
| `TestGitLabSource_CtxCancellation` | Sweep returns promptly on ctx timeout against a hanging server |
|
||
| `TestGitLabSource_InterfaceAssertion` | Static recon.ReconSource conformance |
|
||
|
||
## Verification
|
||
|
||
```
|
||
go build ./... # clean
|
||
go test ./pkg/recon/sources/ -run TestGitLab -v # 6/6 PASS
|
||
go test ./pkg/recon/sources/ # full package PASS (3.164s)
|
||
```
|
||
|
||
## Deviations from Plan
|
||
|
||
None for must-have behavior. Two minor adjustments:
|
||
|
||
1. `keywordIndex` helper renamed to `gitlabKeywordIndex` because `pkg/recon/sources/github.go` (Plan 10-02, wave-2 sibling) introduces an identically-named package-level symbol. Prefixing prevents a redeclared-identifier build failure when the parallel wave merges.
|
||
2. Provider name lookup simplified to direct `map[string]string` access on the bare keyword because `formatQuery("gitlab", k)` returns the keyword verbatim (no wrapping syntax), avoiding a second `extractKeyword`-style helper.
|
||
|
||
## Deferred Issues
|
||
|
||
None.
|
||
|
||
## Known Stubs
|
||
|
||
None.
|
||
|
||
## Self-Check: PASSED
|
||
|
||
- pkg/recon/sources/gitlab.go — FOUND
|
||
- pkg/recon/sources/gitlab_test.go — FOUND
|
||
- .planning/phases/10-osint-code-hosting/10-03-SUMMARY.md — FOUND
|
||
- commit 0137dc5 — FOUND
|