salvacybersec/keyhunter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(10-03): complete GitLabSource plan summary

Browse Source
This commit is contained in:
salvacybersec
2026-04-06 01:16:34 +03:00
parent 0137dc57b1
commit 223c23e672
1 changed files with 90 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

90
.planning/phases/10-osint-code-hosting/10-03-SUMMARY.md Normal file
View File

@@ -0,0 +1,90 @@
---
phase: 10-osint-code-hosting
plan: 03
subsystem: recon/sources
tags: [recon, osint, gitlab, wave-2]
requires:
- pkg/recon/sources.Client (Plan 10-01)
- pkg/recon/sources.BuildQueries (Plan 10-01)
- pkg/recon.LimiterRegistry (Phase 9)
- pkg/providers.Registry
provides:
- pkg/recon/sources.GitLabSource
affects:
- pkg/recon/sources
tech_stack_added: []
patterns:
- "Per-keyword BuildQueries loop driving search API calls"
- "PRIVATE-TOKEN header auth with shared retry-aware Client"
- "Disabled-when-empty-token semantics (Sweep returns nil with no requests)"
- "Bare-keyword → provider-name lookup via local keyword index"
key_files_created:
- pkg/recon/sources/gitlab.go
- pkg/recon/sources/gitlab_test.go
key_files_modified: []
decisions:
- "Bare keyword BuildQueries output (gitlab case in formatQuery) — reverse lookup is a direct map[string]string access"
- "gitlabKeywordIndex helper named with gitlab prefix to avoid collision with peer github.go keywordIndex during parallel wave"
- "Finding.Source uses constructed /projects/<id>/-/blob/<ref>/<path> URL (per plan) rather than extra /api/v4/projects/<id> lookup to keep request budget tight"
- "Confidence=low across all recon findings; Phase 5 verify promotes to high"
metrics:
duration: ~8 minutes
completed_date: 2026-04-05
tasks_completed: 1
tests_added: 6
---
# Phase 10 Plan 03: GitLabSource Summary
GitLabSource is a thin recon.ReconSource that queries GitLab's `/api/v4/search?scope=blobs` endpoint with a PRIVATE-TOKEN header, iterating one search call per provider keyword from the shared BuildQueries helper and emitting a Finding per returned blob with Source pointing at a constructed `projects/<id>/-/blob/<ref>/<path>` URL.
## What Was Built
`pkg/recon/sources/gitlab.go` contains:
- `GitLabSource` struct exposing Token, BaseURL, Registry, Limiters (lazy Client)
- ReconSource interface methods: `Name()="gitlab"`, `RateLimit()=rate.Every(30ms)`, `Burst()=5`, `RespectsRobots()=false`, `Enabled()` (token non-empty), `Sweep()`
- `glBlob` response DTO matching GitLab's documented blob search schema
- `gitlabKeywordIndex()` local helper (prefixed to avoid colliding with peer plan helpers during parallel wave execution)
- Compile-time `var _ recon.ReconSource = (*GitLabSource)(nil)` assertion
`pkg/recon/sources/gitlab_test.go` covers all behaviors the plan called out:
| Test | Verifies |
| --- | --- |
| `TestGitLabSource_EnabledFalseWhenTokenEmpty` | Enabled gating + Name/RespectsRobots accessors |
| `TestGitLabSource_EmptyToken_NoCallsNoError` | No HTTP request issued when Token=="" |
| `TestGitLabSource_Sweep_EmitsFindings` | PRIVATE-TOKEN header, `scope=blobs`, two queries × two blobs = 4 Findings, Source URLs contain project_id/ref/path |
| `TestGitLabSource_Unauthorized` | 401 propagates as `errors.Is(err, ErrUnauthorized)` |
| `TestGitLabSource_CtxCancellation` | Sweep returns promptly on ctx timeout against a hanging server |
| `TestGitLabSource_InterfaceAssertion` | Static recon.ReconSource conformance |
## Verification
```
go build ./... # clean
go test ./pkg/recon/sources/ -run TestGitLab -v # 6/6 PASS
go test ./pkg/recon/sources/ # full package PASS (3.164s)
```
## Deviations from Plan
None for must-have behavior. Two minor adjustments:
1. `keywordIndex` helper renamed to `gitlabKeywordIndex` because `pkg/recon/sources/github.go` (Plan 10-02, wave-2 sibling) introduces an identically-named package-level symbol. Prefixing prevents a redeclared-identifier build failure when the parallel wave merges.
2. Provider name lookup simplified to direct `map[string]string` access on the bare keyword because `formatQuery("gitlab", k)` returns the keyword verbatim (no wrapping syntax), avoiding a second `extractKeyword`-style helper.
## Deferred Issues
None.
## Known Stubs
None.
## Self-Check: PASSED
- pkg/recon/sources/gitlab.go — FOUND
- pkg/recon/sources/gitlab_test.go — FOUND
- .planning/phases/10-osint-code-hosting/10-03-SUMMARY.md — FOUND
- commit 0137dc5 — FOUND