Files
keyhunter/.planning/phases/10-osint-code-hosting/10-03-SUMMARY.md
2026-04-06 01:16:34 +03:00

4.0 KiB
Raw Blame History

phase, plan, subsystem, tags, requires, provides, affects, tech_stack_added, patterns, key_files_created, key_files_modified, decisions, metrics
phase plan subsystem tags requires provides affects tech_stack_added patterns key_files_created key_files_modified decisions metrics
10-osint-code-hosting 03 recon/sources
recon
osint
gitlab
wave-2
pkg/recon/sources.Client (Plan 10-01)
pkg/recon/sources.BuildQueries (Plan 10-01)
pkg/recon.LimiterRegistry (Phase 9)
pkg/providers.Registry
pkg/recon/sources.GitLabSource
pkg/recon/sources
Per-keyword BuildQueries loop driving search API calls
PRIVATE-TOKEN header auth with shared retry-aware Client
Disabled-when-empty-token semantics (Sweep returns nil with no requests)
Bare-keyword → provider-name lookup via local keyword index
pkg/recon/sources/gitlab.go
pkg/recon/sources/gitlab_test.go
Bare keyword BuildQueries output (gitlab case in formatQuery) — reverse lookup is a direct map[string]string access
gitlabKeywordIndex helper named with gitlab prefix to avoid collision with peer github.go keywordIndex during parallel wave
Finding.Source uses constructed /projects/<id>/-/blob/<ref>/<path> URL (per plan) rather than extra /api/v4/projects/<id> lookup to keep request budget tight
Confidence=low across all recon findings; Phase 5 verify promotes to high
duration completed_date tasks_completed tests_added
~8 minutes 2026-04-05 1 6

Phase 10 Plan 03: GitLabSource Summary

GitLabSource is a thin recon.ReconSource that queries GitLab's /api/v4/search?scope=blobs endpoint with a PRIVATE-TOKEN header, iterating one search call per provider keyword from the shared BuildQueries helper and emitting a Finding per returned blob with Source pointing at a constructed projects/<id>/-/blob/<ref>/<path> URL.

What Was Built

pkg/recon/sources/gitlab.go contains:

  • GitLabSource struct exposing Token, BaseURL, Registry, Limiters (lazy Client)
  • ReconSource interface methods: Name()="gitlab", RateLimit()=rate.Every(30ms), Burst()=5, RespectsRobots()=false, Enabled() (token non-empty), Sweep()
  • glBlob response DTO matching GitLab's documented blob search schema
  • gitlabKeywordIndex() local helper (prefixed to avoid colliding with peer plan helpers during parallel wave execution)
  • Compile-time var _ recon.ReconSource = (*GitLabSource)(nil) assertion

pkg/recon/sources/gitlab_test.go covers all behaviors the plan called out:

Test Verifies
TestGitLabSource_EnabledFalseWhenTokenEmpty Enabled gating + Name/RespectsRobots accessors
TestGitLabSource_EmptyToken_NoCallsNoError No HTTP request issued when Token==""
TestGitLabSource_Sweep_EmitsFindings PRIVATE-TOKEN header, scope=blobs, two queries × two blobs = 4 Findings, Source URLs contain project_id/ref/path
TestGitLabSource_Unauthorized 401 propagates as errors.Is(err, ErrUnauthorized)
TestGitLabSource_CtxCancellation Sweep returns promptly on ctx timeout against a hanging server
TestGitLabSource_InterfaceAssertion Static recon.ReconSource conformance

Verification

go build ./...                                       # clean
go test ./pkg/recon/sources/ -run TestGitLab -v      # 6/6 PASS
go test ./pkg/recon/sources/                         # full package PASS (3.164s)

Deviations from Plan

None for must-have behavior. Two minor adjustments:

  1. keywordIndex helper renamed to gitlabKeywordIndex because pkg/recon/sources/github.go (Plan 10-02, wave-2 sibling) introduces an identically-named package-level symbol. Prefixing prevents a redeclared-identifier build failure when the parallel wave merges.
  2. Provider name lookup simplified to direct map[string]string access on the bare keyword because formatQuery("gitlab", k) returns the keyword verbatim (no wrapping syntax), avoiding a second extractKeyword-style helper.

Deferred Issues

None.

Known Stubs

None.

Self-Check: PASSED

  • pkg/recon/sources/gitlab.go — FOUND
  • pkg/recon/sources/gitlab_test.go — FOUND
  • .planning/phases/10-osint-code-hosting/10-03-SUMMARY.md — FOUND
  • commit 0137dc5 — FOUND