100 lines
4.5 KiB
Markdown
100 lines
4.5 KiB
Markdown
---
|
|
phase: 10-osint-code-hosting
|
|
plan: 05
|
|
subsystem: recon
|
|
tags: [codeberg, gitea, osint, rest-api, httptest]
|
|
|
|
requires:
|
|
- phase: 09-osint-infrastructure
|
|
provides: ReconSource interface, LimiterRegistry, Engine
|
|
- phase: 10-osint-code-hosting/01
|
|
provides: shared sources.Client (retry/backoff), BuildQueries helper
|
|
provides:
|
|
- CodebergSource implementing recon.ReconSource against Gitea REST API
|
|
- Reusable pattern for any Gitea-compatible instance via BaseURL override
|
|
- Dual-mode rate limiting (unauth 60/hr, auth ~1000/hr)
|
|
affects: [10-09 register-all, future Gitea-compatible sources, verification pipeline]
|
|
|
|
tech-stack:
|
|
added: []
|
|
patterns:
|
|
- "Keyword → ProviderName index built at Sweep() entry to re-attribute BuildQueries output"
|
|
- "BaseURL override enables generic Gitea targeting"
|
|
- "httptest.Server with request-capturing handlers for header presence/absence assertions"
|
|
|
|
key-files:
|
|
created:
|
|
- pkg/recon/sources/codeberg.go
|
|
- pkg/recon/sources/codeberg_test.go
|
|
modified: []
|
|
|
|
key-decisions:
|
|
- "Sweep ignores its query argument and iterates provider keywords, matching sibling code-hosting sources"
|
|
- "Findings use Confidence=low since /repos/search matches repo metadata, not file contents — verification downstream separates real hits"
|
|
- "Token is optional; Enabled() always returns true because public API works anonymously"
|
|
- "DefaultCodebergBaseURL constant exported so Plan 10-09 can point at alternate Gitea hosts"
|
|
|
|
patterns-established:
|
|
- "Dual-mode rate limiting: if Token == \"\" return unauth rate else auth rate"
|
|
- "Per-source httptest suite covers: interface assertion, rate limits, decoding, header auth presence, header auth absence, ctx cancellation"
|
|
|
|
requirements-completed: [RECON-CODE-05]
|
|
|
|
duration: ~10min
|
|
completed: 2026-04-05
|
|
---
|
|
|
|
# Phase 10 Plan 05: CodebergSource Summary
|
|
|
|
**Gitea REST API source targeting Codeberg.org via /api/v1/repos/search with optional token auth and dual-mode rate limiting.**
|
|
|
|
## Performance
|
|
|
|
- **Duration:** ~10 min
|
|
- **Started:** 2026-04-05T22:07:00Z
|
|
- **Completed:** 2026-04-05T22:17:31Z
|
|
- **Tasks:** 1 (TDD)
|
|
- **Files modified:** 2 created
|
|
|
|
## Accomplishments
|
|
- CodebergSource implements recon.ReconSource with compile-time assertion
|
|
- Unauthenticated operation against /api/v1/repos/search (60/hour rate limit)
|
|
- Optional token mode sends `Authorization: token <t>` and raises limit to ~1000/hour
|
|
- Findings keyed to repo html_url with SourceType="recon:codeberg" and ProviderName resolved via keyword→provider index
|
|
- Shared sources.Client handles retries/429s; no bespoke HTTP logic in the source
|
|
- Six httptest-backed tests covering interface, both rate modes, sweep decoding, auth header presence/absence, and context cancellation
|
|
|
|
## Task Commits
|
|
|
|
1. **Task 1: CodebergSource + tests (TDD combined)** — `4fafc01` (feat)
|
|
|
|
## Files Created/Modified
|
|
- `pkg/recon/sources/codeberg.go` — CodebergSource struct, rate mode selection, Sweep over /api/v1/repos/search
|
|
- `pkg/recon/sources/codeberg_test.go` — httptest fixtures for all six behaviors
|
|
|
|
## Decisions Made
|
|
- TDD RED+GREEN collapsed into a single commit because the file pair is small and was verified end-to-end in one iteration (all six tests pass on first green build).
|
|
- `Confidence="low"` on emitted Findings: repo-metadata match is a weak signal until content verification runs.
|
|
- `Sweep` ignores the `query` parameter; the plan specifies driving queries from the provider registry via `BuildQueries`, consistent with sibling code-hosting sources.
|
|
|
|
## Deviations from Plan
|
|
None — plan executed exactly as written.
|
|
|
|
## Issues Encountered
|
|
- **Worktree path confusion (environmental, not code):** Initial Write tool calls targeted the main repo path instead of the active worktree. Files silently failed to persist and `go test` surfaced unrelated pre-existing `github_test.go` references in the main repo. Recovered by writing into the worktree path `/home/salva/Documents/apikey/.claude/worktrees/agent-a2637f83/`. No code changes resulted from this; purely a path fix.
|
|
|
|
## Next Phase Readiness
|
|
- Ready for Plan 10-09 (RegisterAll) to wire CodebergSource into `RegisterAll` with `cfg.CodebergToken` (field to be added when 10-09 finalizes SourcesConfig).
|
|
- No blockers.
|
|
|
|
## Self-Check: PASSED
|
|
- FOUND: pkg/recon/sources/codeberg.go
|
|
- FOUND: pkg/recon/sources/codeberg_test.go
|
|
- FOUND: commit 4fafc01
|
|
- Tests: 6/6 passing (`go test ./pkg/recon/sources/ -run TestCodeberg -v`)
|
|
- Package: `go vet` clean, full package tests green
|
|
|
|
---
|
|
*Phase: 10-osint-code-hosting*
|
|
*Completed: 2026-04-05*
|