docs(14-01): complete CI/CD log sources plan
- 5 sources: GitHubActions, TravisCI, CircleCI, Jenkins, GitLabCI - RegisterAll at 45 sources total
This commit is contained in:
salvacybersec
@@ -3,14 +3,14 @@ gsd_state_version: 1.0
|
||||
milestone: v1.0
|
||||
milestone_name: milestone
|
||||
status: executing
|
||||
stopped_at: Completed 13-04-PLAN.md
|
||||
last_updated: "2026-04-06T10:06:43.774Z"
|
||||
stopped_at: Completed 14-01-PLAN.md
|
||||
last_updated: "2026-04-06T10:18:24.542Z"
|
||||
last_activity: 2026-04-06
|
||||
progress:
|
||||
total_phases: 18
|
||||
completed_phases: 13
|
||||
total_plans: 73
|
||||
completed_plans: 74
|
||||
completed_plans: 75
|
||||
percent: 20
|
||||
---
|
||||
|
||||
@@ -96,6 +96,7 @@ Progress: [██░░░░░░░░] 20%
|
||||
| Phase 13 P02 | 3min | 2 tasks | 8 files |
|
||||
| Phase 13 P03 | 5min | 2 tasks | 11 files |
|
||||
| Phase 13 P04 | 5min | 2 tasks | 3 files |
|
||||
| Phase 14 P01 | 4min | 1 tasks | 14 files |
|
||||
|
||||
## Accumulated Context
|
||||
|
||||
@@ -142,6 +143,7 @@ Recent decisions affecting current work:
|
||||
- [Phase 13]: KubernetesSource uses Artifact Hub rather than Censys/Shodan dorking to avoid duplicating Phase 12 sources
|
||||
- [Phase 13]: RegisterAll extended to 32 sources (28 Phase 10-12 + 4 Phase 13 container/IaC)
|
||||
- [Phase 13]: RegisterAll extended to 40 sources (28 Phase 10-12 + 12 Phase 13); package registry sources credentialless, no new SourcesConfig fields
|
||||
- [Phase 14]: RegisterAll extended to 45 sources (40 Phase 10-13 + 5 Phase 14 CI/CD); CircleCI gets dedicated CIRCLECI_TOKEN
|
||||
|
||||
### Pending Todos
|
||||
|
||||
@@ -156,6 +158,6 @@ None yet.
|
||||
|
||||
## Session Continuity
|
||||
|
||||
Last session: 2026-04-06T10:04:38.660Z
|
||||
Stopped at: Completed 13-04-PLAN.md
|
||||
Last session: 2026-04-06T10:18:24.538Z
|
||||
Stopped at: Completed 14-01-PLAN.md
|
||||
Resume file: None
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
---
|
||||
phase: 14-osint_ci_cd_logs_web_archives_frontend_leaks
|
||||
plan: 01
|
||||
subsystem: recon
|
||||
tags: [ci-cd, github-actions, travis-ci, circleci, jenkins, gitlab-ci, osint]
|
||||
|
||||
requires:
|
||||
- phase: 10-osint-code-hosting
|
||||
provides: ReconSource interface, shared Client, BuildQueries, LimiterRegistry
|
||||
- phase: 13-osint_package_registries_container_iac
|
||||
provides: RegisterAll with 40 sources baseline
|
||||
|
||||
provides:
|
||||
- GitHubActionsSource for GitHub Actions workflow log scanning
|
||||
- TravisCISource for Travis CI public build log scanning
|
||||
- CircleCISource for CircleCI pipeline log scanning
|
||||
- JenkinsSource for open Jenkins console output scanning
|
||||
- GitLabCISource for GitLab CI pipeline log scanning
|
||||
- RegisterAll extended to 45 sources
|
||||
|
||||
affects: [14-02, 14-03, 14-04, 14-05, recon-engine]
|
||||
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns: [credential-gated CI/CD sources, credentialless scraping sources]
|
||||
|
||||
key-files:
|
||||
created:
|
||||
- pkg/recon/sources/githubactions.go
|
||||
- pkg/recon/sources/githubactions_test.go
|
||||
- pkg/recon/sources/travisci.go
|
||||
- pkg/recon/sources/travisci_test.go
|
||||
- pkg/recon/sources/circleci.go
|
||||
- pkg/recon/sources/circleci_test.go
|
||||
- pkg/recon/sources/jenkins.go
|
||||
- pkg/recon/sources/jenkins_test.go
|
||||
- pkg/recon/sources/gitlabci.go
|
||||
- pkg/recon/sources/gitlabci_test.go
|
||||
modified:
|
||||
- pkg/recon/sources/register.go
|
||||
- pkg/recon/sources/register_test.go
|
||||
- pkg/recon/sources/integration_test.go
|
||||
- cmd/recon.go
|
||||
|
||||
key-decisions:
|
||||
- "GitHubActions and GitLabCI reuse existing GitHub/GitLab tokens from SourcesConfig; CircleCI gets its own CIRCLECI_TOKEN"
|
||||
- "TravisCI and Jenkins are credentialless (public API access); GitHubActions, CircleCI, GitLabCI are credential-gated"
|
||||
- "RegisterAll extended to 45 sources (40 Phase 10-13 + 5 Phase 14 CI/CD)"
|
||||
|
||||
patterns-established:
|
||||
- "CI/CD sources follow same ReconSource pattern as all prior sources"
|
||||
|
||||
requirements-completed: []
|
||||
|
||||
duration: 4min
|
||||
completed: 2026-04-06
|
||||
---
|
||||
|
||||
# Phase 14 Plan 01: CI/CD Log Sources Summary
|
||||
|
||||
**Five CI/CD build log sources (GitHubActions, TravisCI, CircleCI, Jenkins, GitLabCI) for detecting API keys leaked in CI/CD pipeline outputs**
|
||||
|
||||
## Performance
|
||||
|
||||
- **Duration:** 4 min 32s
|
||||
- **Started:** 2026-04-06T10:13:06Z
|
||||
- **Completed:** 2026-04-06T10:17:38Z
|
||||
- **Tasks:** 1
|
||||
- **Files modified:** 14
|
||||
|
||||
## Accomplishments
|
||||
- Implemented 5 CI/CD log scanning sources following established ReconSource pattern
|
||||
- GitHubActions searches GitHub code search for workflow YAML files referencing provider keywords
|
||||
- TravisCI queries Travis CI v3 API for public build logs
|
||||
- CircleCI queries CircleCI v2 pipeline API for build pipelines
|
||||
- JenkinsSource queries open Jenkins /api/json for job build consoles
|
||||
- GitLabCISource queries GitLab projects API filtered for CI-enabled projects
|
||||
- All 5 sources integrated into RegisterAll (45 total), with full integration test coverage
|
||||
|
||||
## Task Commits
|
||||
|
||||
Each task was committed atomically:
|
||||
|
||||
1. **Task 1: Implement 5 CI/CD sources + tests + wiring** - `e0f267f` (feat)
|
||||
|
||||
## Files Created/Modified
|
||||
- `pkg/recon/sources/githubactions.go` - GitHub Actions workflow log source (token-gated)
|
||||
- `pkg/recon/sources/githubactions_test.go` - Unit tests with httptest fixture
|
||||
- `pkg/recon/sources/travisci.go` - Travis CI public build log source (credentialless)
|
||||
- `pkg/recon/sources/travisci_test.go` - Unit tests with httptest fixture
|
||||
- `pkg/recon/sources/circleci.go` - CircleCI pipeline source (token-gated)
|
||||
- `pkg/recon/sources/circleci_test.go` - Unit tests with httptest fixture
|
||||
- `pkg/recon/sources/jenkins.go` - Jenkins console output source (credentialless)
|
||||
- `pkg/recon/sources/jenkins_test.go` - Unit tests with httptest fixture
|
||||
- `pkg/recon/sources/gitlabci.go` - GitLab CI pipeline source (token-gated)
|
||||
- `pkg/recon/sources/gitlabci_test.go` - Unit tests with httptest fixture
|
||||
- `pkg/recon/sources/register.go` - Extended RegisterAll to 45 sources, added CircleCIToken to SourcesConfig
|
||||
- `pkg/recon/sources/register_test.go` - Updated expected source count and name list to 45
|
||||
- `pkg/recon/sources/integration_test.go` - Added fixtures and source registrations for all 5 new sources
|
||||
- `cmd/recon.go` - Wired CIRCLECI_TOKEN env var into SourcesConfig
|
||||
|
||||
## Decisions Made
|
||||
- GitHubActions and GitLabCI reuse existing GitHub/GitLab tokens; CircleCI gets dedicated CIRCLECI_TOKEN
|
||||
- TravisCI and Jenkins are credentialless (target public/open instances); other 3 are credential-gated
|
||||
- RegisterAll extended to 45 sources total
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
None - plan executed exactly as written.
|
||||
|
||||
## Issues Encountered
|
||||
None
|
||||
|
||||
## User Setup Required
|
||||
None - no external service configuration required.
|
||||
|
||||
## Next Phase Readiness
|
||||
- 5 CI/CD sources ready for production use
|
||||
- RegisterAll wires all 45 sources; future Phase 14 plans (web archives, frontend leaks) will extend to 50+
|
||||
|
||||
---
|
||||
*Phase: 14-osint_ci_cd_logs_web_archives_frontend_leaks*
|
||||
*Completed: 2026-04-06*
|
||||
Reference in New Issue
Block a user