From 7020c57905771cc9d589fe44fd5f8bd55c6ebb34 Mon Sep 17 00:00:00 2001 From: salvacybersec Date: Mon, 6 Apr 2026 17:48:06 +0300 Subject: [PATCH] docs(17-05): complete serve & schedule CLI commands plan Co-Authored-By: Claude Opus 4.6 (1M context) --- .planning/ROADMAP.md | 4 +- .planning/STATE.md | 13 ++- .../17-telegram-scheduler/17-05-SUMMARY.md | 100 ++++++++++++++++++ 3 files changed, 110 insertions(+), 7 deletions(-) create mode 100644 .planning/phases/17-telegram-scheduler/17-05-SUMMARY.md diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 524c8a7..a2583e4 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -28,7 +28,7 @@ Decimal phases appear between their surrounding integers in numeric order. - [x] **Phase 14: OSINT CI/CD Logs, Web Archives & Frontend Leaks** - Build logs, Wayback Machine, and JS bundle/env scanning (completed 2026-04-06) - [x] **Phase 15: OSINT Forums, Collaboration & Log Aggregators** - StackOverflow/Reddit/HN, Notion/Trello, Elasticsearch/Grafana/Sentry (completed 2026-04-06) - [x] **Phase 16: OSINT Threat Intel, Mobile, DNS & API Marketplaces** - VirusTotal/IntelX, APK scanning, crt.sh, Postman/SwaggerHub (completed 2026-04-06) -- [ ] **Phase 17: Telegram Bot & Scheduled Scanning** - Remote control bot and cron-based recurring scans with auto-notify +- [x] **Phase 17: Telegram Bot & Scheduled Scanning** - Remote control bot and cron-based recurring scans with auto-notify (completed 2026-04-06) - [ ] **Phase 18: Web Dashboard** - Embedded htmx + Tailwind dashboard aggregating all subsystems with SSE live updates ## Phase Details @@ -377,5 +377,5 @@ Phases execute in numeric order: 1 → 2 → 3 → ... → 18 | 14. OSINT CI/CD Logs, Web Archives & Frontend Leaks | 1/1 | Complete | 2026-04-06 | | 15. OSINT Forums, Collaboration & Log Aggregators | 2/4 | Complete | 2026-04-06 | | 16. OSINT Threat Intel, Mobile, DNS & API Marketplaces | 0/? | Complete | 2026-04-06 | -| 17. Telegram Bot & Scheduled Scanning | 0/? | Not started | - | +| 17. Telegram Bot & Scheduled Scanning | 2/1 | Complete | 2026-04-06 | | 18. Web Dashboard | 0/? | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index 4d206f9..f1e7959 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -3,14 +3,14 @@ gsd_state_version: 1.0 milestone: v1.0 milestone_name: milestone status: executing -stopped_at: Completed 17-03-PLAN.md -last_updated: "2026-04-06T14:36:30.928Z" +stopped_at: Completed 17-05-PLAN.md +last_updated: "2026-04-06T14:47:58.267Z" last_activity: 2026-04-06 progress: total_phases: 18 completed_phases: 15 total_plans: 86 - completed_plans: 84 + completed_plans: 85 percent: 20 --- @@ -101,6 +101,7 @@ Progress: [██░░░░░░░░] 20% | Phase 15 P03 | 4min | 2 tasks | 11 files | | Phase 16 P01 | 4min | 2 tasks | 6 files | | Phase 17 P03 | 5min | 3 tasks | 6 files | +| Phase 17 P05 | 6min | 1 tasks | 10 files | ## Accumulated Context @@ -155,6 +156,8 @@ Recent decisions affecting current work: - [Phase 16]: URLhaus tag lookup with payload endpoint fallback - [Phase 17]: Telego v1.8.0 handler context pattern: *th.Context implements context.Context for scan/recon timeout propagation - [Phase 17]: /key enforced private-chat-only via chat.Type check; all other bot commands use masked keys +- [Phase 17]: Scheduler runs inside serve command process; schedule add/list/remove/run are standalone DB operations +- [Phase 17]: OnFindings callback pattern decouples scheduler from bot for Telegram notifications ### Pending Todos @@ -169,6 +172,6 @@ None yet. ## Session Continuity -Last session: 2026-04-06T14:36:30.924Z -Stopped at: Completed 17-03-PLAN.md +Last session: 2026-04-06T14:47:58.263Z +Stopped at: Completed 17-05-PLAN.md Resume file: None diff --git a/.planning/phases/17-telegram-scheduler/17-05-SUMMARY.md b/.planning/phases/17-telegram-scheduler/17-05-SUMMARY.md new file mode 100644 index 0000000..477d931 --- /dev/null +++ b/.planning/phases/17-telegram-scheduler/17-05-SUMMARY.md @@ -0,0 +1,100 @@ +--- +phase: "17" +plan: "05" +subsystem: cli-commands +tags: [telegram, scheduler, gocron, cobra, serve, schedule, cron] +dependency_graph: + requires: [bot-command-handlers, engine, storage, providers] + provides: [serve-command, schedule-command, scheduler-engine] + affects: [web-dashboard] +tech_stack: + added: [github.com/go-co-op/gocron/v2@v2.19.1] + patterns: [gocron-scheduler-with-db-backed-jobs, cobra-subcommand-crud] +key_files: + created: [cmd/serve.go, cmd/schedule.go, pkg/scheduler/scheduler.go, pkg/scheduler/source.go, pkg/storage/scheduled_jobs.go, pkg/storage/scheduled_jobs_test.go] + modified: [cmd/stubs.go, pkg/storage/schema.sql, go.mod, go.sum] +decisions: + - "Scheduler runs inside serve command process; schedule add/list/remove/run are standalone DB operations" + - "gocron v2 job registration uses CronJob with 5-field cron expressions" + - "OnFindings callback on Scheduler allows serve to wire Telegram notifications without coupling" + - "scheduled_jobs table stores enabled/notify flags for per-job control" +metrics: + duration: 6min + completed: "2026-04-06" +--- + +# Phase 17 Plan 05: Serve & Schedule CLI Commands Summary + +**cmd/serve.go starts scheduler + optional Telegram bot; cmd/schedule.go provides add/list/remove/run CRUD for cron-based recurring scan jobs backed by SQLite** + +## Performance + +- **Duration:** 6 min +- **Started:** 2026-04-06T14:41:07Z +- **Completed:** 2026-04-06T14:47:00Z +- **Tasks:** 1 (combined) +- **Files modified:** 10 + +## Accomplishments +- Replaced serve and schedule stubs with real implementations +- Scheduler package wraps gocron v2 with DB-backed job persistence +- Serve command starts scheduler and optionally Telegram bot with --telegram flag +- Schedule subcommands provide full CRUD: add (--cron, --scan, --name, --notify), list, remove, run + +## Task Commits + +1. **Task 1: Implement serve, schedule commands + scheduler package + storage layer** - `292ec24` (feat) + +## Files Created/Modified +- `cmd/serve.go` - Serve command: starts scheduler, optionally Telegram bot with --telegram flag +- `cmd/schedule.go` - Schedule command with add/list/remove/run subcommands +- `cmd/stubs.go` - Removed serve and schedule stubs +- `pkg/scheduler/scheduler.go` - Scheduler wrapping gocron v2 with DB job loading, OnFindings callback +- `pkg/scheduler/source.go` - Source selection for scheduled scan paths +- `pkg/storage/schema.sql` - Added scheduled_jobs table with indexes +- `pkg/storage/scheduled_jobs.go` - CRUD operations for scheduled_jobs table +- `pkg/storage/scheduled_jobs_test.go` - Tests for job CRUD and last_run update +- `go.mod` - Added gocron/v2 v2.19.1 dependency +- `go.sum` - Updated checksums + +## Decisions Made +1. Scheduler lives in pkg/scheduler, decoupled from cmd layer via Deps struct injection +2. OnFindings callback pattern allows serve.go to wire Telegram notification without pkg/scheduler knowing about pkg/bot +3. schedule add/list/remove/run are standalone DB operations (no running scheduler needed) +4. schedule run executes scan immediately using same engine/storage as scan command +5. parseNullTime handles multiple SQLite datetime formats (space-separated and ISO 8601) + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 1 - Bug] Fixed parseNullTime to handle multiple SQLite datetime formats** +- **Found during:** Task 1 (scheduled_jobs_test.go) +- **Issue:** SQLite returned datetime as `2026-04-06T17:45:53Z` but parser only handled `2006-01-02 15:04:05` +- **Fix:** Added multiple format fallback in parseNullTime +- **Files modified:** pkg/storage/scheduled_jobs.go +- **Verification:** TestUpdateJobLastRun passes + +**2. [Rule 3 - Blocking] Renamed truncate to truncateStr to avoid redeclaration with dorks.go** +- **Found during:** Task 1 (compilation) +- **Issue:** truncate function already declared in cmd/dorks.go +- **Fix:** Renamed to truncateStr in schedule.go +- **Files modified:** cmd/schedule.go + +--- + +**Total deviations:** 2 auto-fixed (1 bug, 1 blocking) +**Impact on plan:** Both essential for correctness. No scope creep. + +## Issues Encountered +None beyond the auto-fixed items above. + +## Known Stubs +None. All commands are fully wired to real implementations. + +## Next Phase Readiness +- Serve command ready for Phase 18 web dashboard (--port flag reserved) +- Scheduler operational for all enabled DB-stored jobs +- Telegram bot integration tested via existing Phase 17 Plan 03 handlers + +## Self-Check: PASSED