---
name: news-crawler
description: Fetch and analyze RSS news feeds by topic (iran, cyber, mideast, russia, ukraine, defense, intel, tech, turkey). Use when user asks about news, briefings, headlines, current events, or intelligence updates. Cache auto-refreshes every 4h — data is ALWAYS available.
version: 1.2.0
tags: [news, rss, intel, briefing, osint]
metadata:
  {
    "openclaw": {
      "emoji": "📰",
      "requires": {
        "bins": ["python3"]
      }
    }
  }
---

# News Crawler

RSS news cache: 36 topics, 3160+ feeds, 8 cached topics. **Cache is always fresh** — auto-refreshed every 4h by linux cron.

## HOW TO USE (IMPORTANT — READ THIS FIRST)

**You do NOT need to crawl anything.** The cache is already there. Just read it.

When user asks about news/headlines/briefing for any topic, run this ONE command:

```bash
bash /home/ali0606/news-crawler/scripts/quick-news.sh <topic>
```

That's it. No flags needed. Just the topic name.

### Examples

```bash
bash /home/ali0606/news-crawler/scripts/quick-news.sh iran
bash /home/ali0606/news-crawler/scripts/quick-news.sh cyber
bash /home/ali0606/news-crawler/scripts/quick-news.sh morning    # geopolitics overview
bash /home/ali0606/news-crawler/scripts/quick-news.sh russia
bash /home/ali0606/news-crawler/scripts/quick-news.sh mideast
bash /home/ali0606/news-crawler/scripts/quick-news.sh turkey
bash /home/ali0606/news-crawler/scripts/quick-news.sh defense
bash /home/ali0606/news-crawler/scripts/quick-news.sh intel
bash /home/ali0606/news-crawler/scripts/quick-news.sh ukraine
```

### Topic aliases (Turkish also works)

| User says | Topic |
|-----------|-------|
| iran, İran | iran |
| cyber, siber, cyber-cti | cyber |
| morning, evening, briefing, all | all geopolitics |
| russia, rusya | russia |
| ukraine, ukrayna | ukraine |
| mideast, ortadoğu | mideast (iran+turkey+region) |
| turkey, türkiye | turkey |
| defense, savunma | defense |
| intel, istihbarat | intel |

## Cache Details

- **Location:** `/home/ali0606/news-crawler/data/cache/`
- **Auto-refresh:** Every 4h via cron (`crawl-cache.sh`)
- **Cached topics (8):** iran, russia, ukraine, cyber-cti, mideast, turkey, defense, intel
- **Symlinks:** `{topic}_latest.json` → most recent crawl (e.g. `iran_latest.json`)
- **Format:** JSON with title, link, published date, source, summary per article

You can also read cache files directly:
```bash
cat /home/ali0606/news-crawler/data/cache/iran_latest.json | python3 -m json.tool | head -100
```

## OpenClaw Cron Briefings

6 cron jobs send automated briefings to Telegram using `ollama/llama3.1:latest`:
- morning (08:20), iran (09:20, 21:20), russia (10:20), mideast (11:20), cyber (13:20), evening (20:20)
- These read from the same cache, summarize with LLM, and deliver via Telegram

## Advanced (only if user specifically asks)

### Custom time/count
```bash
bash /home/ali0606/news-crawler/scripts/read-cache.sh <topic> --hours 24 --count 50 --compact
```

### Manual cache refresh (normally not needed)
```bash
bash /home/ali0606/news-crawler/scripts/crawl-cache.sh
```

### Live crawl (slow, 2-10 min — only for topics NOT in cache)
```bash
cd /home/ali0606/news-crawler && python3 src/export_feeds_async.py --topic <topic> --format json --max-content-length 500
```

### All 36 topics
```bash
cd /home/ali0606/news-crawler && python3 src/export_feeds_async.py --topics
```
