# 2026 July 24

## websearch options

Yes — you **don’t need SearXNG on ailaba**. DESIGN already says search should be **pluggable**. For `localllm`, pick a backend that returns **title + URL + snippet**, then let Ollama write the digest to `aispace/search/`.

---

### Options (no ailaba SearXNG)

| Option | What it is | Good for | Caveats |
|--------|------------|----------|---------|
| **`ddgs` (DuckDuckGo)** | Python package; you used this in `ddgs.agent.py` | Fast, no local server, fits `kb search` | Rate limits; package renamed from `duckduckgo-search` → `ddgs`; needs Geph when blocked |
| **Firecrawl** | API: search + fetch/clean pages (Hermes notes mention it) | “Search then read page” in one step | API key; online service |
| **Gemini / ChatGPT web** | Model does search via provider | Easiest when VPN works | Not local; tied to one vendor |
| **Brave / Tavily / Exa / SerpAPI** | Paid/freemium search APIs | Stable JSON for agents | Keys + cost |
| **`urllib` scrape Google/Bing** | Raw HTTP + parse HTML | — | **Not recommended**: CAPTCHAs, ToS, breaks often |

“Firesearch” is probably **Firecrawl** ([firecrawl.dev](https://www.firecrawl.dev)) — search + extract, not just a search box.

---

### `urllib` alone?

**Fine for a known URL** (fetch one page you already have).

**Bad as “web search”:**
- Search pages aren’t simple static HTML
- You need ranking, snippets, many URLs
- Engines block bots quickly

Pattern that works:

```text
search API (ddgs / Firecrawl / …)  →  list of URLs
urllib/httpx fetch each URL        →  optional full text
Ollama digest                      →  aispace/search/<slug>.md
```

Same idea as `extract_topics.py`: **stdlib HTTP for the engine you choose**, not for scraping Google results by hand.

---

### What I’d use for `localllm` (simplest path)

You already proved **`ddgs`** in `userdocs/chat.log.md`:

```python
from ddgs import DDGS
with DDGS() as ddgs:
    results = ddgs.text("ollama lxc docker", max_results=5)
```

That maps cleanly to Phase 2:

```text
kb search "ollama host lxc client"
  → ddgs.text(...) or Firecrawl search
  → Ollama summarizes (like extract_topics)
  → aispace/search/ollama-host-lxc.md
```

No ailaba, no Docker-in-LXC, no port 8080.

---

### When to pick which

| You want… | Pick |
|-----------|------|
| Minimal deps, you already know it | **`ddgs`** |
| Search + clean page text for wiki | **Firecrawl** |
| Fully offline / no third-party search | **Skip web search**; use only `userdocs` + chat slices |
| Best quality, OK with cloud | **Gemini** (your `gemini.tmol`) with search enabled |

---

### Fit with topic work (ollama, docker, lxc…)

Web search is **enrichment**, not replacement for chat slices:

```text
aispace/topics/chats/…     ← your notes (primary)
aispace/search/…           ← fresh external digest (optional)
aispace/topics/ollama.md   ← merge slices + maybe one search digest
```

Example: tag `ollama` on 12 slices → promote to `topics/ollama.md` → optionally `kb search "ollama lxc remote host setup"` to fill gaps.

---

### Practical note

With Geph up, **`ddgs` on the host** is usually enough to start. SearXNG on ailaba is optional later if you want self-hosted, multi-engine search.

If you want this wired as `kb search` with `ddgs` first (Firecrawl as optional backend), switch to **Agent mode** and we can sketch the module alongside `extract_topics.py`.