Research

Writing that cites nothing is writing the critic can’t verify. Research is on by default (use_researcher: true) - turn it off and the critic treats specific uncited statistics as fabrication risks.

Tier 1: shallow (default)

One search query per chapter/section → snippets → a short brief (facts + style cues) injected into the writer. Cheap, fast, cached on disk for 7 days.

Search providers

The search backend is chosen with search_provider:

Provider	Behavior
duckduckgo (default)	Uses duckduckgo-search - no API key, no cost
firecrawl	The Firecrawl search API - needs FIRECRAWL_API_KEY in .env; paid, but more reliable under volume, and it also switches deep-research page scraping to Firecrawl

Selecting firecrawl without a key degrades gracefully back to duckduckgo, and a Firecrawl search failure falls back to duckduckgo before giving up - a run never dies on a search backend.

Tier 2: deep multi-source (deep_research: true)

focus → LLM query expansion (3-5 distinct queries)
      → concurrent search fan-out → URL dedup + max 2 per domain
      → full PAGE TEXT fetch of the top 6 sources (concurrent)
      → cross-source synthesis that cites sources by number

The synthesis reads across full pages, not snippets - it can pull specific numbers and quotes, and it flags where sources disagree. For articles, the real fetched URLs become the persisted references.

How much text the verifier reads

When verify_claims is on, the verifier checks each cited claim against its source’s full text. verify_excerpt_chars (default 6000) caps how many characters per source the verifier reads - larger catches more paraphrases at higher token cost; 0 means no cap.

The fetch backend is pluggable

• Scrapo (optional, pip install -e ".[deep]"): returns clean page markdown and escalates HTTP → browser → stealth on real failure signals, reaching JS-rendered and soft-blocked pages.
• Stdlib fallback (always available): urllib + an html.parser extractor - zero extra dependencies, CI-green everywhere.

The fetch safety gate

Search results (and the LLM’s query expansion behind them) decide which URLs get fetched - so every uncached fetch passes three checks:

Check	Behavior
SSRF guard	The host must resolve to globally-routable addresses only - loopback, private ranges, link-local, and cloud-metadata IPs are blocked, and every redirect hop is re-validated
robots.txt	Honored per host (cached for the process); unreachable robots = allow; WRITINGAGENT_IGNORE_ROBOTS=1 opts out
Politeness	Requests to the same host are spaced ≥ 1 second apart

Prompt-injection defense

Everything fetched from the web is fenced before it touches a prompt: data-only markers (spoofed markers inside the content are neutralized) plus a standing instruction that the block is never instructions. A hostile page can’t steer the writer.