Skip to main content
POST /web_search/contents turns up to 20 URLs into clean, model-ready content in one request. Every parameter and schema is in the API Reference; this page covers behavior.

​
Formats Are Best-Effort Per URL

Every result carries url, and title whenever the page has one. Beyond that, which fields come back varies by URL and cache state:
  • With formats omitted, html and metadata are the defaults.
  • Requesting markdown does not suppress html — a freshly crawled page may return both, while cached content returns exactly the requested formats. If you only need markdown, ignore the html field in the response.
  • metadata.site_name is frequently an empty string; rely on title instead.

​
Caching

Content is served from a crawl cache when possible: cached responses return in well under a second, fresh crawls take one to a few seconds per URL. max_age bounds how old a cached copy may be, and crawl_timeout caps each URL’s fetch time.