My number one question about search APIs is always if they allow you to store and resyndicate results you get from them.
If I'm running an agent system but I'm not allowed to store the responses - or provide a "share transcript" button - that's a pretty significant limitation.
The answer to that question is inevitably buried deep in the terms. Here's the relevant section I found for Ceramic, in their list of things you can't do:
> (n) collect, aggregate, store, or compile Output, including search results, relevance scores, or rankings, for the purpose of creating or contributing to any database, dataset, index, or corpus, whether or not such database, dataset, index, or corpus is used for a purpose that competes with Ceramic; (o) resell, syndicate, or otherwise make Output available to any third party on a standalone basis or as a separately accessible component of another product or service; provided that you may display Output to your authorized end users within your own application so long as such Output is integrated into your application's functionality, is incident to the end user’s real-time query, and is not independently accessible, extractable, or downloadable by end users or third parties; or (p) retain, cache, or store Output beyond what is reasonably necessary to display such Output to your authorized end users in the ordinary and real-time course of use, unless expressly permitted in an applicable Order Form.
For those developers out there, the best is still Gemini Flash Lite 2.5 believe it or not. It gives you 1000 google searches per day for free. Compare to Flash Lite 3.x which is 5k PER MONTH and then a few pennies PER SEARCH. Nuts. Didn’t realize search was so expensive.
Perhaps realizing all of this, Google hasn’t yet deprecated 2.5, bit limits access to it to “those who have used it before.”
It’s really really good for low cost search!
show comments
binarymax
Why not use those providers directly? Does Cloudflare need to be in the middle of everything?
show comments
qznc
My coding agent uses the hister cli, i.e. a local index. That often requires me to seed it manually as a downside. The upside is that it caches website contents via browser plugin, which is a nice workaround for bot blocking.
sorry this might be a dumb question but I am not really clear if they have a pricing structure and how much that is. I couldnt find any pricing directly linked to the Web Search API but then i see references that you use 'AI Gateway credits', but I also couldnt find pricing or free limits for those ones as well. Can somebody cue me in?
jasonjmcghee
> All three support Zero Data Retention for requests made through Cloudflare
And then on the providers page:
Property Value
provider exa
Zero Data Retention No
show comments
karmakaze
I was just looking into these as DeepSeek Harness w/ Qwen3.8-27B relies heavily on search. I was going to go with Serper.dev[0] $1 per 1000 (or lower in quantity).
The providers[1] behind this Web Search API have very different rates:
Ceramic.ai: $0.25 per 1,000 requests
Linkup: $5.00 per 1,000 requests
Exa: $7.00 per 1,000 requests
Many people have outsourced the decision on who can access their websites to CloudFlare ("bot protection"), which incidentally makes these websites harder to access by bots working for humans.
Now there is an official paid search API, and I'm guessing the certified providers will be allowed through the Cloudflare "bot protection"?
This is very worrying.
sreekanth850
Create bot detection and bot protection, then sell crawlers. Is this the peak of hypocrisy?
show comments
freakynit
Tried one query on ceramic.ai (the default provider for cloudflare web search api): "qwen-3.8 flash next and rtx 5090 best inference setup" ... 0 results ... same query on google and ddg both yield proper results.
Then shortened the query to just "qwen-3.8 flash next" ... results came.. all unrelated. In fact, these were almost all paper links .... no relation to actual search term.
And I had thought that I finally had found a cheaper search alternative.
show comments
yellow_lead
Reselling APIs seems lazy to me. I wonder if CF plans to make their own provider. That's what I had assumed when I read the title.
show comments
hrideshmg
Surprised no one in this thread has mentioned running a self hosted search API.
I personally used to use Firecrawl's paid credits (got a bunch of em for free at an event) before I realized that they allow you to self-host your own instance (albeit missing some features I never use anyways).
It's been working really well for my agents, I even hosted a small observability tool that proxies the requests so I can see how many are failing and the percentages are always below 2%.
show comments
laumars
Lately I've been using SearXNG for personal models. It's free and seems ok thus far.
I wonder if the three search engines get access to cloudflare protected sites without any captcha or bot interventions
show comments
chews
I love that businesses that would've been considered too risky to get in are all the rage if they feed the LLM demon. Scraping other's results? NO PROBLEM! They carry so much traffic they can literally just syndicate their network pipeline and find a new bullets for the money gun.
EcommerceFlow
Spent a few months building a product scraper using a mad mash up of various LLMs, OCR, etc. The pricing for their providers is 3x-8x higher than something like Luna 5.6 w/ Web Search. Not sure what their differentiator is, unless they just wanted to launch something.
0fflineuser
I am pretty sure exa specifically say it trains on your data in it's privacy policy, so how can it be ZDR ?
I remember as I was looking at the available web tools for hermes agent not to long ago and looked through the keyless web providers privacy policies, which exa is one of them.
show comments
8bite
The pricing is so different between these:
ceramic.ai - $0.25 per 1,000 requests
Exa - $7.00 per 1,000 requests
Linkup - $5.00 per 1,000 requests
Does anyone have insights on the quality differences? Web search API pricing for AI agent usecases has always felt so expensive for what it is, but I have no grounding on the economics of running a web index.
EDIT: formatting
show comments
solaire_oa
Anthropic plausibly uses Brave Search... and Brave search maintains its own index. Makes sense: cheaper search API, leveraging non-Google, etc.
Here we are, one layer of indirection more: Ceramic, Exa, Linkup. Who knows what they use. If you told me that those 3 build and maintain their own index, I'd first question whether that was true, and if it is, I would question whether it was any good (relative to Google/Bing/Brave).
So what is CF providing here? Maybe some free credits to entice us to use their router? No, not that either ("billed to your AI Gateway credits"). Maybe a comparison of which agent search yields the best results? Nope.
It's a crappy proxy- probably less efficient and more volatile than hitting the agent API directly.
This is only if I understand the product correctly (which I admittedly skimmed) due to the sheer number of screeching vibey nothingburgers coming out of CF over the past month.
show comments
blurbleblurble
At first I thought this was a new web standard and was intrigued to hear what they'd come up with, sad to find otherwise
starcast2026
I am trying to understand the value.. This is for customers who have their agents already on CF? Improved latency & same eco-system etc., Right? Because others can always use Google Search APIs
show comments
blakeashleyjr
When I saw this, I assumed CF was going to offer an API to access the pages they otherwise protect.
NO SCRAPERS (except ours) -> $$$$$$$$$$$$$$$$
hmartin
I've been working on a TypeScript package to provide a unified search API across these providers:
So this gives a unified search experience without adding another cloud hop and dependency.
saltysalt
Wow I guess I am in the minority of folks building a search engine for humans now, this is a wild business model but best of luck to the 3 search index providers sitting behind this proxy, I hope it's worth their while financially speaking. Building an index is hard and expensive (I know).
duncangh
Cloudflare has been shipping more than FedEx lately. Would love to learn more about how they are going about this from a strategy, planning and execution standpoint.
anon373839
> All three support Zero Data Retention for requests made through Cloudflare
But does CloudFlare itself commit to zero data retention? If not, this isn’t too meaningful.
ronfriedhaber
Interesting to see how this can be compared with Exa, Alas, Cloudflare really is shipping many great orthogonal products recently.
show comments
Oras
Weird choice by CloudFlare, would been great if they have shared why it was created.
I use CloudFlare developer platform and quite happy with tools, but I didn’t use the gateway API and always used OpenRouter which does support web search.
I can see it useful for those who didn’t do any integrations or like to keep logs at one place, but did customers actually ask for this?
innocent_name
Of course Firecrawl isn't "Verified bot". Their customers were responsible for 95% of my traffic bill overcharge.
daft_pink
Wow, will they offers some sort of reduce a web page into a markdown file api as welL so we can get full web pages at reduced token sizes?
I guess I'm not understanding the value here - to compete with Google and the likes, the scale, cost and complexity would be huge. Appreciate new entrants in an existing field but not seeing this one.
show comments
ByeByeSpace
I run a small side project on Cloudflare Workers, so having search available right from a Worker without adding another vendor is appealing. Curious how the pricing compares to Brave's search API.
babelfish
What value does Cloudflare provide over using the linked providers directly...?
asjq178
Cloudflare protects against bots, Cloudflare sells out its customers to AI scrapers.
MITM service, Internet gatekeeper and robber baron.
show comments
corentin88
Funny how search was a graveyard for startups for almost two decades. And since ChatGPT releases (or so) it’s a trending place again.
show comments
0xbadcafebee
SearXNG works pretty well for my personal agents. FYI it's a free search gateway you can host locally, and there are many public instances. It's like the old days when many different people provided the same free service for all.
okokwhatever
Who thought 5 years ago that searching online would have a cost...
the zero-retention promise from the search providers is useful, but the requests also show up in gateway logs. can you keep the billing data without storing the actual search queries?
timpera
Interesting to see that they didn't include the Brave Search API, which is really great and imo a better experience than Exa.
The absence of the Perplexity Search API is to be expected though, knowing how much these two companies despise each other.
show comments
1vuio0pswjnm7
Explore Cloudflare's services without the web page bloat:
As a textmode command line and text-only browser user this textfile is faster for me to use that the usual Silicon Valley style web pages
Not quite as good as sitemap-0.xml but it's nice to have this in addition
Nan0pk
why are they selling search as a paid feature...
[deleted]
measurablefunc
Very cool.
zergrush
is this serpapi ?
OutOfHere
This has got to be a serious antitrust violation. First Cloudflare bans all the other bots, then it allows its liaised bots.
locitra
Search is an interesting building block for agentic systems. The challenge isn't just retrieving results, but deciding what to search for, evaluating the results, and determining when the information is sufficient to move to the next step.
As AI systems increasingly use search as a tool, the quality and reliability of that tool become an important part of the overall agent workflow.
My number one question about search APIs is always if they allow you to store and resyndicate results you get from them.
If I'm running an agent system but I'm not allowed to store the responses - or provide a "share transcript" button - that's a pretty significant limitation.
The answer to that question is inevitably buried deep in the terms. Here's the relevant section I found for Ceramic, in their list of things you can't do:
> (n) collect, aggregate, store, or compile Output, including search results, relevance scores, or rankings, for the purpose of creating or contributing to any database, dataset, index, or corpus, whether or not such database, dataset, index, or corpus is used for a purpose that competes with Ceramic; (o) resell, syndicate, or otherwise make Output available to any third party on a standalone basis or as a separately accessible component of another product or service; provided that you may display Output to your authorized end users within your own application so long as such Output is integrated into your application's functionality, is incident to the end user’s real-time query, and is not independently accessible, extractable, or downloadable by end users or third parties; or (p) retain, cache, or store Output beyond what is reasonably necessary to display such Output to your authorized end users in the ordinary and real-time course of use, unless expressly permitted in an applicable Order Form.
https://www.ceramic.ai/terms-of-service
Am I alone in caring about this?
For those developers out there, the best is still Gemini Flash Lite 2.5 believe it or not. It gives you 1000 google searches per day for free. Compare to Flash Lite 3.x which is 5k PER MONTH and then a few pennies PER SEARCH. Nuts. Didn’t realize search was so expensive.
Perhaps realizing all of this, Google hasn’t yet deprecated 2.5, bit limits access to it to “those who have used it before.”
It’s really really good for low cost search!
Why not use those providers directly? Does Cloudflare need to be in the middle of everything?
My coding agent uses the hister cli, i.e. a local index. That often requires me to seed it manually as a downside. The upside is that it caches website contents via browser plugin, which is a nice workaround for bot blocking.
Thanks asciimoo for https://github.com/asciimoo/hister
sorry this might be a dumb question but I am not really clear if they have a pricing structure and how much that is. I couldnt find any pricing directly linked to the Web Search API but then i see references that you use 'AI Gateway credits', but I also couldnt find pricing or free limits for those ones as well. Can somebody cue me in?
> All three support Zero Data Retention for requests made through Cloudflare
And then on the providers page:
I was just looking into these as DeepSeek Harness w/ Qwen3.8-27B relies heavily on search. I was going to go with Serper.dev[0] $1 per 1000 (or lower in quantity).
The providers[1] behind this Web Search API have very different rates:
[0] https://serper.dev/[1] https://developers.cloudflare.com/web-search/providers/
How does Cloudflare manage to hit the HN front page almost daily? Don't get me wrong, they build cool stuff, but the frequency is wild.
I've been quite satisfied with Kagi[1]'s API.
1: https://kagi.com/api/docs/openapi
Many people have outsourced the decision on who can access their websites to CloudFlare ("bot protection"), which incidentally makes these websites harder to access by bots working for humans.
Now there is an official paid search API, and I'm guessing the certified providers will be allowed through the Cloudflare "bot protection"?
This is very worrying.
Create bot detection and bot protection, then sell crawlers. Is this the peak of hypocrisy?
Tried one query on ceramic.ai (the default provider for cloudflare web search api): "qwen-3.8 flash next and rtx 5090 best inference setup" ... 0 results ... same query on google and ddg both yield proper results.
Then shortened the query to just "qwen-3.8 flash next" ... results came.. all unrelated. In fact, these were almost all paper links .... no relation to actual search term.
And I had thought that I finally had found a cheaper search alternative.
Reselling APIs seems lazy to me. I wonder if CF plans to make their own provider. That's what I had assumed when I read the title.
Surprised no one in this thread has mentioned running a self hosted search API.
I personally used to use Firecrawl's paid credits (got a bunch of em for free at an event) before I realized that they allow you to self-host your own instance (albeit missing some features I never use anyways).
It's been working really well for my agents, I even hosted a small observability tool that proxies the requests so I can see how many are failing and the percentages are always below 2%.
Lately I've been using SearXNG for personal models. It's free and seems ok thus far.
https://docs.searxng.org/
I wonder if the three search engines get access to cloudflare protected sites without any captcha or bot interventions
I love that businesses that would've been considered too risky to get in are all the rage if they feed the LLM demon. Scraping other's results? NO PROBLEM! They carry so much traffic they can literally just syndicate their network pipeline and find a new bullets for the money gun.
Spent a few months building a product scraper using a mad mash up of various LLMs, OCR, etc. The pricing for their providers is 3x-8x higher than something like Luna 5.6 w/ Web Search. Not sure what their differentiator is, unless they just wanted to launch something.
I am pretty sure exa specifically say it trains on your data in it's privacy policy, so how can it be ZDR ?
I remember as I was looking at the available web tools for hermes agent not to long ago and looked through the keyless web providers privacy policies, which exa is one of them.
The pricing is so different between these:
ceramic.ai - $0.25 per 1,000 requests
Exa - $7.00 per 1,000 requests
Linkup - $5.00 per 1,000 requests
Does anyone have insights on the quality differences? Web search API pricing for AI agent usecases has always felt so expensive for what it is, but I have no grounding on the economics of running a web index.
EDIT: formatting
Anthropic plausibly uses Brave Search... and Brave search maintains its own index. Makes sense: cheaper search API, leveraging non-Google, etc.
Here we are, one layer of indirection more: Ceramic, Exa, Linkup. Who knows what they use. If you told me that those 3 build and maintain their own index, I'd first question whether that was true, and if it is, I would question whether it was any good (relative to Google/Bing/Brave).
So what is CF providing here? Maybe some free credits to entice us to use their router? No, not that either ("billed to your AI Gateway credits"). Maybe a comparison of which agent search yields the best results? Nope.
It's a crappy proxy- probably less efficient and more volatile than hitting the agent API directly.
This is only if I understand the product correctly (which I admittedly skimmed) due to the sheer number of screeching vibey nothingburgers coming out of CF over the past month.
At first I thought this was a new web standard and was intrigued to hear what they'd come up with, sad to find otherwise
I am trying to understand the value.. This is for customers who have their agents already on CF? Improved latency & same eco-system etc., Right? Because others can always use Google Search APIs
When I saw this, I assumed CF was going to offer an API to access the pages they otherwise protect.
NO SCRAPERS (except ours) -> $$$$$$$$$$$$$$$$
I've been working on a TypeScript package to provide a unified search API across these providers:
https://github.com/hbmartin/agent-web-search
So this gives a unified search experience without adding another cloud hop and dependency.
Wow I guess I am in the minority of folks building a search engine for humans now, this is a wild business model but best of luck to the 3 search index providers sitting behind this proxy, I hope it's worth their while financially speaking. Building an index is hard and expensive (I know).
Cloudflare has been shipping more than FedEx lately. Would love to learn more about how they are going about this from a strategy, planning and execution standpoint.
> All three support Zero Data Retention for requests made through Cloudflare
But does CloudFlare itself commit to zero data retention? If not, this isn’t too meaningful.
Interesting to see how this can be compared with Exa, Alas, Cloudflare really is shipping many great orthogonal products recently.
Weird choice by CloudFlare, would been great if they have shared why it was created.
I use CloudFlare developer platform and quite happy with tools, but I didn’t use the gateway API and always used OpenRouter which does support web search.
I can see it useful for those who didn’t do any integrations or like to keep logs at one place, but did customers actually ask for this?
Of course Firecrawl isn't "Verified bot". Their customers were responsible for 95% of my traffic bill overcharge.
Wow, will they offers some sort of reduce a web page into a markdown file api as welL so we can get full web pages at reduced token sizes?
Same as https://staan.ai, the European index.
I guess I'm not understanding the value here - to compete with Google and the likes, the scale, cost and complexity would be huge. Appreciate new entrants in an existing field but not seeing this one.
I run a small side project on Cloudflare Workers, so having search available right from a Worker without adding another vendor is appealing. Curious how the pricing compares to Brave's search API.
What value does Cloudflare provide over using the linked providers directly...?
Cloudflare protects against bots, Cloudflare sells out its customers to AI scrapers.
MITM service, Internet gatekeeper and robber baron.
Funny how search was a graveyard for startups for almost two decades. And since ChatGPT releases (or so) it’s a trending place again.
SearXNG works pretty well for my personal agents. FYI it's a free search gateway you can host locally, and there are many public instances. It's like the old days when many different people provided the same free service for all.
Who thought 5 years ago that searching online would have a cost...
so this is like a competitor to https://parallel.ai/ ?
the zero-retention promise from the search providers is useful, but the requests also show up in gateway logs. can you keep the billing data without storing the actual search queries?
Interesting to see that they didn't include the Brave Search API, which is really great and imo a better experience than Exa.
The absence of the Perplexity Search API is to be expected though, knowing how much these two companies despise each other.
Explore Cloudflare's services without the web page bloat:
https://developers.cloudflare.com/llms.txt
As a textmode command line and text-only browser user this textfile is faster for me to use that the usual Silicon Valley style web pages
Not quite as good as sitemap-0.xml but it's nice to have this in addition
why are they selling search as a paid feature...
Very cool.
is this serpapi ?
This has got to be a serious antitrust violation. First Cloudflare bans all the other bots, then it allows its liaised bots.
Search is an interesting building block for agentic systems. The challenge isn't just retrieving results, but deciding what to search for, evaluating the results, and determining when the information is sufficient to move to the next step.
As AI systems increasingly use search as a tool, the quality and reliability of that tool become an important part of the overall agent workflow.