website indexing

Cloudflare is helping OpenAI crawl the web it once helped publishers protect

July 10, 2026 by Brian Fagioli

Cloudflare says it wants to protect publishers from aggressive AI crawlers. Now it is helping OpenAI discover and index web content more efficiently.

Add us to
Google

NERDS.xyz
Newsletter

⚡

Support
NERDS

Enjoying NERDS.xyz?
Send a Lightning tip.

Lenovo Tab Plus Gen 2 packs nine JBL speakers and AI features into a $400 Android tablet
Sovereign AI sounds independent until you notice everything still runs on NVIDIA
Q&A: Ditto says social media became sterile and the internet should be fun again
Tim Cook is stepping down and Apple desperately needs this reset
Joanna Stern’s exit from The Wall Street Journal is a shock, and a sign of the times

Search for: