Skip to content
NERDS.xyz
  • Home
  • AI
  • Linux
  • Open source
  • Hardware
  • Security
  • All Articles

AI crawler

Cloudflare says Perplexity is using hidden crawlers to bypass site blocks and I’m honestly shocked

August 4, 2025 by Brian Fagioli
stealth crawler security

Cloudflare has accused Perplexity of disguising its crawlers and ignoring robots.txt rules, a shocking move that could have legal consequences.

Categories Artificial Intelligence Tags AI crawler, bot evasion, ChatGPT, Cloudflare, hidden bots, OpenAI, Perplexity, robots.txt, site blocking, web scraping Leave a comment

Join our newsletter!

Real tech news for real nerds

Join
Subscribe via RSS

Featured Articles

  • Tim Cook is stepping down and Apple desperately needs this reset
  • Joanna Stern’s exit from The Wall Street Journal is a shock, and a sign of the times
  • Stop paying TurboTax when IRS Free File covers most taxpayers for FREE
  • Steve Jobs honored on new 2026 U.S. coin celebrating innovation
  • Sora 2 AI is making disturbing Martin Luther King Jr. videos and it needs to stop
© 2026
NERDS.xyz
All rights reserved
About Contact Privacy Policy Editorial Policy
© 2026 NERDS.xyz • Built with GeneratePress