Do AI assistants read my website?

By the Pickgap team · Updated Oct 15, 2026 · 1 min read

Sometimes. When an AI assistant searches the web and your site comes up in the results, it can read your pages, if its crawler isn't blocked and your content is in plain HTML. But for local recommendations, assistants often rely more on directories and review sites than on business websites. Your site's job is to confirm and add detail to what others say.

When they read it

• During a search, when your page is among the results for the question.
• Ahead of time, when their search crawlers index your site, like OAI-SearchBot or PerplexityBot.
• On request, when a user pastes your URL or asks the assistant to look at your site.
• Google's AI uses Google's index, so whatever Googlebot sees is available to AI Overviews and AI Mode.

What can stop them

• Blocked crawlers in robots.txt, or by your CDN's bot settings.
• Content loaded by JavaScript. Several AI crawlers read raw HTML and don't run scripts.
• Text in images, PDFs or widgets.
• Pages behind logins or pop-ups.
• Very slow pages that time out.

What they look for

Plain facts that answer the question: what you do, where, when you're open, what it costs or which insurance you take, and the questions customers ask. See how to write a homepage AI can understand.

A quick test

Paste your homepage URL into ChatGPT or Perplexity and ask: "Based on this page, what does this business do, where is it, what are its hours and prices?" If the answer is vague or wrong, the page needs work. Then check robots.txt and your CDN settings. See robots.txt for AI crawlers.