Getting recommended by AI
More customers now ask ChatGPT, Claude, Perplexity, Copilot or Google's AI Mode for a recommendation instead of scrolling search results. These tools can only recommend you if they can read your site and understand who you are.
This check covers three things. Can each AI and search crawler access your site? Is your content readable without JavaScript? Is it structured so AI can quote it? It also covers Bing, because Bing's index feeds Copilot and DuckDuckGo, and ChatGPT search draws on it too.
Questions
- Should I block AI crawlers?
- Block training crawlers if you like (GPTBot, ClaudeBot, Google-Extended); that doesn't stop you appearing in AI answers. Never block the search crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot, Bingbot, Googlebot). If you do, those tools can't cite or recommend you.
- What is llms.txt?
- A proposed standard: a plain-text file at yoursite.com/llms.txt that summarises your business and links your key pages for AI tools. Support is still emerging, but it's quick to add and does no harm.
- Why does Bing matter if nobody uses it?
- Bing's index powers Microsoft Copilot and much of DuckDuckGo, and ChatGPT search draws on it. Being missing from Bing can mean being missing from several AI assistants at once.