AI crawlers in robots.txt
GPTBot, Google-Extended and PerplexityBot must be allowed to fetch your pages. Blocking them hides you from AI search entirely.
Free tool · GEO / AI search
Is your website invisible to AI?
AI crawlers read your raw HTML — not a rendered browser. Run a free 30-second check and see which of ten technical factors stop ChatGPT, Perplexity and Google AI Overviews from reading and citing your site.
Free checker
Technical AI visibility score
Most local business sites score poorly because content lives behind JavaScript, AI bots are blocked in robots.txt, or there is no llms.txt and FAQ schema. This tool shows exactly what is failing — the same signals we fix with GEO andentity pages.
Functional tool: enter a website URL to score ten technical factors that affect whether AI crawlers can read and cite your site.
Technical AI visibility checkLive
Scanning raw HTML, robots.txt and llms.txt…
What we check
Ten factors AI crawlers use
Inspired by how answer engines classify pages — adapted for UK trades, professional services and local SEO. Ecommerce stores: use the sister checker onloudcrowd.agency.
llms.txt and ai.txt
Machine-readable manifests point AI systems to your entity facts, service pages and citation URLs — the same pattern we use on /info-for-ai/ and /llms.txt.
Structured data & FAQs
JSON-LD Organization, Service and FAQPage schema give AI extractable facts. Semantic FAQ lists (<dl>) help Google AI Overviews and ChatGPT cite direct answers.
Readable HTML (not JS-only)
AI crawlers often skip JavaScript. If your H1, body copy and schema only exist after client-side rendering, you are invisible — regardless of how good the content is.
Google guidance · GEO
Agent-friendly sites vs citation-ready sites
Google’s web.dev guidance covers AI agents that browse and act on your site. Our checker scores the crawlability layer those agents and answer engines share — raw HTML, robots.txt, llms.txt and schema.
See Google’s overview:Build agent-friendly websites(web.dev). Below is how that advice maps to our free checker and GEO work.
| Agent-friendly guidance | LoudCrowd checker & GEO |
|---|---|
| Readable content in raw HTML (not JS-only shells) | Readable text in HTML — H1, title and body copy visible without executing JavaScript |
| Machine-readable entity and page signals | llms.txt, ai.txt, JSON-LD Organization and FAQ schema |
| AI crawlers allowed to fetch pages | robots.txt — GPTBot, Google-Extended, PerplexityBot not blocked |
| Accessibility tree: roles, names and labelled inputs | Not scored automatically — we fix via semantic HTML, <label for>, real <button>/<a> CTAs and stable layouts in web design |
| Stable CTAs and semantic actions (not divs pretending to be buttons) | GEO + conversion-focused web design — consistent contact paths agents and humans can follow |
Technical crawlability is step one. GEO adds extractable FAQs, reviews, list inclusion and entity consistency so AI systems recommend you — not just read you.
Readable HTML beats more blog posts
A page can rank in Google and still be invisible in ChatGPT. Google renders JavaScript; many LLM crawlers do not. If your service pages are built in React, Wix or a heavy page builder without server-side HTML, AI systems may never see your H1, FAQs or schema.
Technical readiness is step one. Step two is GEO — entity clarity, reviews, list inclusion, extractable FAQs and consistent facts across your domain. LoudCrowd is Sheffield-based and works with UK local businesses only.
AI visibility checker — common questions
Direct answers on what this tool does and how it relates to GEO.
- Is this AI visibility checker free?
- Yes. Enter any URL and get a score in about 30 seconds — no signup required. It checks technical factors AI crawlers use, not whether you will rank or get cited.
- What does the AI visibility score mean?
- The score combines ten technical checks into one figure out of 100. A higher score means AI systems are more likely to read, classify and extract content from your raw HTML.
- Does a high score guarantee ChatGPT will recommend my business?
- No. This is a technical readiness check — whether AI can read your page at all. Citations also depend on authority, reviews, entity consistency and content quality. That is what GEO and SEO work addresses.
- How is this different from Google Search Console?
- Google renders JavaScript and uses its own indexing systems. This checker inspects the raw HTML that many AI crawlers actually fetch, plus AI-specific signals like llms.txt and FAQ schema.
- Why might my React or WordPress site score low?
- Many AI crawlers do not execute JavaScript. If your main content only appears after scripts run, LLMs may see an empty or thin page. Static HTML, server rendering or prerendering fixes this.
- We serve UK local businesses — is this relevant?
- Yes. When buyers ask ChatGPT or Google AI Overviews for the best roofer, accountant or solicitor in their town, technical AI readiness plus local entity signals decide whether you appear at all.
- How does this relate to Google’s agent-friendly websites guidance?
- Google’s web.dev article explains how AI agents read raw HTML, the accessibility tree and screenshots. This checker scores the crawlability side — robots.txt, llms.txt, schema and readable HTML. Semantic CTAs, labelled forms and stable layouts are fixed in web design and GEO programmes.
Page updated
Get started
Ready to grow your business?
Get a free marketing review — no hard sell, just an honest conversation about SEO, GEO or web design for your trade.
