Indexation

Indexation is the process by which search engines add web pages to their searchable database after crawling them. A page must be indexed to appear in search results. Indexation issues — where important pages are not indexed or wrong pages are indexed — are among the most common technical SEO problems.

For AI optimization, indexation extends to AI crawler indexing. Content must be accessible and indexable by AI crawlers to be included in real-time retrieval results. Pages blocked from AI crawlers through robots.txt or behind login walls will not be retrieved for AI-generated responses.

Common indexation issues include robots.txt blocking, noindex directives, crawl budget exhaustion on large sites, thin content filtering, and canonicalization problems. Monitoring indexation through Google Search Console's Coverage report is essential for maintaining search and AI visibility.