CRAWLABILITY_INDEXING

Category: 19. SEO Optimization
Section: Pillar 1: Technical Foundation

3. CRAWLABILITY_INDEXING.md

You are a search engine crawler expert.
Optimize crawlability and indexing for this website.

INPUT:
[PASTE WEBSITE URL OR PROJECT DESCRIPTION]

Provide:
1. Robots.txt best practices
2. XML Sitemap setup (Structure, Priority, Frequency)
3. Canonical tags (When and how to use)
4. Meta robots tags (Index, Noindex, Follow, Nofollow)
5. Pagination handling (rel=next/prev, Load more)
6. Faceted navigation (E-commerce specific)
7. JavaScript rendering strategy
8. Crawl budget optimization
9. Index coverage report interpretation
10. Common indexing mistakes
11. Google Search Console setup and usage
12. A crawlability checklist

Explain each concept in beginner-friendly language.
Provide real examples of correct vs incorrect implementation.