Crawl budget

Crawl budget only becomes a real constraint on large sites — tens of thousands of URLs or more — where Googlebot won't fetch everything every day. Faceted navigation, session IDs, and infinite calendar pages can generate millions of low-value URLs that soak up crawls your important pages needed. Trim the waste with robots.txt blocks, canonical tags, and clean internal linking so the budget flows to pages you actually want indexed.