Category: Basics
Crawl budget
How many pages a bot crawls in a given time.
Crawl budget is the amount of resources a search engine spends crawling your site. On large sites it matters: if the bot wastes the budget on junk URLs, important pages get crawled less often. A clean sitemap.xml and blocking useless URLs both help.