[ad_1]
Gary Illyes from Google posted on LinkedIn that his mission this 12 months is to “work out the way to crawl even much less, and have fewer bytes on wire.” He added that Googlebot ought to “be extra clever about caching and inside cache sharing amongst person brokers, and we must always have fewer bytes on wire.”
He added, “Lowering crawling with out sacrificing crawl-quality would profit everybody.”
Right now, Gary added, that Google is crawling as a lot because it did earlier than – regardless of some people pondering Google is crawling much less. He mentioned “Within the grand scheme of issues that is simply not the case; we’re crawling roughly as a lot as earlier than.”
What Google is healthier at that earlier than is scheduling. “Nonetheless scheduling bought extra clever and we’re focusing extra on URLs that extra prone to deserve crawling,” he defined.
It appears Microsoft Bing, particularly Fabrice Canel from Microsoft and Gary Illyes from Google have the identical objectives. Microsoft is tackling it by encouraging website house owners to make use of IndexNow. Google mentioned in November 2021 that Google would possibly think about adopting IndexNow however that got here and went…
John Mueller from Google commented on the put up suggesting, “We may simply crawl RSS feeds and create some form of Reader.” A joke about Google Reader…
Anyway – we are going to see what Google finally ends up doing right here. Right here is his full put up:
My mission this 12 months is to determine the way to crawl even much less, and have fewer bytes on wire.
A couple of days in the past there was a put up on a Reddit neighborhood about how, within the OC’s notion, Google is crawling lower than earlier years. Within the grand scheme of issues that is simply not the case; we’re crawling roughly as a lot as earlier than, nonetheless scheduling bought extra clever and we’re focusing extra on URLs that extra prone to deserve crawling.
Nonetheless, we must always, the truth is, crawl much less. We must always, for instance, be extra clever about caching and inside cache sharing amongst person brokers, and we must always have fewer bytes on wire.
In case you’ve seen an fascinating IETF (or different requirements physique) web draft that would assist with this effort, or an precise normal I’d’ve missed, ship it my means. Lowering crawling with out sacrificing crawl-quality would profit everybody.
Discussion board dialogue at LinkedIn.
[ad_2]
Supply hyperlink
