Google indexes my AdSense URLs? Good!
There's a big fuss going on in the SEO community right now because pages being crawled by the mediabot (the AdSense crawler) are finding their way into the Google search results index. See AdSense mediapartners bot adding to the Google search index and Google AdSense Bot Updating Page Caches for all the details.
I'm not an SEO bigwig, so take what I say with a grain of salt, but I have no problems with what Google is doing here. There are really two issues involved:
- Google is indexing AdSense pages
- Google is using information from the AdSense crawler for that indexing
Let's discuss each issue separately.
Google indexes AdSense pages
Let's revisit Google's corporate mission:
Google's mission is to organize the world's information and make it universally accessible and useful.
Knowing what their mission is — and they've never been shy about stating it — it only makes sense to me that they would scour and use any and all page links they get. I've always assumed that any URL I provide to Google via any means is fair game for them to crawl for their search index. This includes:
- Pages linked to from pages already in the index
- Pages displaying AdSense ads
- AdWords landing pages
- Pages in Blogger-managed blogs
- Pages in Google Pages
- Pages input via Google Sitemaps
- Links in data from Google Base
There are probably other ways to get your page indexed by Google that I've missed. But as you can see, there are many ways to provide Google with a URL. AdSense is just one more input into this system.
And this is good. If I put AdSense ads on a page, the implicit assumption is that I want traffic, I want people to find that page. If they can't find the page, they won't be clicking on the ads. And the page is public — the AdSense terms and conditions state that you can only place AdSense ads on public pages — so it's fair game to be found.
Google uses AdSense crawler data
This is actually what the fuss is about in my mind. The fact that the information obtained by the AdSense crawler — not just the URL of the page — is finding its way into the search index.
The computer scientist part of me thinks it only makes sense for this to happen. After all, one of Google's computers has already spidered the page, so instead of just submitting the URL to the search engine indexing algorithms, it can also submit its view of the page at the same time. Sounds more efficient to me.
Of course, Google has always stated that there's a “Chinese wall” between the search and ad sides of the company so that advertisements can't affect the organic search results. I guess everyone's assumed that this meant that the data returned by the different crawlers wouldn't be comingled. But I don't know why that would be: they should, after all, be returning the same data.
And here's where things aren't so clear cut anymore. The fact is, there are sites out there that return different things for different crawlers. They might return stripped down (mostly text) versions of pages for the Googlebot (the main crawler) but not for other crawlers. Or they might return very different pages for the different crawlers.
But Google doesn't want sites to send it different pages based on what crawler is visiting. They want to see the same page that humans see in their browsers. So from their viewpoint, it shouldn't matter which crawler is doing the crawling — they should all be returning the same data. In fact, using different crawlers to see what's being returned is a good way to flag a site for discrepencies. Google sometimes sends out anonymous crawlers for similar reasons.
Who says rankings are affected?
All we're talking about is the way data gets into the Google search index. There's no proof that pages are ranking any differently because they came in from the advertising side. If there are differences, they're likely because different data is being returned to the different crawlers. I'm sure Matt Cutts will chime in shortly with assurances that the ranking algorithms are unchanged and unaffected by this.
If anything, this whole process demonstrates the continued importance of Google AdSense Tip #1. Make sure your page is right before you give its URL to any part of the Google machine.
Back to work, folks!
Eric Giguere is the contextual advertising expert who wrote Make Easy Money with Google and Uncommon AdSense. You can read this blog by mail if it's more convenient for you, just send a blank email to memwg-blog@aweber.com to subscribe.
 |
Enjoyed this post? Get free updates by mail or by RSS! |
Tags
Comments
One Response to “Google indexes my AdSense URLs? Good!”
Thank for making this valuable information available to the public.