email from Google Product Search

For general questions and discussions specific to the AbleCommerce 7.0 Asp.Net product.
Post Reply
azar
Ensign (ENS)
Ensign (ENS)
Posts: 15
Joined: Sat Nov 27, 2010 11:43 am

email from Google Product Search

Post by azar » Mon Apr 11, 2011 12:40 pm

Hi all,

Just got an email from Google Product search:
Hello,

Thank you for participating in Google Product Search. It has come to our attention that a robots.txt file is preventing us from crawling some or all of the images on your site. In order for us to access and display the images you provide in your product listings, we'd like you to modify your robots.txt file to allow user-agent 'googlebot' to crawl your site. Failure for Google to access your images may affect the visibility of your items on Google Product Search and Product Ad results
Anyone knows what this is about? Could this be because of "Disallow: /Assets/" in the robots.txt, which is the default configuration in Able? But we had AC running for about 4 months now and I checked couple of products in google product search, all seem to have images. Should I remove "Disallow: /Assets/"? What are the risks of doing this?

JaminLeather
Lieutenant, Jr. Grade (LT JG)
Lieutenant, Jr. Grade (LT JG)
Posts: 27
Joined: Fri Feb 05, 2010 9:37 am
Location: Myrtle Beach, South Carolina
Contact:

Re: email from Google Product Search

Post by JaminLeather » Mon Apr 11, 2011 2:29 pm

The robots.txt file "suggests" relevant content to search engine bots. If you have content on your site that you would rather not have crawled, list it in this file. The search engines may or may not take your suggestions. In the case of your images on Google Product Search, Google is probably taking the link directly from your product detail page.

Given that you are using the Product Search, Google is simply advising you to ensure your pictures will continue to be crawled in the future by removing the assets directory from the disallow statement.

Just remember, if you chose to make the assets directory visible to search engines, that content may start appearing on search engines. This may include old, outdated images you still have floating around.
Jamin' Leather Catalog Outlet
http://www.jaminleather.com

azar
Ensign (ENS)
Ensign (ENS)
Posts: 15
Joined: Sat Nov 27, 2010 11:43 am

Re: email from Google Product Search

Post by azar » Tue Apr 12, 2011 2:21 pm

Thanks George.

I will keep "disallow: /Assets/". I want to see what happens. If this really causes our product images to disappear from google product search, then it is an important issue that will require attention from AC dev team to include a fix in the next release and inform all customer about this problem.

snap624
Lieutenant (LT)
Lieutenant (LT)
Posts: 58
Joined: Mon Jan 19, 2009 2:53 pm

Re: email from Google Product Search

Post by snap624 » Sun Apr 17, 2011 10:33 am

I got a similar message from google on April 11:
Thank you for participating in Google Product Search. It has come to our attention that a robots.txt file is preventing us from crawling some or all of the images on your site. In order for us to access and display the images you provide in your product listings, we'd like you to modify your robots.txt file to allow user-agent 'googlebot' to crawl your site.
Failure for Google to access your images may affect the visibility of your items on Google Product Search and Product Ad results.

To ensure the 'googlebot' is not being blocked, please add the following two lines of text to the end of your robots.txt file:

User-agent: googlebot
Disallow:
We had certain products that would appear very high in google's search results. Now, however, they are way down on the list. They're there, but not displaying as they had been (the images still display). Anyone else seeing changes or having issues with google search? I am not sure if I should be concerned about following google's advice on adding the 2 lines they suggest. It seems it would grant their bot access to everything.

Post Reply