Using the SiteMap feature to get into search engines

This forum is where we'll mirror posts that are of value to the community so they may be more easily found.
Post Reply
User avatar
AbleMods
Master Yoda
Master Yoda
Posts: 5170
Joined: Wed Sep 26, 2007 5:47 am
Location: Fort Myers, Florida USA

Using the SiteMap feature to get into search engines

Post by AbleMods » Fri Feb 22, 2008 9:54 pm

Introduction
I am by no means an authority on SEO optimization. I just learned .Net a year ago and Solunar.com is my first online storefront experience.
What I do know is my experiences in taking advantage of the SiteMap feature included in AC7.
The purpose of this document is to detail my experiences, the steps taken and the bumps in the road along the way.

What is SiteMap?
The three largest search engines came together several years ago and agreed on a standard for bulk site URL submission. You can easily submit a single domain name to any search engine, but getting all 12,000 products in your store submitted as SEO-friendly URLs is a different story.

This standard resulted in the definition of what is known today as the "Sitemap.xml" file format. The format specifies the fields necessary to submit any number of URLS to the search engine for indexing.

Why should I care?
Well, you can't really grow a business if nobody knows it's there. Getting into search engines is the single most important step with an online store. Plus you'd like your business growth to be more than one more order this week than last week.
Using a sitemap file also reduces webmaster effort to maintain search engine presence. And it reduces unnecessary spider/crawler traffic on your site.
Whatever the reason, the benefit of getting into the search engines quickly and accurately is significant. Sales pick up quicker, loss-to-profit timetables accelerate and your customer base grows sooner and at a faster pace.

Where to Start
First you need your products set up in your store. It's not going to do alot of good to tell everyone you're open for business if your shelves are empty. Get your images, descriptions and prices all the way you want before proceeding any further.
Once you feel you're ready, login into your site Admin and click the Website, XML Site Map option.

Don't change anything. Let's not get fancy at all right now. Especially don't change the sitemap.xml file name. That's part of the standard and if you mess with it, you might not be successful in getting your sitemap file read by the search engine provider.
At the bottom of the page, set the dropdown menu to "Create Sitemap" and click the blue arrow. Don't click save settings, that's not going to do the task - that only saves the screen changes I just got done telling you not to make :)

When it's done, you'll get a confirmation at the top. The sitemap.xml file is created in the root of your site. Check it out if you want, but don't change it.

Autodiscovery
In the Spring of 2007, the 3 big players (and a few smaller ones) in the search engine provider industry enhanced the sitemap standard by implementing a uniform way to upload your file. it's really incredibly simple: Just send them a URL with a parameter for the URL to your sitemap file.

For example, to tell Ask.com to come get your sitemap file and process it, just enter this URL in your browser:

Code: Select all

http://submissions.ask.com/ping?sitemap=http://www.example.com/sitemap.xml
Or Google.com

Code: Select all

www.google.com/webmasters/tools/ping?sitemap=http://www.example.com/sitemap.xml
Other major search engines should support this same format but check if you aren't sure it worked. You should receive a confirmation page from the search provider indicating the request was received.

How Often?
Well, that's the million dollar question. Nobody really knows for sure. I do mine about once a month, or whenever the mood hits me. I doubt doing it every day is going to help your cause, so you should probably limit it to when you have truly meaniful changes to be uploaded.

Spider autodiscovery
You can also tell any spiders that crawl your site to read your sitemap.xml file. Just add this line to your robots.txt file in the root of your store:

Code: Select all

Sitemap: http://www.example.com/sitemap.xml

Advanced stuff with Google
Google.com has some amazing site tools for webmasters. Just go to Google.com and search for "Google Webmaster Tools". Sign up for it. It's free, it's awesome and you really should check it out. It's worth it, trust me.
One important part of the webmaster tools is "Google Feed" which lists all of your store products into the Google Product Search.
And yes, this just happens to also be where you'll upload the file produced by the AC7 Marketing, Feeds, Google Base Feed menu option. And it's still free because it's beta. But don't let the word "beta" scare you away, I am getting ALOT of orders from Google Products Search.

More Information
There are quite a few great articles on sitemap implementation. Try http://www.sitemap.org or search any search engine provider for "sitemap information".

Conclusion
Making yourself known to the internet is the most difficult and most rewarding step in running an online store. AC7 has made this much easier for the site admin by supporting the SiteMap standard. Combined with other forms of online presence development, your site awareness is sure to grow as will your business.
Joe Payne
AbleCommerce Custom Programming and Modules http://www.AbleMods.com/
AbleCommerce Hosting http://www.AbleModsHosting.com/
Precise Fishing and Hunting Time Tables http://www.Solunar.com

rdevarona
Lieutenant, Jr. Grade (LT JG)
Lieutenant, Jr. Grade (LT JG)
Posts: 41
Joined: Tue Jan 15, 2008 7:24 am

Post by rdevarona » Fri Feb 29, 2008 8:39 am

Here's my contribution...

I've added the following lines to \Admin\Website\CommonXMLSiteMap.aspx to make it easier to ping these search engines when I make a new SiteMap file.

Code: Select all

(<a href="http://search.yahooapis.com/SiteExplorerService/V1/ping?sitemap=http://www.YOURDOMAIN.com/sitemap.xml">Yahoo</a>
&nbsp;|&nbsp;
<a href="http://www.google.com/webmasters/sitemaps/ping?sitemap=http://www.YOURDOMAIN.com/sitemap.xml">Google</a>
&nbsp;|&nbsp;
<a href="http://submissions.ask.com/ping?sitemap=http%3A//www.YOURDOMAIN.com/sitemap.xml">Ask</a>
&nbsp;|&nbsp;
<a href="http://api.moreover.com/ping?u=http://www.YOURDOMAIN.com/sitemap.xml">Moreover</a>)
This lets you ping ASK, Google, Yahoo and Moreover. I've read that MSN pulls requests from Moreover. You'll have to replace YOURDOMAIN with your actual domain name. I added this directly after

Code: Select all

let search engines know</a>
Hope it helps.

rdv

User avatar
AbleMods
Master Yoda
Master Yoda
Posts: 5170
Joined: Wed Sep 26, 2007 5:47 am
Location: Fort Myers, Florida USA

Post by AbleMods » Fri Feb 29, 2008 12:47 pm

Very nicely done!
Joe Payne
AbleCommerce Custom Programming and Modules http://www.AbleMods.com/
AbleCommerce Hosting http://www.AbleModsHosting.com/
Precise Fishing and Hunting Time Tables http://www.Solunar.com

rdevarona
Lieutenant, Jr. Grade (LT JG)
Lieutenant, Jr. Grade (LT JG)
Posts: 41
Joined: Tue Jan 15, 2008 7:24 am

Post by rdevarona » Fri Feb 29, 2008 12:52 pm

Turns out that you can do the same for Microsoft Live Search. Use the following url for the ping:

Code: Select all

http://webmaster.live.com/ping.aspx?siteMap=[Your sitemap web address]
rdv

User avatar
Willisski
Lieutenant Commander (LCDR)
Lieutenant Commander (LCDR)
Posts: 82
Joined: Thu Oct 27, 2005 1:46 am
Location: http://www.Thejibshop.com
Contact:

Is away to automate somthing like this?

Post by Willisski » Tue Mar 11, 2008 12:59 am

While SEO's generally hate things that are automated. Could you setup something like this to run every 167 hours.
1 week is 168
It seems that one less thing to do would be great
any suggestions?
Thanks

User avatar
jmestep
AbleCommerce Angel
Posts: 8164
Joined: Sun Feb 29, 2004 8:04 pm
Location: Dayton, OH
Contact:

Post by jmestep » Tue Mar 11, 2008 5:09 am

I don't think the automation would be a problem, at least with Google. They give you instructions about how to ping them if you have a new sitemap.
Judy Estep
Web Developer
jestep@web2market.com
http://www.web2market.com
708-653-3100 x209
New search report plugin for business intelligence:
http://www.web2market.com/Search-Report ... -P154.aspx

User avatar
AbleMods
Master Yoda
Master Yoda
Posts: 5170
Joined: Wed Sep 26, 2007 5:47 am
Location: Fort Myers, Florida USA

Post by AbleMods » Tue Mar 11, 2008 4:23 pm

I'm trying to find an effective automation mechanism for launching web pages on a schedule.

Seems like there's 200 different ways to do it on the net when I search for it.

Until you have a solution to schedule webs, just add the pings to the end of your sitemap module so every engine is automatically pinged when you generate a new file. Then just do an appointment reminder in Outlook etc.
Joe Payne
AbleCommerce Custom Programming and Modules http://www.AbleMods.com/
AbleCommerce Hosting http://www.AbleModsHosting.com/
Precise Fishing and Hunting Time Tables http://www.Solunar.com

User avatar
Willisski
Lieutenant Commander (LCDR)
Lieutenant Commander (LCDR)
Posts: 82
Joined: Thu Oct 27, 2005 1:46 am
Location: http://www.Thejibshop.com
Contact:

Sitemap additonal pages

Post by Willisski » Fri Mar 14, 2008 12:17 pm

We have several pages that are not products or categories. such as equipement sizing guides in basic html. Is there a way to include these pages to the sitemap automatically or are the pages already added ?

User avatar
AbleMods
Master Yoda
Master Yoda
Posts: 5170
Joined: Wed Sep 26, 2007 5:47 am
Location: Fort Myers, Florida USA

Post by AbleMods » Fri Mar 14, 2008 4:51 pm

No, not in the current design. Sitemap.xml only includes pages derived from the catalog and the base AC7 pages like ContactUs, Default, etc.

Spiders and bots will handle those for you. Sitemap doesn't "limit" what spiders will crawl, it aids them in knowing how to crawl them efficiently. Your other pages will still get crawled.

I have the same situation with several non-catalog pages of product information, testimonials etc.
Joe Payne
AbleCommerce Custom Programming and Modules http://www.AbleMods.com/
AbleCommerce Hosting http://www.AbleModsHosting.com/
Precise Fishing and Hunting Time Tables http://www.Solunar.com

DonAshby
Commander (CMDR)
Commander (CMDR)
Posts: 172
Joined: Mon Oct 15, 2007 2:53 pm
Location: Palo Alto, CA
Contact:

Post by DonAshby » Fri Mar 14, 2008 5:27 pm

Hi,

We export the AC Site Map to our desktop, then add in any custom pages we make to the list, and then upload it all to Google.

Google will crawl your entire site but this is a proactive way to tell them about any new pages without waiting. However, we still see that indexing can take a while... Not sure how long.

Don :)

User avatar
Shopping Cart Admin
AbleCommerce Admin
AbleCommerce Admin
Posts: 3055
Joined: Mon Dec 01, 2003 8:41 pm
Location: Vancouver, WA
Contact:

Re: Using the SiteMap feature to get into search engines

Post by Shopping Cart Admin » Wed Apr 16, 2008 1:22 pm

Thanks for your support

Shopping Cart Guru
AbleCommerce.com
Follow us on Facebook

iffautomation
Lieutenant, Jr. Grade (LT JG)
Lieutenant, Jr. Grade (LT JG)
Posts: 27
Joined: Sat Jun 06, 2009 7:44 am
Location: Indiana
Contact:

Re: Using the SiteMap feature to get into search engines

Post by iffautomation » Tue Aug 31, 2010 3:48 pm

Is there an update yahoo webaddress to ping our sitemap to?
The Google, Ask and Microsoft Live links work great!

(<a href="http://search.yahooapis.com/SiteExplore ... ">Yahoo</a>

Thanks,
Tom K.

User avatar
AbleMods
Master Yoda
Master Yoda
Posts: 5170
Joined: Wed Sep 26, 2007 5:47 am
Location: Fort Myers, Florida USA

Re: Using the SiteMap feature to get into search engines

Post by AbleMods » Tue Sep 07, 2010 7:30 am

This works for me with Yahoo. If successful, you will see an XML response "Update notification has successfully submitted."

Be sure to change the domain name....

http://search.yahooapis.com/SiteExplore ... itemap.xml
Joe Payne
AbleCommerce Custom Programming and Modules http://www.AbleMods.com/
AbleCommerce Hosting http://www.AbleModsHosting.com/
Precise Fishing and Hunting Time Tables http://www.Solunar.com

iffautomation
Lieutenant, Jr. Grade (LT JG)
Lieutenant, Jr. Grade (LT JG)
Posts: 27
Joined: Sat Jun 06, 2009 7:44 am
Location: Indiana
Contact:

Re: Using the SiteMap feature to get into search engines

Post by iffautomation » Thu Jan 13, 2011 4:51 pm

Thanks that works much better than the link I had for yahoo.

Post Reply