Site Map

A site map should use Absolute Path (full path) links to each page in the Web site, and the code should be as uncluttered as possible. The page should be named Site_Map.htm or site_map.html and there must be a full path link from the main page of the Web site to the site map page so that the search engines can find it. The site map for this site is here:

The site map page for this Web site is here: site map. If a site map is larger than 100 or 200 links, it is recommended that you start a new sitemap, linked to the first. However, the actual capacities or limitations of major search engine spiders are probably much larger,

Site Map Files for Search Engines

Recently a standard (the ROR standard) was introduced for XML site map files that can be submitted to different search engines. These allow you to append various information such as the frequency with which the pages might change, You can now tell search engine spiders where the XML sitemap is with this directive in your Robots.txt file:

Sitemap: http://www.mysite.com/sitemap.xml

You can and should submit site maps to Google Webmaster. Site map submissions should be consistent. Always use one form of the URL for pages. That is, list every page as

http://www.mysite.com/sitemap.xml or http://mysite.com/sitemap.xml but never both.

www.sitemaps.org/  explains the standard.

There are various free or partly free tools that supposedly help you create an XML standard site map automatically, such as this one: http://www.xml-sitemaps.com/. Many of the ones I tried are of limited capacity or do not work at all.

A Google explanation of XML site maps is here:

http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=40318

Google explains that Site Maps are especially important for special materials such as flash files that might not otherwise be intexed properly. Fair enough.

Google has a tool for generating sitemaps, but it is a python script you need to have installed on your Web server. It's OK if you have access to the server and are up for that. Read about the tool here:

http://www.google.com/support/webmasters/bin/answer.py?answer=34634

A word of caution: Be careful what you tell Google and other search engines about your pages in an XML site map and make sure it is correct. It is better to give no information than bad information sometimes. For example, marking pages "Dynamic" (sounds good) is a bad idea, as it may degrade the listings of those pages. The search engine algorithm "decides" that if the page content is going to change all the time, it is not worthwhile giving that page a high ranking for a particular key word. That's logical, but it can mess you up.

Text Site Map Files

Google and some other search engines can read text site map files. You can prepare these by hand in various ingenious ways and without too much trouble. They are just a list of URLS at the site. This page tells you how to create a text file site map:

http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=71450

Google wants you to submit the text files to their site map generate, which can then turn them into XML sitemaps, but you do not have to do that unless you need to give the search engine the special information needed in the map.

Usefulness of XML and Text Site Maps

If you have a site that is not being indexed properly by Google and/or other search engines then you should certainly try submitting a site map. "Indexed properly" means that a new page is indexed after a day or two for sites with Google Pagerank 6 on the main page, or it can mean that a page is indexed after a month for a small new site that has pagerank 3 or less. If a page doesn't show up for 6 months there might be something wrong with the page or the links to that page, or there just might be some glitch in the search engine that prevents it from being listed.  Search engines do not guarantee to list pages submitted by site map, but in practice, if the pages are not junk, they do seem to get listed within a few hours of submitting the site map. For small, new Web sites, this can be really important.

However, before resorting to site maps, you should try propagating the new links to Web logs and forums and directories. Web logs have RSS feeds that Google and other search engines can check to pick up new links quickly.

Obviously, if you have thousands or tens of thousands of pages at your Web site, it is impossible to put them all in a single XML site map, and you will need to submit several maps. It is usually unnecessary to submit such maps, since Google and other search engines will get around to indexing all your pages eventually especially if you have a large site. Sites with a high pagerank tend to be crawled very frequently

Note - Definitions of Search Engine Optimization terms are based on inferences from common usage and definitions given by other sources. Conclusions about search engine behavior are based on understanding of the behavior of the most popular search engines. Both are subject to error or may change. Search engine company management may define or use a term or set or change any policy in any way they see fit, and may make these definitions and specifications public or not. These decisions and definitions are beyond our control.  

Notice: Copyright

SEO

SEO Basics

MidEastWeb: Middle East