How To: Get your web site indexed by Google

One of the most frequently asked questions in the Google Support newsgroup and similar forums is "Why is my site not listed by Google?" or "Why are my pages not in the Google index?" Sometimes the question is asked about other search engines like MSN or Yahoo, too.

Search engine submission

Search engines are not magic. In order to find a site, they need directions. Before it can include a site in its index, a search engine has to know that the site exists. The simplest way to achieve this is to go and tell the search engine, by submitting the details of the site.

However, although submitting your site will get it listed in some search engines, it won't get it into Google. Google is the most popular Internet search engine, with the largest index, but perhaps for that reason, it needs to be a bit fussy about what gets included. Google has decided that if there are no links from other pages that point to a site, the site is of no interest to anyone but its creator, and so it will not be indexed. Therefore, before submitting your site to Google, you need to get some links to it from other sites that are in Google's index.

Getting inbound links

Getting links to your site can be a tough job, because you are basically asking other people to do you a favour. If your site is of interest to a particular group of people, you may know of other sites that are also of interest to this group. Write to the webmaster of each site, and ask if they would place a link to your site. However, if the site doesn't have a convenient place to put the link, they may consider it too much trouble to do so. If your site is a commercial site, the only similar sites you know of may be competitors, in which case they are probably not going to want to link to you.

Many sites that are willing to link to you, will do so on condition that you link back to them. This is a tried and tested way of getting links to a site, and it is called link exchanging or reciprocal linking. However, search engines are getting clever at detecting links that exist only to make sites more noticeable to search engines, and some webmasters are getting a bit fussy about who they link to. A link from a site that has material related to yours is worth more than a link from a completely unrelated site. And a link from an established site is worth more than a link from a new one. This means that the newer your site, and the more desperately you need a link, the harder it is to get one.

An easier way to get some incoming links to your site is to submit it to web directories. As you can guess, web directories are indexed lists of sites, so your site is unlikely to be refused a link unless it is a poor quality site or in some other way unacceptable. Many web directories specialize in certain topics such as health, religious or family matters. You should not submit your site to a specialized directory if its purpose is completely unrelated. The most popular web directories will require payment for an entry, while for others, payment will get you a more prominent entry or speedier inclusion.

Some web directories only require a link back to you to guarantee inclusion. These directories have little value. However, as long as the directory looks respectable and is not a "bad neighborhood" then a link from it is still worth having. Here is a list of web directories you can submit to, to help you get started.

An good way to get useful links to your site is to locate web forums and blogs related to the topic of your site, and post comments with links back to your site. This needs to be done sensibly and tactfully so that you are not accused of spamming. Many people post completely inappropriate comments to blog entries that just spoil the experience for everyone. But if comment posting is done properly, people who read the comments may actually be interested in following the link to your site, so that you don't just get a valuable link, but actual visitors interested in what your site has to tell or sell them.

An even better method – if you're reasonably good at writing – is to write an article or two on topics related to your site, and submit the articles to relevant article directories. Other people may pick up your article to use in their own website, so not only do you get links and visitors from the article directory itself, you may also get new links from other sites. These links will count for more than links from links pages do, so they will do more to boost your site's position in search engine results. 

Using pay-per-click ads

A commonly used way to get a site noticed by search engine users is to use a pay-per-click advertising program such as Google AdWords. This won't necessarily get your site into the index, but it will get it on to the results page, though in the "Sponsored Links" section not the main search results. This can be an expensive way to get traffic, and may be impractical if your site is non-commercial in nature. Research shows that search engine users tend to trust the natural search results over the sponsored ones, so this isn't even the best method.

Because not every site can get listed at the top of the search results for every keyword, pay-per-click is a valid approach for any commercial site, even one that is already listed in Google. But this is not an option we will discuss further in this How-To.

Paying for submission

Generating sufficient links to your website to build visitor numbers and get it noticed by search engines is a time-consuming activity. To save time, you may wish to pay for a program that can automate (or semi-automate) the task for you. Axandra's ARELIS program helps you locate relevant sites that are willing to exchange links, and helps you complete a form or letter to request reciprocal linking.

Axandra's Internet Business Promoter package, which we use here at Tech-Pro Limited, is another useful tool that will save time for anyone who is developing websites. Apart from link building, it can help with search engine and directory submissions, and offers guidance to help optimize your web pages so that they achieve better search engine rankings.

You can find services which, for a fee, offer to submit your site to search engines. Whilst we can't comment on any of them, not having tried them, we can only say that many people who have used such services have been disappointed with the results. Ask yourself: how many search engines do I know of? And if I haven't heard of a search engine, is it likely that many other Web users will know of it? Many of these services are a waste of money, and some could even bring your site into disrepute by listing it on sites you would not want it listed on.

Some services claim that for a fee, they can guarantee inclusion in Google. Google insiders say that it is in no way possible to get a faster inclusion into the index by paying for it. The only way to get your site indexed by Google is the method described here: get some incoming links from related sites that are themselves in the index, submit your site, and then wait and be patient.

The sandbox

You might find that, whatever you do, your site does not get listed in Google. If it's a new site, using a newly-registered domain, then one reason for that may be the "Google sandbox" which causes new sites to be kept out of the index for  period of time.

The success of Google's AdSense program, which allows sites like this one to display (and receive revenue from) targeted ads, has encouraged many Internet entrepreneurs to create websites purely with the aim of milking AdSense revenue. Because this is not, on the whole, beneficial – these sites offer nothing new, and sometimes blatantly copy content from other sites – Google uses tools like the sandbox to discourage them. Google doesn't publish details of the sandbox, and it applies to a varying extent depending on the type of site. New sites containing content that is popular with AdSense publishers (because it attracts high-paying ads) are likely to be sandboxed for longer than others. It's probably also a good idea not to use AdSense on a site until after it has been included in the index by Google.

If a site does not get included in Google's index even after several months of existence with good quality links in to it, then it is possible that the site has been banned from the index due to the activities of the domain's previous owner. (This happened to us with our site The PC Guru.) Sites get banned from search engines for employing techniques to try to trick them into giving them a better ranking than they would otherwise earn. There is no way to be sure if that has been the case, but if you don't know whether your domain has been used before, you can find out by searching for it at the Internet Archive.

If you suspect that your domain has been banned by Google, you will have to file a reinclusion request. The procedure for doing that has been documented by Google employee Matt Cutts in his blog: see Matt Cutts: Gadgets, Google, and SEO - Filing a reinclusion request.