Search engine spidering

Getting indexed

Your forum will be spidered (indexed) by search engines if it is linked to a page that the search engine can get to. For example, you should place your forum link somewhere on your website so that the search engine can follow that link and then index your forum.

How forums rank

Forums generally get very high rankings on search engines because they are filled with content all related to the same general topic. Search engines love this because it is exactly what they are looking for. Forums will typically help boost your website's ranking in the search engines as well.

Additionally, Website Toolbox forums have been developed with search engine optimization concepts in mind. For example, topic URLs contain the keywords used in the topic title, increasing the changes that a search engine will find those keywords relevant to that page.

Boosting your website ranking

Be sure to customize your forum domain name to match your website. Doing so will allow your main website to receive full credit for all the valuable content on your forum. Search engines will not be able to tell that your forum is hosted on Website Toolbox.

Preventing indexing

You can prevent your entire forum or specific forums from getting spidered by turning on password protection or by setting the Can view forum or Can view forum permissions for the Unregistered / Not Logged In user group. Another way to prevent your entire forum from getting spidered is to insert the following HTML code into the Head textbox in the "Custom HTML" section of your Website Toolbox account:
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

Reasons for index exclusion

If all of the pages in your forum are not being indexed, it may be due to the following:

• You have not linked to your forum from a prominent page on your website.
• Your pages include the robots META tag mentioned above.

If some of the pages in your forum are not being indexed, it may be due to the following:

• The page does not have enough content (wordage) on it.
• The page has mirrored content. Search engines sometimes avoid indexing mirrored content to prevent spamming.

If you still need assistance, please contact customer support.