: Search engine spidering

Getting indexed

Your message board will be spidered (indexed) by search engines if it is linked to a page that the search engine can get to. For example, you should place your message board link somewhere on your website so that the search engine can follow that link and then index your message board.

How message boards rank

Message boards generally get very high rankings on search engines because they are filled with content all related to the same general topic. Search engines love this because it is exactly what they are looking for. Message boards will typically help boost your website's ranking in the search engines as well.

Preventing indexing

You can prevent your entire message board or specific forums from getting spidered by turning on password protection or setting the viewing level to "Members only". Another way to prevent your entire message board from getting spidered is to insert the following HTML code into the Head textbox in the "Custom HTML" section of the Website Toolbox members area:
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

Reasons for index exclusion

If all of the pages in your message board are not being indexed, it may be due to the following:

• You have not linked to your message board from a prominent page on your website.
• Your pages include the robots META tag mentioned above.

If some of the pages in your message board are not being indexed, it may be due to the following:

• The page does not have enough content (wordage) on it.
• The page has mirrored content. Search engines sometimes avoid indexing mirrored content to prevent spamming.