Most people who have websites don’t understand what search engine submissions are or why they are important. Of course, most people have no idea how to find a search engine, let alone find any search engines. So this article is meant to be a general overview of what they look like and why they are important.
In a nutshell, Google, Yahoo!, MSN, Altavista, Ask, and all the rest of the search engines that “crawl” the web looking for sites to return in search results. When you submit your site to them, they will look at your site and, if it’s ready, index it. If it isn’t ready, it will be crawled and then if it is still “too small” it will be discarded as the site cannot be included in the search index.
Many people think that when their site is listed in a directory, it will be indexed and listed in the search results right away. That’s not necessarily true. There’s some instances that a site may be crawled and indexed but the rank will be ” wireless effect” for search engines. wireless effect is when multiple sites are crawled and indexed at the same time proving to be difficult for the search engine or robot to distinguish one from the other.
In other words, some robots may index a site but, for example, not link to it. It might as well not exist.
Some robots are given a high rank or index a site. These are often web crawlers or robots written to index and link to data on a regular basis.
Now despite what you have said about robots and spiders, I’m sure there are not more than a few robot specialists who would assure you that their robots are as adept as possible at reading your website. But I’m afraid you are in for a big disappointment.
robots.txt and the robots meta tag
Meta tags have no SEO value. They were meant almost entirely for the use of search engine robotSto tell which pages of a site should be indexed and which should be ignored. This tag, however, is often abused by website owners in order to make their website design more “search engine friendly”. For example, you could create a.htaccess file that would allow you to tell certain robots which folders or files should not be indexed.
Using the meta tag, you effectively choose which pages or files you want to be indexed by the search engine robots.
My advice is: Never, everuse the meta tag to tell robots which pages or files you don’t want them to index. If a page or file is not indexed by the robots then it will not be displayed in the search results. Using the meta tag correctly is safe for SEO purposes but, from an engine’s perspective, is not required. On the other hand, you really don’t want to confuse the search engine robots so they know what to index and which pages or files to keep in the index.
Using the meta tag:
Website Content
Headers
Title of the page
Description
Keywords used in the page but not in the URL
Links to the page
enlarged image sizes, and site pages of fewer than 150 – 250 words
Use of heading tags and phrasesthat accurately describe the topic of the page
Appropriate linking
Many search engines look for keywords in the text of links that lead to websites so if you lead your visitor to a page and they find the actual content to be irrelevant, your site could be penalised inflated in the rankings.