HOW SEARCH ENGINES WORK

A Practical Guide to Search

Ever wonder how search engines like Google decide which websites to show on the first page of results and which sites to exile to page 6?

In this post, I’m going to share with you how search engines decide which pages get the best position within search results so you can improve your own online visibility.

Links

Search engines use sophisticated computer algorithms and machine learning techniques to categorize and rank web pages on the Internet. These processes are continuously solving problems and making adjustments based on information gathered by search bots.

If you think of the Internet as a web—much like a spider web—that connects millions of web pages through links, then you can see how almost every page on the Internet is connected.

Your website links most of its pages together through a navigation bar. I’m sure there are other links throughout your website, perhaps in the content, which connect to other pages and other websites. Some websites link to pages on your site as well.

Even a Google search results page is full of links to web pages that match (hopefully) your search query.

Crawlability & Indexing

Just like a web, search engines use spiders to crawl links, up and down, back and forth, to gather information, sending it to the servers to be indexed. The spiders (also called bots or crawlers) not only gather content, but they collect activity data on these pages as well. Words are identified, page titles are logged, visitor clicks are counted and so on…

It’s important to realize that these spiders would not be capable of crawling and gathering information without the links from one web page to another. The links are what allow them to move around and index the content and activity on each page in sort of a virtual library. It’s a massive undertaking.

Content

If that wasn’t enough, the content and activity on indexed pages are subject to the calculations and problem-solving operations of search algorithms. So information isn’t just stored, it’s used to enhance the consumer experience by allowing search engines to categorize web pages appropriately, which results in your website getting ranked in search results.

The objective of the search engines with all this is to render the most relevant pages possible for each consumer search query, in order of most relevant, often performing this highly sophisticated task in a matter of milliseconds.

For example, words on a web page like San Diego, lawsuit, child custody and judgment can help render that page in search results when a consumer searches for a San Diego family law attorney.

Terms on a web page such as spring bathing suits, fashion ideas, styles, ladies, petite and shoesmight render a blog about women’s clothing.

Therefore, much of the information on your pages helps search engines know how to categorize your website so consumers find you.

OK, so if there are hundreds of pages that fall under the same category, how do search engines know which ones to show on the first page, or even in that coveted top spot?

Ahh, this is the question search marketers have been trying to solve since the beginning; how to get at the top of Google. In fact, if you’ve got a website, I’m sure you’ve received phone calls and emails from people suggesting they have the answer.

Don’t fall for the hype. Search algorithms are proprietary so nobody has the answer, not even me. We know what we know through research, testing and experience, but the precise search algorithms are closely held secrets.

Search Signals

We’ve been able to determine through our research that search engines use a series of signals, picked up from the crawlers, to determine page trust and rank. Some of these signals can be for categorization (page topic), localization (which location does your web page represent, if any), quality (how relevant and important your content is) and traffic (the level of activity on your page such as the number of visitors, type of visitors, visit duration and visit frequency).

All these signals, as well as others, factor in to where in the pecking order you belong. If your website is exiled to page 3 or 4 in a relatively uncompetitive market, then chances are the signals search engines are picking up from your site show weak content and little user engagement.

Of course, this entire explanation is an over-simplification, but I’ve had so many business owners comment to me about their theories on how search works, that I thought a very basic explanation was in order.

Indeed, some search signals are very complicated such as the level of site-wide signals versus page signals, the authority of web pages that link to yours, the freshness of articles and the depth of engagement a user shows on your website.

Google has stated that it has over 200 search factors that it uses as well as thousands of sub factors; all designed to categorize your web page in the right location and in the right order.

I hope this post helps you understand your own place among the results. It’s important to know how search works if you want to improve your visibility to the consumers who are searching for a product or service like yours. To know more, get it right from the horses mouth.