You've created a great site -- now what? Whether your Web site sells products or provides information, your effort is wasted if no one sees it. And the way to get your site noticed is to make it friendly to search engines as well as your human audience.
Marketing Web sites to search engines has become a business in itself, and many consultants, tools, and search engine optimization (SEO) sites are available to help your site do well in search engines. The information and resources can be overwhelming. But if you have an accessible and usable site, you are already on your way. If you have any responsibilities for a Web site -- whether you want to be involved with a professional search engine company or just want to take charge of it yourself -- you need to know the basics about optimizing your site for search engines.
This four-part series will give you the tools you need to begin your SEO campaign, no matter how big or small. In Parts 1 and 2, you'll learn how search engines work and how to create usable and accessible content for the benefit of search engines and users. In Part 3 of this series, Mike Moran and Bill Hunt, authors of Search Engine Marketing, Inc., will help you get your pages into the search indexes, and in Part 4, they'll tackle search marketing issues specific to large Web sites (see Resources for more information on Search Engine Marketing, Inc.)
Here's the terminology you'll need to get started with this series.
- A directory is a human-compiled search. Most directories rely on submissions instead of spiders. (Read more on SEO and search engines.)
- Keywords, keyterms, and keyphrases
- Keywords, keyterms, and keyphrases are the words you want your Web site to rank well for in the search engine results pages, also called SERPs. Depending on your audience, yours can be one word, a combination of words, or an entire phrase. To reduce word-bloat, I'll use the term keywords to encompass all three types.
- Link farm
- In SEO, a link farm is a page full of links that have very little to do with each other and exist just as links without any real context. People who practice black hat SEO use link farms to increase the number of links to a page in hopes of fooling Google™ into thinking the page is more link-worthy than it actually is.
- Organic listings
- Organic listings are the free listings in the SERPs. SEO for organic listings usually involves improving the actual content of your Web site, often at the page or infrastructure level.
- PageRank is a measurement that the Google-obsessed use to test their rankings in Google. SEO and search engine marketing (SEM) professionals also use the term to describe your ranking in the SERPs and the ranking algorithm points given to your site by Google. No matter how you define it, PageRank is an important part of your SEO success. (More on Google and PageRank below.)
- Paid listing
- Like the name says, paid listings are paid for in search engines. Depending on the search engine, a paid listing can mean paying for inclusion in the index, pay per click (PPC), a sponsored link, or other ways of making your site show up in the SERPs for targeted keywords and phrases.
- A ranking is where your page is listed in the SERPs for your targeted keywords. The goal of SEO is high rankings for the keywords that your Web pages target.
- Ranking algorithm
- A ranking algorithm is the set of rules that a search engine uses to evaluate and rank the listings in its index. The ranking algorithm is what determines which results are relevant to a specific query.
- Search engine marketing (SEM)
- SEM is used interchangeably with SEO, but SEM often refers more to marketing your Web site to the search engines through paid placement and ads, as well as using SEO techniques.
- Search engine optimization (SEO)
- SEO is creating Web pages that are picked up by the search engines through optimizing your content for search engine attractiveness and visibility. SEO is mostly used to increase the rankings of your organic listings. I'll use the term SEO to describe the techniques I recommend, although many of these techniques also fall under the umbrella of SEM.
- Search engine results page (SERP)
- SERPs are the listings, or results, displayed for a particular search. SERP is sometimes defined as search engine results placement. For the purposes of this series, I'll refer to it as a page rather than a placement. In the world of SEO, a good showing in the SERPs is what it's all about.
- Spamming is a method of SEO that attempts to trick a spider and scam loopholes in the ranking algorithm to influence rankings for targeted keywords. Spamming can take many forms, but the most simple definition for spam is any technique a Web site uses to misrepresent itself and influence ranking. The two methods of SEO are based on whether you want to spam or not.
- Black hat SEO: Spamming the search engines. Black hat SEO is lying, cheating, and stealing your way to the top of the SERPs.
- White hat SEO: Optimizing your site so it serves the user, as well as attracts spiders. In white hat SEO, anything that leads to a good user experience is considered also good for SEO.
- A spider crawls through the Web looking for listings to add to a search engine index. It is sometimes referred to as a Webcrawler, robot, or bot. When optimizing your page for organic listings, you are catering to the spider. (Read more on SEO and search engines below.)
Why SEO is important
Now that you have the definitions, you still may be wondering: What is the purpose of SEO? And isn't SEO a little on the shady side?
SEO creates accessible and usable Web sites
Good SEO practices make your site more usable and more accessible, as well as search engine friendly. However, aggressive SEO marketing firms using spammy black hat SEO techniques have given SEO a bad name. But this is true of any type of marketing: There are good ways and bad ways to promote a product, and sometimes the bad ways work -- but only for a little while. Scamming the search engines through black hat SEO techniques is a dangerous road to take and leads to a bad user experience. That's why this series only focuses on white hat SEO practices. With white hat SEO techniques, you and your users benefit. It's a win-win situation.
SEO is crucial for commerce sites
If your business is online, your survival depends on getting your Web site noticed. People can't buy from a commerce site they can't find. And it isn't just a matter of potential customers finding your main page and navigating through the drop-down menus, links, and what-have-you on your site. It's no longer the case that if you have a good main index page, then people will come to your site and continue to navigate through until they find the answer.
SEO is part of the evolution of Web use
People use search engines to find exactly what page they want within your site. If your individual page doesn't resolve in the top of the SERPs, your potential audience might never find your site. Users are more likely to enter your site from a search query than to navigate through a hierarchy of pages. What this means to the Webmaster is that each individual page must be robust enough for searches that it can stand alone. You have to make your case for the search engine spider, and then make your page attractive enough to users that they'll want to stay. White hat SEO is the way to do this.
SEO in practice
To illustrate what it means to have pages that are robust enough to stand alone, I'll use the developerWorks Web site as an example. Because the main page at www.ibm.com/developerworks is the entry for all developerWorks content, we have optimized it very generally for "IBM resource [for] developers," and the main page is the first result in the Google SERPs for a search using this term. Using organic optimization techniques has paid off. Google understands what the page is about and users won't be disappointed when they click on the link and come to a page full of developer resources from IBM.
However, the developerWorks Web site is a lot more than the main page. In the case of developerWorks, some of the potential audience who look for more specific developer resources might use our navigation or our internal search, but many other users prefer to search for content in an external search engine and go directly to the pertinent page.
For example, if a searcher looks for "linux developer tutorials" and the developerWorks Linux page at www.ibm.com/developerworks/linux doesn't surface within the first 10 results or so in the SERPs, then all the optimization on the main developerWorks page is useless to the searcher. But we've been practicing good organic SEO on the developerWorks site for many years, and currently the developerWorks Linux page is the number-one result in the Google SERPs for "linux developer tutorials." Good SEO means optimizing every single page on your site so the search engine knows exactly what you think is important on each page.
SEO can seem daunting when you think of optimizing every single page on your Web site for good placement in search engines. And you might wonder if perhaps there is a trick or a shortcut. There is no trick to optimizing for search engine placement. The bottom line is, create good content that your users will appreciate and the search engines will rank you well for that. You'll learn how to do this as this series progresses.
SEO and users
When you implement good SEO properly, you create a site that is friendly to search engines, and also attractive to users. A bonus benefit of organic SEO is an extremely accessible page. Search engine spiders look at pages the same way a page reader does for a person who is visually impaired. You can use a lynx browser to get an idea of what a page reader sees when it views your site (see Resources for a link to the lynx browser). Page readers also reveal how your site is reproduced on small screens, such as phones or other tiny Web browsing tools (see Resources for more on accessibility). If you can commit to make your pages accessible and easy to understand by a page reader, then you can commit to make your pages search engine friendly.
In the history of SEO, most well-known search engine spamming techniques involved created a Web page that wasn't user friendly or accessible. Two major user-unfriendly black hat SEO techniques are:
- Meta tag spamming: In the early days, search engines read the meta keyword, description, and other tags. Based on the content of those tags, the search engine determined what your page was about and where to rank it in the SERPs. Unfortunately, people took advantage of this and filled their meta tags with the same words repeated to make the search engine think that the page had more content than actually existed. This practice misleads the user and the search engine.
For example, if we wanted to bring in Linux users to the developerWorks main page, but lacked any Linux content, we might do this:
<meta name="keywords" content="linux, ibm, linux, developer, tutorials, ibm, developer, linux, tutorial, tutorial, tutorials, resources, linux, tutorials, developer" />. The user will be disappointed when he or she clicks on the site listed in the SERPs and is shown the developerWorks main page, which might have one Linux tutorial in the rotation for the week, but is certainly not focused specifically on Linux tutorials the same way as the developerWorks Linux section. So many people practiced the black hat SEO technique of meta tag spamming, search engines no longer use the information in meta tags to rank pages.
- Alt tag stuffing: Misuse of alt tags is also black hat SEO because it gives the user and the search engine misleading information about a graphic. For example, a graphic with the alt tag stuffed with keywords on the developerWorks Linux page might read as follows:
<img alt="linux, ibm, linux, developer, tutorials, ibm, developer, linux, tutorial, tutorial, tutorials, resources, linux, tutorials, developer" />. Although the Linux page might be about Linux tutorials, ensure that the graphic itself sends clear information to the page reader about its content. Otherwise, it's a misuse of the alt tag.
The most important thing to remember is that good text is important for search engines and users. Write your text to tell the search engines clearly and truthfully what your pages are about, and the users will follow. (Find more information on creating well-written Web pages under Resources.)
SEO and search engines
When people think about search engines, most think of Google (you can read more about the Google domination factor in the forums in Resources), but in the interest of fairness to all search methods and engines, I'll use the term search engine. The tips in Parts 1 and 2 of this series will be Google-centric, but they will also work for most other major search engines. The search engine you will want to target might vary depending on your intended audience. Because I'll only describe white hat SEO techniques, my information should give you the tools and knowledge to make your Web site attractive to all of the major search engines in the United States. Search engine popularity varies outside of the United States, but good white hat SEO will help your site in most English-based search engines and other spider-based search engines similar to ones in the United States.
Types of search engines
The difference between directories and true search engines is that directories are compiled by humans and search engines are compiled by spiders that crawl the Web by following links. The main focus of white hat SEO is to improve the rank of your site in spider-compiled search engines.
How search engines work -- simplified version
To gather data, search engines send the spiders out to crawl from link to link, organize data, and figure out exactly what the text is about in each page. The search engine uses the information from the spider to organize a list of sites, determined through propriety ranking algorithms, that have the words most related to what the searcher is looking for in the SERP.
How directories work -- simplified version
A pure directory works more like the computerized card catalogs at the library. It has a compiled list of sites that have been submitted to the directory. It already knows what's in this list and in which categories to search for the words entered into the search field. This is because the original submitter often chooses the category. Also, the searcher can browse through the categories of the directory to look for a site with the words the searcher is requesting.
Google is a true search engine. It does pull some directory results from the Open Directory, or the DMOZ (see Resources), but the bulk of the results come from the spider crawl and the PageRanking algorithm (more on PageRank). Yahoo!®, which searches in its own directory first and then uses the Yahoo Bot Slurp! for the search engine search, is an example of a directory-hybrid.
More on Google
Many Webmasters report that Google's search index accounts for a majority of their entire search-related traffic. Because of Google's dominance, you might want to concentrate on optimizing your site for Google (see Resources for Google's own metrics).
It's lucky for Web site developers that Google has become the top search engine not through clever marketing or other non user-friendly tricks, but because it is a true search engine that searches based on quality of content and quality of links. Google has always maintained that whatever is good for the user is good for high rankings in the SERPs.
Google has an Add URL tool but ranks sites by link analysis; if Google isn't led to your site by other sites to be indexed, Google might never give you a high ranking, regardless of whether you submit. Submitting through Add URL doesn't mean a page will automatically be listed, but it does bring the page to the search engine's attention. In the past, SEO experts recommended that you use the Add URL tool to "deep submit" pages that weren't easily found by the crawlers. But now that Google has introduced the Google Sitemaps, you might do better with submissions in that format. Part 3 of this series will elaborate on Google Sitemaps.
In the long run, your SEO energy is best spent optimizing your site to encourage deep crawling by the Google and other spiders.
The Google ranking algorithm is top secret. Here is what's known about Google:
- Google gets its main results from a crawler that spiders the Web.
- Google's spider only views the visible text on the page.
- Google indexes the following file types: html, pdf, ps, wk1, wk2, wk3, wk4, wk5, wki, wks, wku, lwp, mw, xls, ppt, doc, wks, wps, wdb, wri, rtf, swf, ans, and txt.
- The Google index is created with an emphasis on page content and link popularity. One factor that determines the rank of your page in Google listings is the quantity of links that point to the site, the quality of the sites that link to the site, the text in and around the links that point to the site, and what the page itself links to.
- Google reports its results are determined by more than 100 factors.
- Google uses PageRank to examine the link structure of the Web to determine which pages are most important.
- Google also conducts what it calls hypertext-matching analysis to determine which pages are relevant to the specific search query.
Basically, Google uses PageRank to evaluate the quality of inbound links (sites that link to you) and outbound links (sites you link to) by assigning a value to the number and quality (popularity) of these links. High-quality sites (according to Google) receive a higher PageRank. Google then combines PageRank rating with hypertext-matching analysis techniques to find pages that are relevant to your search.
To deter people from scamming the search engine, Google constantly changes its ranking algorithm and updates its index. The best option to rank well in search engines is to make a good site that doesn't rely on taking advantage of ranking algorithm loopholes to rank highly. As in real life, honest SEO is the best policy.
SEO next steps
In Part 2 of this series, you'll learn how to focus on your keywords and find out about the top-left-down keyword optimization strategy. You can determine how technical you want to be with your SEO campaign as the series progresses in Parts 3 and 4, where you'll find out how to get your pages into the search indexes and tackle search marketing issues specific to large Web sites. For now, get ready to roll up your sleeves and start optimizing.
- Search Engine Watch: Read more detailed and technical information on search engines, as well as participate in an active discussion forum.
- Jill Whalen's High Rankings: Get advice on good writing for white hat SEO.
- Bruce Clay's Search Engine Relationship Chart: Get a better idea how all of the search engines are related. These relationships change frequently and this chart is also updated often.
- Jakob Nielsen's Alertbox The Power of Defaults (September 26, 2005): See how search engine users click the top of the SERPs entry more often than can be explained by relevancy ratings.
- Jakob Nielsen's Alertbox When Search Engines Become Answer Engines (August 16, 2004): Find out how people use search engines to retrieve answers to their current questions.
- Jakob Nielsen's Alertbox Information Foraging: Why Google Makes People Leave Your Site Faster (June 30, 2003): Read why, as it gets easier to find places with good information, users will spend less time visiting any individual Web site.
- The IBM Accessibility Center: Discover numerous resources to assist you in learning about building accessible applications.
- The U.S. government's Section 508: Make this your first stop to learn more about section 508 of the U.S. Rehabilitations Act and related accessibility standards.
- Google's own metric reports: Get a better idea of how your target audience uses Google.
- Google PageRank: Learn more direct from the source.
- Search Engine Marketing, Inc.: For tips on how to get into the search index, research which words searchers use to find your site, optimize your pages to be found for those searches, and other steps to drive search traffic to your site, check out the new IBM Press book.
- DMOZ or Open Directory project: Visit this example of an early human-compiled directory search. The Open Directory Project is the largest, most comprehensive human-edited directory of the Web. It is constructed and maintained by a vast, global community of volunteer editors.
- Safari book store: Find a large selection of books on related topics.
- Web Architecture zone's technical library: Find articles and tutorials on various Web-based solutions.
Get products and technologies
- IBM trial software: Build your next development project with software downloads available directly from developerWorks.
- lynx browser: See how your site looks to a page reader (as well as a search engine spider).
- WebmasterWorld Google News Forum: Discuss the latest Google News.
- The Cre8asite Forums on Google: Check out these Search Engine Marketing, Usability and Web Design forums.
- developerWorks blogs: Get involved in the developerWorks community.
Dig deeper into Web development on developerWorks
Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.
Keep up with the best and latest technical info to help you tackle your development challenges.
Software development in the cloud. Register today to create a project.
Evaluate IBM software and solutions, and transform challenges into opportunities.