Original Page[1]
Features comparison chart
Most comparison charts of search engines are made from the search engine
users' perspective. This chart is designed for webmasters who care about
how search engines catalog their sites. It is meant to give an
at-a-glance summary of factors that can affect how a site is indexed.
New categories will be added, as they become apparent or known.
Service URLs (mills) Content Meta Tag Support Search Levels Catalog Date
URL Status Check Sells words Page Popularity Used? Catalog Update Repeat
Penalty Alta Vista[2] 30 Full-text Yes 3 Yes No ??? No monthly? No
Excite[3] 50 Full-text to concepts No 3 No No Ads only Some weekly No
InfoSeek[4] 4 Full-text Yes 2 No No Ads only No 3 weeks Yes HotBot[5] 36
Full-text? Yes 3? No No Ads only No 1-2 weeks No Lycos[6] 50 Abstracts
No 3 No No ??? Yes 2 weeks Yes Magellan[7] 10 Full-text? No ? No No ???
Yes 3-4 weeks? No Open Text[8] 1.5 Full-text No 1 No No Yes No monthly
No WebCrawler[9] 0.5 Full-text No 1 No Yes ??? Yes monthly+ No
URLs Cataloged
The more URLs cataloged, the more likely pages from your web site will
be found though an engine. Figures are as reported by each site or other
sources. Debate rages about who's biggest. Excite has a nice explanation
of issues involved in the size debate at
http://www.excite.com/ice/counting.html . No doubt the other search
engines have their own opinions, of course.
Content
Shows whether the engine catalogs the full-text of a page or instead
creates a description or abstract based on a page's text. Full-text may
be better because it ensures that every word from your web site will be
available to match keywords entered by those consulting search engines.
Meta Tag Support
Many believe all search engines acknowledge keywords and descriptions
placed in meta tags. In reality, only a few do. The sites that support
the tag explain how to use it in their online help files (see the " How
search engines work page[10] "). Keep in mind that using the tag doesn't
guarantee your page will become more relevant that other sites, but it
does allow you to control the description that appears.
Search Levels
Some search engines catalog everything on a home page but go no farther
than this "first" level. Others go to all the pages linked to the home
page, the "second level." The process continues if the search engines
continues to follow links deeper and deeper into the site. The levels
shown for each engine were determined during my study.
Catalog Date
The search engines may go out nightly to find new web pages, but that
work means nothing if the search engine catalogs aren't updated with
this new information. Kudos to the engines that tell you how old their
catalogs are.
URL Status Check
WebCrawler is the only engine that allows you to check[11] when your
site was visited by the engine, a very nice feature that other engines
might consider adding.
Sells Words
Only Open Text sells keywords. It began selling preferred listings in
June 1996. Pay the money (beginning at $2,00 for 6 months), and you can
appear in the top ten, guaranteeed. A link to an article about this
service is on the resources page.
* Open Text Preferred Listing rate page
http://www.opentext.com/omw/preferred_c.html
More common place are search engines that let banner advertisers
purchase keywords. These engines are noted on the chart as "ads only."
If the keywords an advertiser has purchased are entered, the company's
ad will appear at the top of the page. Keyword-linked ads do not change
the order pages are listed.
Page Popularity Used?
Some search engines determine the popularity of a page by analyzing how
many links there are to it from other pages. Pages that are more popular
may turn up earlier in a search over less popular pages, or popularity
may be used in other ways. See the page on how search engines work for
more information.
Catalog Update
My estimate on how often catalogs are updated, based on my experience
with managing various sites. Better search engines will constantly
update their catalogs, but since the majority don't date their catalogs,
it can be hard to tell how old the catalogs are.
Repeat Penalty
Two engines will penalize sites using words too often as an attempt to
"spam" the results. InfoSeek penalizes if keywords are used more than
seven times in a meta tag. Lycos began penalizing repetition in May
1996.
Lycos should be applauded for trying to control spamming. Unfortunately,
the engine doesn't define what constitutes excessive repetition. As a
result, webmasters might inadvertently have their web pages fall lower
in the rankings. This is because many relevant web pages have keywords
that repeat "naturally."
For example, Lycos used to top rank the home pages of County of Orange
and the Orange County Register when the keywords "Orange County" were
entered. Few would challenge the relevancy of highly ranking the home
pages of the county government and the area's major newspaper. Now,
those pages are buried in the listings. In their place are lots of pages
of arguably lower relevancy.
Ironically, one of the best ways to avoid the spamming problem without
overt penalties might be to give more weight to a page's popularity, as
explained above. A page with lots of links to it is in effect "ranked"
by those across the web. Good pages get lots of links; bad pages don't.
-----------------------------------------------------------------------
[1] http://calafia.com/webmasters/chart.htm
[2] http://www.altavista.digital.com/
[3] http://www.excite.com/
[4] http://guide.infoseek.com/
[5] http://www.hotbot.com/
[6] http://www.lycos.com/
[7] http://www.mckinley.com/
[8] http://www.opentext.com/
[9] http://www.webcrawler.com/
[10] work.htm
[11] http://www.webcrawler.com/WebCrawler/Status.html