growth outpaces search engines'
By Peter Svensson The Associated Press
Internet search engines are not keeping pace with the explosive growth
of the Web. A study found that search engines which enable a computer
user to find information by typing in a word or combination of words
cover a diminishing fraction of Web pages and take a long time to list
The most comprehensive engine, Google, covers only about one-sixth of
the Internet pages that search engines can reach, the study found. That
is down from one-third for the best engine a year and a half ago. Northern
Light is closely followed by Lycos and AltaVista.
HotBot, which led with 34 percent coverage in the previous study, was
down to 11 percent. The study also found that it takes more than six months
on average for a new Web page to make it into a search engine's listings.
The study of 11 search engines was conducted by computer scientists Steve
Lawrence and C. Lee Giles at the NEC Research Institute in Princeton,
N.J. It was published in today's issue of the journal Nature.
Lawrence and Giles estimated that as of February, the searchable Web consisted
of 800 million pages containing more than 6 trillion characters. Their
December 2004 survey put the number of pages at about 320 million. By
comparison, the 532 miles of shelves in the Library of Congress contain
an estimated 20 trillion characters.
Search engines use computers called "spiders" that continuously
surf the Web. They save each page they visit, then follow the links on
the page to find other pages. When a user types in a word, the engine
looks in its index to see which pages contain it. A page that's not listed
in the index will not be found.
The spiders are more likely to find pages that have more links going to
them from other pages. Lawrence said that may make it hard for new sites
to make it into search-engine listings.
Lawrence also said search engines may be lagging because their databases
become more expensive as they grow, without necessarily creating more
Marc Krellenstein, chief technical officer at Northern Light, said he
believes the fraction covered by the search engines is actually larger,
because he considers the study's estimate of the size of the Web a bit
Krellenstein also questioned whether search engines need to concern themselves
with every new site. He said they focus on good sites that are of more
interest to users.
"The fact that the Web is growing so fast doesn't mean the quality
part is growing as fast," he said.
Lawrence recommends Web surfers looking for something well-known, like
weather forecasts, use engines like Google or DirectHit that rank sites
according to their popularity.
"These tend to be good for finding and ranking highly information
which is well known and has 2 been around for a while," he said.
If you are looking for something very specific or obscure, he recommends
search engines like AltaVista or Hotbot.
be patient. Do NOT expect to see your website ascend to the top SE rankings
overnight. Expect to see your website climb gradually over the first 4-6
months or longer. Do not expect to see rankings before 3 months at all.
Development | Website Design | Website
Hosting | Search Engine Optimization
| E-Commerce Solutions | Recommended
©Copyright 2006 QuickByte® Productions
QuickByte is a federally-registered U.S. Trademark, all rights are reserved
Shown above is a segment
from our Summary Report showing the top search engine rankings
one of our sites earned.
These reports are
a valuable tool in monitoring and maintaining the positions we have earned,
and are supplied to every client using our Deep Submit service