Users of search engines would be more successful with
retrieving information if they understood how search engines worked. There are many ways that search engines
work, but we will discuss two common ways.
According to Kurt Franklin in the article How Internet Search Engines Work, he describes keywords as one way search engines work. First search engines have "robots" called spiders, which crawl the web and search for keywords or documents that match common search query terms used by search engine users. When web pages are designed, authors of web pages specify certain keywords for their document(s), which are called metatags. The spiders look for these metatags that match a search word query.
If the author of the web page does not specify certain metatags, then the search engine goes through the document itself and pulls out keywords that may be significant, either if they appear numerous times throughout the whole document or in the lead paragraph. The search engine then makes a list for every keyword in an index. According to Danny Sullivan’s article Search Engine Sizes, “The larger the index, the more likely the search engine will be a comprehensive record of the web.” A larger index means that a search engine has searched more of the web, and has more information than a search with a smaller index. This does not always mean that a user will get better results. In the index, the keywords that were found, or even the entire document, are put into a database for the search engines, where everything is stored. When a person puts in a query term to be searched, the search engine goes to the database to find any matches. After it searches its database, a page shows up with hyperlinked web sites that contain matches to the query term.
Concept
Based Searches
Another way search engines do searches
is called concept-based search, also called clustering. In this kind of search, the search
engine tries to determine exactly what a user means with its query term. The search also tries to determine what
a user means by examining the other words in the document in relation to the
query term. This concept
based search works by
calculating the frequency where certain words appear. When numerous words or phrases are tagged to signal certain
ideas or concepts that are close to each other in a document, the search engine
decides that document is about a particular subject.