Notes

Change Engineering Search Motor Ranking Algorithms
Back throughout 1997 Used to do some research so that they can reverse-engineer algorithms utilized by search engines. In that year, the large ones included AltaVista, Webcralwer, Lycos, Infoseek, and a few others.

I seemed to be able to mostly declare my exploration a success. Inside fact, it absolutely was consequently accurate that throughout one case I used to be able to compose a program of which produced the very same lookup results as 1 of the engines like google. This article describes how I did this, and how its still beneficial right now.

Step 1: Figure out Rankable Traits

The particular first thing to accomplish is make a list of what you want to assess. I came upward with about fifteen different possible ways to rank an online page. They included things like:

- keywords in title

- keyword denseness

- keyword regularity

- keyword within header

- key phrase in ALT labels

- keyword emphasis (bold, strong, italics)

- keyword in human body

- key phrase in url

instructions keyword in site or sub-domain

: criteria by spot (density in name, header, body, or perhaps tail) etc

Step 2: Invent the New Keyword

The other step is to be able to determine which key word to check with. The key is to pick a word that does not are present in any vocabulary on the planet. Otherwise, a person will not be able to isolate your own variables for this kind of study.

I used to job at a firm called Interactive Visuallization, and our site was Riddler. possuindo plus the Commonwealth Community. At the moment, Riddler had been the largest amusement web site, in addition to CWN was one of the top trafficked websites on the net (in the most notable 3). I took on my personal co-worker Carol plus mentioned I needed some sort of fake word. The girl gave me "oofness". I did a quick search also it was not found upon any search motor.

Note that a special word can also be used to see who has copied content from your own web sites upon their own. Considering that most of my analyze pages are absent (for many years now), a search on the search engines shows some sites that did duplicate my pages.

Step 3: Create Test Webpages

The next thing to do was to create test webpages. I took my home page for my now defunct Amiga search engine "Amicrawler. com" in addition to made about 75 copies of it. I then numbered every file 1. html code, 2 . not html... seventy five. html.

For every single measurement criteria, I actually made a minimum of 3 html files. Intended for example, to determine keyword density within title, I customized the html games of the 1st 3 files to be able to look similar to this:

1. html:

oofness
second . html:

oofness
3. html:

oofness
The particular html files involving course contained the rest of my home webpage. I then logged throughout my notebook that will files 1 instructions 3 were key phrase density in name files.

I duplicated this type of html editing for about 75 or so files, right up until I had each criteria covered. Typically the files where next uploaded to the web server plus placed in the identical directoty so that engines like google can find them.

Step four: Wait around for Search Search engines to Index Evaluation Webpages

Over typically the next few days, some of the webpages started appearing inside search engines. However a site love AltaVista might simply show 2 or perhaps 3 pages. Infoseek / Ultraseek at the time was doing real-time indexing so I have got to test everything instantly. In some cases, I had to await a few months or months with regard to the pages to have indexed.

Simply inputting the keyword "oofness" would bring upwards all pages listed that had of which keyword, in the particular order ranked by simply the search engine. Since only my pages contained that word, I would likely not have contending pages to mistake me.

Step a few: Study Results

In order to my surprise, almost all search engines acquired very poor rank methodology. Webcrawler used a simple word density scoring system. Throughout fact, I used to be in a position to write the program that gave the exact same search powerplant results as Webcrawler. That's right, simply give it some sort of list of twelve urls, and it will rank these people in the exact same same order like Webcrawler. By using this program I would create any of our pages rank #1 if I wanted to be able to. Problem is needless to say that Webcrawler failed to generate any traffic even if I was listed number 1, so We would not bother along with it.

AltaVista answered best most abundant in amount of keywords in the title of typically the html. It rated a couple of pages method at the end, but I don't recall which criteria performed worst type of. Plus the rest involving the pages ranked somewhere in typically the middle. Overall, AltaVista only cared regarding keywords within the name. Everything else failed to seem to matter.

A couple of years later, I actually repeated this test with AltaVista and found it had been giving high preference to be able to domain names. So check here added a wildcard to my DNS and web storage space, make keywords within the sub-domain. There you are! All of my pages had #1 ranking for any keyword I selected. This obviously directed to one issue... Competiting web internet sites don't like shedding their top positions and will perform anything to guard their rankings when it fees them traffic.

Other Methods of Screening Search Engines

I actually is going to be able to quickly list many other issues that can be done in order to test search engines like yahoo methods. But these are lengthy topics to talk about.

I tested some search engines simply by uploading large copies of the dictionary, plus redirecting any visitors to a safe web page. I also analyzed them by indexing massive quantities of documents (in the particular millions) under hundreds of domain names. We found generally of which there are really few magic key phrases found in most documents. The reality still remains that will a few key word search times including "sex", "britney spears", etc introduced visitors but most do not. Hence, most web pages never saw any kind of people traffic.

Downsides

Unfortunately there have been some drawbacks in order to getting listed #1 for a great deal of keywords. We found that it ticked off some sort of lot of folks who competing net sites. They might usually start by burning my winning strategy (like placing key phrases in the sub-domain), after which repeat typically the process themselves, and flood the search engines with 100 times more pages than the a single page I got made. It produced it worthless in order to compete for leading keywords.

And next, certain data can not be measured. You can utilize tools like Alexa to determine traffic or Google's internet site: domain. com to find out the amount of listings a domain has, but until you have got a great deal of this data to measure, you may not get any able to be used readings. What good is it for you to consider and beat some sort of major web web site for a major search term if they already have millions of guests per day, an individual don't, in fact it is element of the look for engine ranking?

Band width and resources can become a problem. My partner and i have had internet sites where 74% of my targeted traffic was search powerplant spiders. And they will slammed my web sites every second involving every day for years. I would actually get 30, 000 hits from typically the Google spider every single day, in improvement to other spiders. And as opposed to precisely what more info believe, they aren't as warm and friendly as they declare.

Another drawback is that should you be undertaking this for a corporate web web-site, it might not look so very good.

For example , you may recall a few weeks ago any time Google was found using shadow webpages, and of course claimed they have been only "test" pages of content. Right. Does Yahoo have no dev servers? No holding servers? Are these people smart enough to be able to make shadow webpages hidden from regular users but is not good enough to hide dev or test web pages from normal users? Have they certainly not figured out precisely how an URL or perhaps IP filter performs? Those pages have got to have served a purpose, and these people didn't want most people to know about it. Maybe these people were merely weather balloon internet pages?

I recall learning about some pages of which were placed with a hot online & print tech magazine (that wired people into the digital world) on lookup engines. That were there positioned numerous blank obtaining pages using font colors matching typically the background, which included large quantities of keywords because of their most significant competitor. Perhaps they wanted to pay digital homage to CNET? Again, this is probably back found in 1998. In truth, they were working articles at the time about how it is wrong to trick search search engines, yet they had been doing it them selves.

Conclusion

While this methodology is great for learning some things about lookup engines, on the whole I actually would not recommend making this the particular basis for the web site promotion. The quantity of pages to contend against, the top quality of any visitors, the particular shoot-first mentality associated with search engines, and many more factors will prove that there are better strategies to do internet site promotion.

This specific methodology works extremely well for reverse engineering other products. For example , if I worked in Agency. com undertaking stats, we applied a product made by a significant micro software company (you might be making use of their fine main system products right now) to analyze word wide web server logs. Typically the problem was that it took more as opposed to the way one day to evaluate 1 days worth of logs, thus it was by no means up to particular date. A little little bit of magic and even a little tad of perl seemed to be able to produce the same reports in forty five minutes simply simply by feeding exactly the same logs into both systems until the outcomes came out typically the same every condition was made up.
Homepage: http://www.pearltrees.com/blalock04holst