The Data From Our Search Engine Field Test
What web sites did teachers review? How did the search engines rate?
Below you find the data from our experiment. We did not want to alienate any of the web sites as a result of this test. We purposely left out the rating for each individual web site that came up in the result and just provided the overall average. We added commentary on our experience with each testing group.
We also took the precaution of placing the URL of the web sites that were virus havens in an image, just to provide you with complete data. This way you are not putting your computer in harm's way. We do not recommend visiting those web sites.
This is the exact letter case that was used to produce the search results for all tests. The time that the search results were retrieved is noted with each test.
Group 1 - water cycle lesson plan
Group 2 - writing worksheets
Group 3 - Thanksgiving Kids Crafts
Group 4 - Lesson Plan Template
Group 5 - Establishing Class Rules
Group 6 - Parent Teacher Conference Forms
Group 7 - Middle School Report Card Comments
On With The Actual Data...
 |
Field Test #1 : water cycle lesson plan |
On our first field test teachers examined the search engine results for the phrase "water cycle lesson plan".
The results were pulled at 4:36 PM EST on April 21,2011
See the search engine results that were tested.
The Overall Results:
- Blekko - 83
- Dogpile - 82.5
- Bing - 80.5
- Google - 77.5
- Ask - 74
Commentary on This Field Test:
This field test produced very similar quality results among all the contestants. The difference maker came from the results having 1 or 2 outstanding sites in comparison with other search engines.
 |
Field Test #2 : writing worksheets |
In this field test teachers examined the search engine results for the phrase "writing worksheets".
The results were pulled at 2:12 PM EST on April 29,2011
See the search engine results that were tested.
The Overall Results:
- Dogpile - 70.5
- Ask - 68.5
- Bing - 64.5
- Blekko - 58
- Google - 55
Commentary on This Field Test:
This field test produced the worst search results by far. This is entirely due to the inability of the algorithms to understand the difference between handwriting worksheets and writing worksheets. When a teacher searches for a writing worksheet they are expecting a series of writing prompts or grammar based work. Instead, most of the results focused on print and cursive handwriting. This really displayed the level of improvement needed by all search engines. Search engines don't need to be mind readers, but there is a need to be more specific with the results.
 |
Field Test #3 : Thanksgiving Kids Crafts |
In this field test teachers examined the search engine results for the phrase "Thanksgiving Kids Crafts".
The results were pulled at 11:51 AM EST on April 23,2011
See the search engine results that were tested.
The Overall Results:
- Dogpile - 90
- Bing - 87.5
- Google - 85
- Ask - 84.5
- Blekko - 73
Commentary on This Field Test:
Teachers were happy with the results for most engines. In fact, most of the teachers printed out a number of the sites they were reviewing to use later. Blekko had trouble with this search in that some of their results were from very weak pages with a single project on them. If we would have used the singular form of the phrase when performing the search, we would have understood.
 |
Field Test #4 : Lesson Plan Template |
In this field test teachers examined the search engine results for the phrase "Lesson Plan Template".
The results were pulled at 9:02 AM EST on April 24,2011
See the search engine results that were tested.
The Overall Results:
- Bing - 83.5
- Dogpile - 81
- Google - 78.5
- Blekko - 69.5
- Ask - 63
Commentary on This Field Test:
Ask produced 2 results that were way off the mark. One site was a dead link that has been dead for over 8 weeks. The other result was a page with a heading but no content at all. As a result Ask's rating score suffered on this field test.
 |
Field Test #5 : Establishing Class Rules |
In this field test teachers examined the search engine results for the phrase "Establishing Class Rules".
The results were pulled at 1:49 PM EST on April 24, 2011
See the search engine results that were tested.
The Overall Results:
- Google - 93
- Dogpile - 90.5
- Bing - 84.5
- Ask - 72
- Blekko - 69.5
Commentary on This Field Test:
Google really blew away the competition on this one. The sites that it displayed were all outstanding and exactly what we were looking for. We expected this from Google on all searches, but the results as a whole were not there.
Blekko and Ask had a few results that were way off the mark. They were not even in anyway related to what the teachers were expecting.
 |
Field Test #6 : Parent Teacher Conference Forms |
On our first field test teachers examined the search engine results for the phrase "Parent Teacher Conference Forms".
The results were pulled at 9:12 AM EST on April 29,2011
See the search engine results that were tested.
Note: Some results are removed from the list above. Below are the URLs in image form, so that you have complete data. We do not recommend you visit these web sites at all. This is because when these 5 sites were tested they tried to install Malware or a virus on our systems. We took a screen shots of the infected pages of content so that teachers could still grade the content of those pages.
Malware Infected Sites From Results
The Overall Results:
- Bing - 81
- Dogpile - 80.5
- Google - 79
- Ask - 68
- Blekko - 42
Commentary on This Field Test:
In this test, Blekko blew it BIG TIME! We had three Blekko results that lit up Norton Antivirus like a House on the December Holidays. Besides that, the actual content on those pages were just lists of words.
Ask also had two sites that tried to give us a drive-by download of malware. The content on that site was related to what we were searching for. You will be ready for your parent-teacher conference but you may have infected your computer.
Unlike typical Google, Google produced one completely worthless result.
 |
Field Test #7 : Middle School Report Card Comments |
On our first field test teachers examined the search engine results for the phrase "Middle School Report Card Comments".
The results were pulled at 4:51 PM EST on April 28,2011
See the search engine results that were tested.
The Overall Results:
- Bing - 78
- Dogpile - 76
- Google - 69
- Ask - 63.5
- Blekko - 21.5
Commentary on This Field Test:
We tried to do a really specific search queries, but the problem was that we always had heterogeneous groups that did not have expertise in those specific areas. However, we were lucky enough to have our 7th group composed entirely of Middle School teachers.
In this test Blekko blew it BIG TIME again! 7 of the results received a 0 rating (The Goose Egg). These results had absolutely nothing to do with what teachers were looking for.
Bing gave some solid results on this phrase. Bing pretty much outperformed the other competitors on this one.
How Did We Get These Results?
Part 2: Which Search Engine Did Teachers Say Was The Easiest To Use?