Written by Emre Sokullu and edited by Richard MacManus
You may feel relatively satisfied with the current search offerings of Google, Yahoo, Ask and MSN. Search today is undoubtedly much better than what it was in the second half of the 1990's. But Internet search is still in its infancy and there's much room for improvement. Moreover, the super high valuation of Google on NASDAQ pushes investors and researchers to find better search solutions - to be The Next Big Thing. And these wannabes are not only working on discovering better indexing techniques, they're exploring new horizons like vertical engines, meaning-based search, intent-driven search, new clustering methods, and much more. In this post, we look into latest trends in the search industry.
We have positioned the latest search trends into 3 main categories:
- UI Enhancements
- Technology Enhancements
- Approach Enhancements (Vertical Engines)
UI Enhancements
Snap promises a better interface for search, using the latest advancements in browsers and AJAX technology. Although there were earlier, similar implementations, preview powered search is perhaps the biggest innovation of Snap. With Snap's preview powered search, you don't necessarily need to visit the site to see if it satisfies your needs - you can see a dynamically loaded screenshot in the right side of your window.
According to a Microsoft study, users spend 11 minutes on a typical search - so potentially Snap can radically shorten this time. Another benefit is that it allows you browse the search results with a few key strokes, which is another big usability enhancement. However it's worth noting that Snap is slow to process searches as a result, because there's too much Javascript and it's too heavy for most modern browsers and hardware. Also, from a technology point of view, Snap doesn't have much to offer - it uses Ask's existing technology. However they have introduced a power of masses approach with options for "This page is Junk" and "This page is Perfect".
Snap's real time query recommendation is also a little similar to an idea once tested at Google Labs. All in all, Snap doesn't bring anything new to the table, but it's a good mashup of some of the innovative ideas in search that we've seen in the last few months.
SearchMash is actually a Google site, to test their latest search innovations. SearchMash follows the basic Google principle - it's cutting-edge, but still plain and simple. When you do a typical web search, you also see image, blog, video and Wikipedia results in the right side of the screen. And there's absolutely no noticeable speed loss, thanks to AJAX. Basically it is a shortcut to reach all the information you need.
The best innovation of SearchMash is perhaps the "More web results" bar. I strongly recommend Google find a way to implement it into their default engine immediately. It makes it much easier to browse the search results. When you need more information, simply click on "More web results" and new results appear at the bottom - enabling you to continue scrolling down on the same page, instead of opening a new page. SearchMash also allows you to give feedback about the results; this may be a sign of the introduction of power of masses into Google Search.
All in all, SearchMash shows that while Google continues to keep itself simple, it also has absolutely no intention of giving way on the innovation front to upcomers. All of the new features in SearchMash are discussed on their About page.
Live.com, the new internet initiative of Microsoft, had many innovative ideas at the beginning. However as Vista's official release date gets closer, it has become a much more traditional search engine. Besides the technology advances in their algorithms, which Microsoft hopes will enable it to compete with Google, there are/were many UI enhancements as well. There used to be, for example, an infinite scrollbar in Live.com - but this seems to have been removed for the final public release.
Most innovations in the image search interface have been kept though - the tiered zooming feature is the most blatant one. Live's Image Search offers seamless user experience enhancements. The infinite scrollbar functionality fits very well and saves you from the hassle of clicking and waiting. And Scratchpad functionality allows you to pick your favourites and compare them smoothly.
Overall we can conclude that Live's interface, when compared to old MSN and Microsoft sites, got more simple and Google-like.
Technology Advancements
Hakia's motto is "Search for Meaning". Founded by seasoned nuclear scientist Riza Berkan, Hakia has raised more than $30M so far, mostly from European private investors. With Hakia you don't search keywords, instead you directly ask questions to the search engine. Hakia makes deep semantic analysis on the pages they crawl. It introduces a new mosaic-like indexing method called QDEX (Query Detection and Extraction). Despite all these nice promises, currently Hakia does not always return the correct results. However they're still in public alpha release and the company is set to debut its full operations in Jan, 2007. After this date, we will have a better chance to judge Hakia's capabilities. Note that Hakia works on top of Microsoft technologies.
Also see Read/WriteWeb's recent post reviewing Hakia.
Clustered Search of Vivisimo and Ask
Neither Vivisimo nor Ask are new companies. Both offer clustered search, which means fragmenting the results of your query so that users can see related terms and go deeper or broader in their data mining. Vivisimo was the first to offer it and it's very useful in cases where you are researching a topic that you're completely new to. Ask's approach is less dense than Vivisimo's and is somehow similar to Live's related results feature. But as stated above, clustered search is probably not something you'll need all the time - it's more a side feature that may be helpful in some cases.
Read/WriteWeb profiled Ask last month.
Intent-Driven Search by Yahoo!
This is a brilliant idea. Yahoo's research project Mindset brings you results according to your search purposes. For instance, when you enter "Rolex Watches" in the search box, you may be willing to buy a Rolex Watch or make an encyclopedic research about the company. Yahoo's intent-driven search allows you to specify your intent and get the most relevant results.
Note that intent-driven search is still in a very early phase, but it's very promising for mainstream users.
Google's Ori Alon
In April this year, Google bought a patented technology that allows them to show related terms after your query. For example, if you search information on the War of Independence, this technology gives you a list of related words - like Etzel, Palmach, Ben-Gurion. The patent was taken by an Israeli phD studying in Australia. Google has not released this feature yet on Google or SearchMash, but it is expected to be shown soon. Also, it is rumored that Microsoft and Yahoo were also after this patent, but Google won the race.
Del.icio.us and Power of Masses
You may ask, what is del.icio.us doing in between all these search sites - isn't it just a bookmarking system? Well, the answer is both yes and no. While it's true that it's a bookmarking site, Yahoo probably didn't buy them just for bookmarking. Actually del.icio.us is also a great tool that empowers the search results of any search engine. Because when you bookmark a site, this indicates the site is a useful resource - so its "pagerank" should be increased. In other words, del.icio.us can actually be used as a search engine, fueled by the power of masses principle. And del.icio.us is not alone in this - Wink and Snap are also trying to use the power of masses in their search offerings.
Supposedly, Google also uses some sort of power of masses with their Personalized Search and Google Toolbar offerings.
NLP (Natural Language Processing) powered Powerset
While still in stealth mode, Powerset has already raised $12.5M in pre-money valuation from several venture capital companies and angel investors like Reid Hoffman, Luke Nosek and early Googlers Aydin Senkut and Zain Khan. The difference between Powerset and the traditional search engines is that while typical search engines like Google and Yahoo don't take into account stopwords (by, after, the, etc), stopwords are a very important part of the engine for Powerset. Why? Because Powerset relies on a semantic capability that can be triggered by using these stopwords. So while the "book by children" and "book for children" queries return exactly the same results in Google, Powerset evaluates them separately and somehow cares about your stopwords as well.
Personalized Search
Palo Alto based Collarity is a very new company entering into the personalized search area. The question that pushed them into this challenge is: "Why are your search results exactly the same as the next person's search results?" This is not a very new idea - Google (with its Kaltix acquisition in 2003) and others already offer this feature, albeit weakly. However Collarity seems very strong with their innovative interface (Collarity Slider), outsourced approach (Collarity Compass) and promising technology.
Social Search
Read/WriteWeb has covered the area of social search very thoroughly already in two articles in July by Ebrahim Ezzy. Two good examples are Eurekster's Swicki and Rollyo. Swicki is a community-driven search engine that allows users to create deep, focused searches on a specific niche - and 'learns' from its community. Rollyo allows users to create and publish their own personal search engines, based on websites they decide to include in their "SearchRoll".
Image Search
Image Search has been around for a very long time, but to be frank it's still very primitive. What most image search engines do is just look for text around images and examine the image tags.
Riya was the first to introduce advanced face recognition technologies in image search. This obviously requires a lot of computing power and just because of this, Riya's weekly burnrate is supposedly over $100K. Co-founded by web 1.0 veteran Munjal Shah and face recognition gurus Burak Gokturk and Azhar Khan, Riya is now entering a whole new space - "search by likeness" with like.com. This may come in very handy, for example when you try to find a watch that is similar to the one you have a digital photo of. That's why Riya is expected to make partnership deals with, or get acquired by, e-commerce companies like Amazon and eBay. It's worth noting that Riya was once in acquisition negotiations with Google, but this never happened - and Google ended up acquiring another face recognition company, Neven Vision. So we can conclude that Google is pursuing this technology very closely!
Approach Enhancements (Vertical Search)
Vertical search is a relatively new discipline in search. Basically, vertical engines look up a very limited subset of the internet - so they are more efficient than generic search engines. Because their search area is not so broad, they can adapt themselves for the specific needs and common points of their area of focus. We won't go in too much detail about vertical search engines, as it has already been covered in a recent article in Read/WriteWeb. But we can categorize the major vertical engines this way:
- Jobs: SimplyHired.com Indeed.com, Bixee.com (India), Eluta.ca (Canada), Recruit.net (Hong Kong)
- Travel: Sidestep.com, Kayak.com, Mobissimo.com
- Health: Amniota.com, CloserLookSearch.com, GenieKnows.com, Healia.com, Healthline.com, Kosmix.com, MammaHealth.com, Google Health
- Classifieds: Edgeio.com, Oodle.com
- Blogs: Technorati, Bloglines, Blogger Search, Sphere, Feedster
- Source Code: Koders.com, Krugle, Google Code
Conclusion
The innovation in search does not stop and there's much to look forward to in the search space. What's more, Google and Yahoo search APIs and the open source Nutch and DMOZ projects allow anyone to try out new ideas. Nutch, supported by Yahoo and shielded under Apache Software Foundation, is providing a free global search engine. DMOZ gives you a very large open source web directory edited by volunteers.
Google will have a hard time competing not only its big adversaries like Microsoft, Yahoo and Ask - but also the ambitious startups that are opening new dimensions and bringing forth new approaches. We will probably hear of acquisitions in this space as well.
We may not have covered all the promising new search offerings here, so please let us know your feedback in the comments below. Also let us know which of the above approaches sounds the most promising to you - and why.