More Powerset Hype

MIT Technology Review has another article on Palo Alto startup Powerset (previously covered here), which promises to use NLP and semantic technologies to revolutionize search. Nothing new here, just the same promises as before. As always, I’ll remain skeptical until I see the product in action.

By the way, I’m still waiting for my beta invitation.

via KurzweilAI.net

Overrated Semantic Search

Several IT news outlets have been fawning over Xerox's new semantic-based search engine, which I've covered before. The general idea behind the technology is to analyze linguistic structures in order to improve search results.

Xerox plans to use this technology in legal software to enable "e-discovery" by sifting through massive amounts of documents, searching for information relevant to a case. Perhaps this will lead to the second instance of software being sued for practicing law without a license.

This is all well and good, and a natural progression for the science of search. Not all that dramatic an improvement as the articles would lead us to believe, buy hey, you gotta sell papers, right? However, it aggravates me when the media makes a factual error while covering a topic I’m familiar with...

For example, common searches using keywords "Lincoln" and "vice president" likely won't reveal President Abraham Lincoln's first vice president. A semantic search should yield the answer: Hannibal Hamlin.

Except a Google search for “lincoln’s first vice president” provides the correct result, as does running “Who was Lincoln’s first vice president?” through my quite unsophisticated Answer Machine. While I can’t fault the reporter for overlooking my humble research, shouldn’t they be capable of running a simple Google query? Wouldn’t this fall under the category of “thorough fact checking?” Shouldn’t they run their “facts” through a subject matter expert before publishing them? And I mean an actual expert, not a PR staffer from the company at hand. Unfortunately, more and more tech articles in the media have regressed to little more than paraphrasing press releases.

Furthermore, if I notice obvious mistakes regarding topics I know a little something about, what other incorrect information am I obliviously consuming? And the media wonders why we don’t trust them anymore...

Semantic Search is Coming

I came across an article about semantic search that does a decent job of explaining the differences between statistics-based and semantics-based approaches to information retrieval. It also describes some of the difficulties and shortcomings of the Semantic Web, along with a few of the related natural language applications.

via Slashdot

Machine Analysis of Scientific Papers

There's a lot of exciting work going on in NLP right now, and it's hard to keep up...and even harder to maintain a blog about all of it! Larry pointed me to an article from last year detailing an automated tool for analyzing and comparing experimental reports.

This project sounds like some sort of XML markup scheme for outlining scientific papers, similar to the ontologies powering the semantic web initiative. It probably involves too much overhead to be widely adopted and therefore be useful, as it likely requires the author to spend an awful lot of additional time constructing papers such that the EXPO system could parse it. A much more elegant method would be for the system to perform an automatic analysis and markup of the text, however that would require NLP technology beyond what's currently available.

As you might imagine, a similar hurdle exists for the adoption of the semantic web in general. However, the analysis & synthesis of peer-reviewed journals presents us with yet another "killer app" for NLP. There is simply far too much information covering any given topic being generated for a human being to digest, even experts in a particular field...let alone a renaissance man or polymath to synthesize from diverse fields. Only a machine with advanced NLP capabilities would be able to make sense of it all and create new knowledge from what's already available.

Semantic Knowledge from Google

Again, people out there are stealing my ideas. Where the heck did I put my tin foil hat?

As posted on Slashdot, a team in Amsterdam is working on an unsupervised system that can perform Automatic Meaning Discovery Using Google. This is quite similar to the idea I have of building a much more broad and extensive knowledge base for AIML bots using a web crawler with built-in semantic analysis. Except these guys are limiting the scope of their project to distinguishing "between colors and numbers, and [...] 17th century Dutch painters."

IBM is scanning my brain...

It seems as though I have some competition. Unfortunately, I am heavily outgunned.

Apparently, Pantera got its technology to steal song lyrics inside people's heads from IBM.

As reported on Slashdot, IBM is developing a system called Piquant, which can perform semantic analysis, extracting 'knowledge' from simple text.

Its nice to know I'm not the only one interested in developing this sort of technology.