This is pretty impressive. It beats average human performance in the history category, but is outperformed by humans in literature.
Note that part of the data preparation process includes building a dependency-parse tree. From the abstract I'd thought the model was learning to do that too, which would have been very impressive.
In general this approach is somewhat related to [1] in that both rely on knowledge representation similarities. I'm not entirely clear about how this groups approach interfaces the learned model and the IR querying.
"Correctness" is determined by usage, and in this case usage has decided that "Factoid" means small fact. If you use it in any other way, people will misunderstand you, which is an impediment to comprehension. Explaining yourself is an impediment to having an actual conversation, as it derails the conversation.
Usage is floating and amenable to change. At some point usage was different from what you claim it is now (do you have sources to back that up, btw?). The "small fact" people sure didn't let 'em stop that from using the word wrongly (at that point of time, at least, "small fact" was wrong according to your definition, also).
Edit: I checked the Merriam-Webster, and there at least the classic definition is listed as the primary meaning. The alleged "current use" is listed as secondary. I think we can agree your claim to be a factoid - we just can't agree on what that means! :)
It would really be interesting to see a "computers-on-Jeopardy" competition that was open to all comers. (The 'Watson' appearance, while impressive in many ways, was essentially a contrived IBM infomercial.)
I think you are describing TREC[1]. Over on the chatbot arena, there is The Loebner Prize[2], also interesting, but in general the systems are over learned.
TREC, as far as I know, doesn't have a competitive question-answering format, with definitive right or wrong answers.
Jeopardy is a widely-understood game format; a competition-via-API would be understandable to a wide variety of competitors, from leading researchers to precocious cranks.
Sure, Jeopardy is peculiar. Still, it's popular and well-understood, so an open competition, where anyone can upload their playing-agents (quirks and all) could draw big interest. It'd be more like recurring competitive computer chess tournaments than the stage-managed Watson-on-Jeopardy series.
Yep, I can see it now. Overhearing conversation about getting high and food and saying "Do you want the 2-day or 1-day premium shipping for your marijuana + cheetos order?" ;-)
Note that part of the data preparation process includes building a dependency-parse tree. From the abstract I'd thought the model was learning to do that too, which would have been very impressive.
In general this approach is somewhat related to [1] in that both rely on knowledge representation similarities. I'm not entirely clear about how this groups approach interfaces the learned model and the IR querying.
[1] Open Question Answering with Weakly Supervised Embedding Models http://arxiv.org/pdf/1404.4326v1.pdf