The Semantic Web Options

[Black_Knight]

Thanks to my friend Stefan Smalla for bringing this one to my attention at http://www.smalla.de/infofeed/archives/001279.html

I've written a few times about the Semantic Web and why it is an important next step. One such aspect is improving search by using semantic approaches. As the scientific and technological underpinnings of many semantic concepts are obviously quite complicated, it's hard to find a good and readable introduction into the subject. But here is one: \"Patterns in Unstructured Data: Discovery, Aggregation, and Visualization\" by a research team at the National Institute for Technology and Liberal Education. It's a fantastic read going into quite some detail on the various steps involved in preparing a semantic search engine (Latent Semantic Indexing or LSI, as the researchers call it). All nicely illustrated with concrete examples.

http://javelina.cet.middlebury.edu/lsa/out.../cover_page.htm

I'm still plowing through this one myself, but it already looks like an important and interesting read. Enjoy.

[bragadocchio]

When Ammon made this post a little more than a year ago, it was a link to a very interesting curiosity.

Or so it seemed.

It's an interesting read.

[compar]

Bill,

I'm waiting for the other shoe to drop. And today it seems????

I haven't looked at it yet but it sound like it might have been a forerunner or a portent of the Applied Semantics and Stemming debates and discussions that are going on today in an attempt to explain Googles latest upgrade.

[bragadocchio]

It is a good introduction to those topics.

You get a sense from this paper how such an index might work.

It also has some great suggestions for how the technology can be used in other ways, too. Such as filtering spam in emails...

[Kali]

It seems to be a well trodden path at the moment. Both Kartoo and Grokker are playing with this kind of analysis.

The real question is are people going to be able to cope the presentation offered by these analysis companies or are the old fashioned listings going to prevail?

[Black_Knight]

are people going to be able to cope the presentation offered by these analysis companies

If you run a simple search for Latent Semantic Indexing on Google you can see that this area of research isn't narrowed to just a few companies. A whole lot of scientists are examining LSI as a missing piece of the wider issues of Data Retrieval.

The paper referenced over a year ago states quite well how LSI fits in with other methods of data retrieval, such as content matching (keyword matching) and link analysis.

Just as link analysis started out being examined separately, but was always intended to be actually used in combination with the prior technologies, I'm certain that LSI is already being integrated into the major search engines to some level. Google's acquisition of Applied Semantics is just one small example of the fact that they are indeed looking to incorporate Semantic indexing technologies.

What I'm saying is that this isn't an alternative to current thinking on search, it is a refinement, which can, and will, and to some considerable extent may already, be integrated into the existing algorithm mixes.

[bwelford]

I find all this semantic stuff fascinating. Here's my thought for the morning.

Let's take a test case. I do a search for Paris Hilton. Instantly my favourite search engine is off looking at the web, clusters, authority sites, etc. and serves me back a SERP with some possibly relevant web pages. If it's Teoma or Vivisimo, then it will also show me some possible categories and it may even show me resources.

Compare this with going into a library. I stop at the desk and ask the librarian, "Where can I find some information on Paris Hilton?" The librarian doesn't give me anything, he or she asks me a question.

So why doesn't a search engine instantly ask me a simple multiple choice question? This interactive process would be the most user-friendly and would have high usability. I'm not aware that anyone has tried that approach in a search engine.

To be precise, I am talking about a two-step process:
1. The search engine asks a few questions to make sure the visitor's question is clearly understood.
2. The search engine delivers possible answers to this more precise question.

In practice it's the same database that will be needed to fuel both steps. However it seems as though this would make the whole process much more effective. Has anyone seen a search engine that tries this approach?

[Black_Knight]

Its a very good point, Barry, and one that SEs have only really addressed with tabs and with refined search suggestions so far.

The tabs could be taken a lot further, as could radio buttons, which are currently only used for country-specific filtering.

Considering Altavista's publication of just five major broad categories of search, it seemed that providing five matching radio buttons, each to 'tune' the algo to that spoecific type of search, was a pretty obvious notion.

[Blue]

Has anyone seen a search engine that tries this approach?

Who knows if that theory is being worked on (probably is), but I dont see in our lifetime, the ability for a machine to have a conversation as such with any human. Ive seen some customer service applications that could do a little bit of that, but never be specific enough to not have to at some point include human intervention.

Many have tried to program a machine to be self operational, self supporting. Fix problems on their own, communicate on their own, do programed tasks on their own, etc. To a degree this can be accomplished, but to a larger degree, its impossible for a machine to take care of a humans needs on its own without any outside help. No machine could ever replace the human brain in our lifetime, therefore no acceptable amount of solid relevancy could be shown with it. Humans are imperfect, and its impossible for a machine to determine or analize a humans question to the exact meaning without failing miserably in most cases. Humans cannot 100% interpret the meaning of all things, how could humans program a machine to do so? I still say the human hands on approach is the best method, and even its subject to corruption, but its not prone to mistakes in interpretation as the limitations of machines are.

When I do a search or ask a question, I dont want to be asked several questions in return to direct me to where to go. I want an immediate vision of what I am looking for. For instance, if I go into a new supermarket where I havent a clue of the stores layout to get a carton of milk, I dont want a clerk to ask me question after question of which kind, brand, or size of milk I want, I just want to know where the dadgum milk is at, so point me there. I can sort the brand, kind, and size as I look for myself.

[tosheroon]

But cows milk, milk chocolate, milk of magnesia and the milk of human kindness could be located in totally different parts of the store