Disney Count support may be spotty from here. We apologize for the inconvenience.

Advice on writing a faster search?

Any tips/resources on writing a faster search engine?

Tools: .Net, SQL Server.
Permalink Kenny 
July 16th, 2007 5:23pm
Use nutch. net.
Permalink Bot Berlin 
July 16th, 2007 5:27pm
I mean this, http://incubator.apache.org/lucene.net/
Permalink Bot Berlin 
July 16th, 2007 5:27pm
'faster' is relative.

Do you want faster web server response? Faster database response?

I really don't like saying 'throw hardware at it'. But SQL really really does act better when it has it's data and logs on separate physical disks from one another and the OS.
Permalink Send private email JoC 
July 16th, 2007 5:31pm
google dotlucene, a dotnettified version of lucene...very, very good
Permalink Bluebeard 
July 16th, 2007 5:38pm
"Bluebeard" - i posted it.
Permalink Bot Berlin 
July 16th, 2007 5:41pm
+1 lucene.
Permalink Send private email Colm 
July 16th, 2007 5:44pm
Precaching seems effective: http://research.yahoo.com/node/579/2922

Maybe use squid in reverse proxy mode.
Permalink son of parnas 
July 17th, 2007 12:13am
Try this search problem.

Start with a keyword -> id index.  Now add a twist: a lat+lon -> id index.

Search algorithm is find ids within lat+lon that match also have a keyword match.

Oh, there are 1 billion ids and 20GB of keyword data.
Permalink Michael B 
July 17th, 2007 12:18am
It's late.

> Search algorithm is find ids within lat+lon that match also have a keyword match.

Search parameters are a location + keywords.  Algorithm is find all ids in index lat+lon -> ids that are within N mile radius of specified location.  Intersect ids with matching keywords.
Permalink Michael B 
July 17th, 2007 12:20am
id -> lat,lon exists too for all intents and purposes.
Permalink Michael B 
July 17th, 2007 12:31am
They're plural too.

keyword -> id1,id2,id3,...,idN

id -> lat+lon1,lat+lon2,lat+lon3,...,lat+lonN
lat+lon -> id1,id2,id3,...,idN
Permalink Michael B 
July 17th, 2007 12:38am
lucene.net looks really cool.  we wouldn't need sql server with it, making it superfast i imagine.

re: latitudes and longitudes: interesting, but i'm not sure i understand.  are we looking at a table as a map? 

i'm still leaning towards utilizing sql server's full text indexing somehow.  while it may be as blazingly fast as a pure search engine, it might be faster and easier to develop with.

and advice in this regard would be much appreciated!  (i'm stilling going to play around with lucene, so thank you for that as well)
Permalink Kenny 
July 17th, 2007 9:10am

This topic is archived. No further replies will be accepted.

Other topics: July, 2007 Other topics: July, 2007 Recent topics Recent topics