Search problems

The search function has been changed. I am trying to find posts and suddenly I can't anymore. Tried searching vor "javascript" and it didn't find the posts which it found before. Also could it be that the search arguments are 'or'ed instead of 'and'ed...?

Yeah, I'm finding the new search function a bit weird. But again, if I give it some time I might get used to it.

The forum search has changed from a simple wildcard LIKE search to using fulltext indexes.

So, in theory, your search results should be more relevant.

Could it be that it's simply the order of results that is different?

As a test you can still use the old search functionality. So, a search for "javascript" will return the following:

Old search : www.scratchlive.net
New search : www.scratchlive.net

Slightly weird thing for me is that I get MORE results with the old search (I can think of a possible explanation for this but I won't bore you with the details).

I'd be curious to know which posts for you are not found in the new search that are found in the old search.

Okay, sorry I was not precise enough with my first post.

The new search function does not support searching for substrings. You can't search for "javascr" thinking that it will also find posts which contain the string "javascri", "javascrip" and "javascript". It would be nice if the new search would support this functionality as well.

The new relevance sorting is more hindrance than support cause I can't see any real benefit (since ... it does not work for my searches). In a highly linked graph (www) it makes sense since you can weight each link making it possible to calculate a relevance value, but... how would it work on a not so much linked graph like you have on a forum? Just curious.

---

Also the display for the search results could be improved. I like the old format where you could easily find the information. Maybe a more visible visual aid to separate the different information (author, thread name etc) could help?

MySQL full text searching doesn't support partial word matches.

There will times when this could be problematic but, dare I say it, I don't think searching for "javascri" or "javascrip" is one of those situations. In that case the *workaround* is to search for the complete word "javascript".

Sorry if I sound like a smart ass.

Again, I like to see some examples of this in action.

To return to the earlier example of searching for "javascript", this is a single word search so the *most* relevant result is one that features this word the most times.

The keyword highlighting in the results is not 100% reliable but a quick look at the results would suggest that they are displayed according to relevance.

This is more evident is multi word searches.

Searching the forum is always going to be tricky. For example, I'd really like a way to treat each thread as a single entity in terms of search results, rather than searching each message seperately.

That way an entire thread about javascript would be considered more relevant than a single post that happened to contain the word "javascript" 5 times.

Our current database structure means that we can't do this is a way that performs adequately. But never say never.

lol, that's not my problem :) It worked fine with whatever was used before.


Sure... but then you could specifically say "hey, I don't want substring matching, I want the fullstring match... enclose in ' ' ."



A very simple example:
I am looking for a "tool". I search for "tool". But damn.. this very relevant thread talks about "tools".

Substring matching was cool. The current matching is not up to par with it.



True.



We could discuss whether this is correct or not :)

True.

Everyone has different ideas as to what constitutes good search results.

For me, search is a constant work in progress. It can always be better. Which means that, as always, your feedback is much appreciated.

MySQL 5.1 (which is in release candidate phase) supports word stemming in its full text search. So while that still wouldn't catch "javascrip", it would match "tool" to "tools", as with other English word stems. And I believe you can plug other vocabularies into it and do stemming in other languages, although I haven't looked deeply into the documentation on it yet.

So... Yeah, that means word stemming will be possible in the future. But right now, not so possible.

One short term improvement that I can see having some pay off is having the option of sorting by date (with most recent first). Which would only really make sense for the forum search, not the others.

Although having sorting by relevance stay as the default seems the most sensible to me, as people who're searching for answers to technical questions or support questions are going to desire relevance over recentness, in most cases, I think.

The current results do not have a link the found post, only to the thread.

Also, I would like to see an option to sort by posting date. For example sometimes I want to check out the recent posts of a certain user - with the new search this is not possible.

And... a visual indication whether this found post has been read or not. The current search offers this - new search does not offer this.

When you click the link from a user's profile page it shows you their messages in date order, newest first. Same with if you put a username in the forum search options and don't enter any search keywords.

I agree it would be useful to have the colourising back, to indicate whether you've read the message before.

Doh! Thanks. I didn't look properly. Time to get some sleep. :)

Oh, before I complain again (yeah... sorry about that constant complaining :-/ ) ... is there a way to search in my PM's?

There isn't. But I have tentative plans (or at least, it's on my personal wish list) to add a search-as-you-type feature to all forum views. So you can apply a live filter to the area you're viewing (or your tracked or private discussions lists). Much like how Whitelabel.net's searching works now.

When that'll happen, I can't say, as it's more of a luxury feature than a requirement at this stage.

This was a silly oversight on my part. Fixed now


Implemented now.


Yes, this makes sense for forum results. Coming soon.

i was comin to make a thread about this. liked it a lot better before. don't know all the technicalities.

I've tried some simple searches such as '1.8' & '1.8.2' Neither have yielded results.

I must be missing something.

shiestO!,
Could you give me an example or two of what you prefer with the older search?
Is it the results themselves or the way that they're displayed etc etc?

If I type

1.8.1

in the search field, shouldn't the search return with something? 1.8.1 is an official release, there's even a link to is on the home page, I believe.

Am I missing something here? Am I being dense?

Thanks.

www.scratchlive.net <- lots of results

www.serato.com <- no results.

www.serato.com <- 2 hits

www.scratchlive.net <- 59 hits.

You have a typo in the serato.com one. There are 48 results, 49 once I post this:

www.serato.com

Its the full text search on Serato.com being more accurate (IMO). The ScratchLIVE.net one returns irrelevant hits such as Astrophysics and Catastrophy.

Are you searching for posts by shaun or Shaun W?

Lets try this with my name

www.serato.com
464 results

www.scratchlive.net
686 results, including posts by Chopper Dave, DaveAwesome, RaveDave, Daveykaywavey, and Lil Dave.

I only wanted to see posts by me. There is no way to do that on the old forum search. I think the new Serato.com search results are more accurate.

Eek, my bad!



Actually... In the old forum I could just enter "shaun".


In this context you are right - if you know about the fact that the name is to entered exactly as it is.

Thanks.

You'll know when your search for posts by "shaun" comes up empty ;)

I thought it was a(nother) bug. :-P

Touché

hey sorry man! didn't see this. yeah i guess it's just the way it's displayed that bothers me cuz it's different, but i can get over that, it's just visual preference. i think i just ran out of patience for searching because the search system seems to load slower (90% of the time i'm on my crappy work pc while i'm posting)

Interesting. I'm guessing that this is more of a perception thing than anything else.

In my testing the new search queries were definitely faster. Not a massive improvement, but an improvement none the less.

Assuming you have javascript enabled the search results are displayed using a bit of AJAX'y cleverness. If you're using a truly crappy machine and/or a browser with poor javascript performance (ie. Internet Explorer) there could be a bit performance hit rendering the results.

You could try disabling javascript and see what happens.

Could it be that the parameters in the search field are OR'ed rather than AND'ed?

I get 14 results if I search for "common files" on the old search, the new search gives me more than 1000.

The new system uses full text searches. This means that some results could be returned that only feature one of the two words in your search query.

But, in theory, the best matches should appear first.

How do I do a "AND"'ed search?

could anyone help me with a problem im having with serato live

An issue that I have noticed with the Search feature is this:

If I search for a given set of terms, and choose which post I would like to go to, if that post is in the collapsed/hidden section of the thread, than I will only be directed to the first post in the thread, not to the specific post that I was searching for. This issue is highlighted in that, while you are in the correct thread to find the post that you are looking for, these threads are usually quite long, ergo the desire to jump directly to the post I clicked on in the search results. For example, I just searched for the following terms (knowing that the result would be a post relatively early in the Serato-setup picture thread started by phil back in 2005): set up mackie grrillatactics (I knew that my post mentioned Mackie speakers).

Results: www.scratchlive.net

As you can see, the search correctly found the post by me referring to my mackies in that thread, but as this post is somewhere in the midst of what is now around 5000 posts, I am unable to jump to that specific post. Is there any way to add functionality to allow users to jump to the correct post in long threads where many of the posts are not (usually) initially loaded?

Hi grrillatactics.

Yes, we're aware of this problem. We've discussed ways to work around it but haven't come up with a satisfactory solution yet.

Coolio, Chris. I had searched to see if there was already a topic addressing this, but didn't find anything, and figured that rather than start a new thread, it might be easier if I put a serious search-related discussion within one that is already active.

Keep up the good work, guys!

Absolutely - much easier to keep track of a smaller number of threads.

I always appreciate it when people take the time to succinctly and clearly report issues and bugs, so thanks.

Speaking of search problems...

www.serato.com

I search for "itch" in all areas, sorted by date, jumped to the last page (31) and the last post is from sometime in September 2009. What happened to the other posts?

At the moment Sphinx is configured to return a maximum of 1000 results. So I guess you're hitting the result limit.

But how useful is it to search for "itch" in all forum areas anyway? Are you really going to read through all the results? Or were you more interested in seeing a count of how many results were found?

Neither. I was looking for the first Itch discussions. I wanted to check some stuff.

OK, thanks for the explanation nik.

Yeah, that's a limitation of the current system for the time being. Sorry :-(