search: improve utilization of Lunr lib for better results

[McShelby]

Split from #888 (comment) by @postb99

Hello, since you intend a rework, an additional thought :) Maybe this could better be seen as new issue/wishes?

After reading Lunr documentation I found out that you decided to keep multi-term search to be only "ORed" and this is not my cup of tea but is basically Lunr's one :D but your configuration prevents using it. I would prefer a search that check for presence of every entered term, by design (especially for my site, lol). I would love to have feedback from other users that use the theme.

Secondly, you also prevented wildcard search, but I would like to try it once.

However I'm fully ok with removing tilde (fuzzy search) or caret (negation) because in French caret can be used for linking two words (two search terms then) or building a compound word (that I think would better give several search terms too).

A typical example where I can't find what I look for on my site is "arc-en-ciel" that would be better break into 3 search terms and all of them required for match.

It seems that you removed Lunr stemmers but you see to keep shipping stemmer files. Another thing that would be explained. Did you notice issues when keeping stemmers and your custom boosts?

Theme search page code:

• hugo-theme-relearn/static/js/search.js

  Line 137 in 3a0ae52

  term = term.replace( /[*:^~+-]/g, ' ' );

  this removes special chars

• hugo-theme-relearn/static/js/search.js

  Line 33 in 3a0ae52

  this.pipeline.remove(lunr.stemmer);

  this removes stemmers?

• hugo-theme-relearn/static/js/search.js

  Line 144 in 3a0ae52

  function searchPatterns(word) {

  I'm fine with this

Lunr documentation:

• https://lunrjs.com/guides/searching.html#term-presence "OR" search by default, altered with "+" and "-" characters, also fuzzy search with "~".
• https://lunrjs.com/guides/searching.html#wildcards

Cheers

Originally posted by @postb99 in #888 (comment)

[McShelby]

Hi Barabara. Thanks for taking a look into this.

I must admit that although I authored all the open search issues here, I haven't a good knowledge about the Lunr library. It worked for my use cases and not many did complain in the past ;-)

Most of the existing code is from the ancient past of this theme and not written by me. Especially the removal of the stemmers was something I missed completly during my past visits of the code. This looks really fishy. Sadly the commit message does not mention the reason or the intention behind it.

I am more than willing to change things here but currently search isn't much of a priority for me and it doesn't seem like a quick task. So if you like to work on this, I would be more than grateful!

[McShelby]

@postb99 This is a horrible mess... MihaiValentin/lunr-languages#115

Seems, I have to do a lot of patching for the Lunr library before getting nicer results.