Commit Graph

19 Commits

Author SHA1 Message Date
Sam
9b7cab589a FIX: revert diacritic stripping
See more details in test case and at: https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/16?u=sam
2018-08-31 11:46:55 +10:00
Sam
ac11f8df52 correct regression searching with diacritics 2018-08-24 10:00:51 +10:00
Régis Hanol
c26de01399 Use NFKD normalization instead of NFD 2018-08-24 00:52:53 +02:00
Régis Hanol
bc7b530b0a FIX: remove diacritics instead of transliterating 2018-08-24 00:38:44 +02:00
Régis Hanol
f4ae53d52b Return immediately when there's no html to scrub 2018-08-23 18:00:07 +02:00
Régis Hanol
2fcf2b899e FIX: remove diacritics when tokenizing html for search 2018-08-23 17:13:52 +02:00
Bianca Nenciu
975a72ab7a FEATURE: Make links indexable. (#6285) 2018-08-20 10:39:19 +10:00
Sam
5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
Sam
6676bbd38b FEATURE: index YouTube titles in search
Previously we omitted the titles for videos that YouTube provided
2018-04-26 15:46:52 +10:00
Sam
86d12bd44b FEATURE: search within title using in:title
Also

- Significantly improved search ranking, title is treated most strongly
- Adds tag names to the index
- Run search re-indexer more aggressively
- Re-index topic and all posts on category change
2018-02-20 14:41:21 +11:00
Sam
57a1190b07 FIX: correct issue with search omitting words with multiple dots
Previously we used to break up words with dots incorrectly leading to
missing search terms
2017-12-19 16:04:24 +11:00
Guo Xiang Tan
77d4c4d8dc Fix all the errors to get our tests green on Rails 5.1. 2017-09-25 13:48:58 +08:00
Neil Lalonde
2c56f8df7c FEATURE: show tags in search results 2017-08-25 11:52:59 -04:00
Erick Guan
6e59149a77 FIX: rebuild index when engine replaced (#5021) 2017-08-16 07:38:34 -04:00
Sam
6002f2ca4a FIX: not splitting words correctly for search tokenizer 2017-08-14 16:19:39 -04:00
Erick Guan
f1eccd113c Replace rmmseg gem for cppjieba_rb since better dictionary (#5006)
* Rename locale to ts config in search module to make it clear

* Replace rmmese-cpp for cppjieba_rb
2017-07-31 15:28:48 -04:00
Guo Xiang Tan
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Guo Xiang Tan
7ed9fbb415 FIX: Use default parameter in method. 2017-06-26 11:14:13 +09:00
Sam
0a78ae739d Remove SearchObserver, aim is to remove all observers
rails-observers gem is mostly unmaintained and is a pain to carry forward
new implementation contains significantly less magic as a bonus
2016-12-22 13:13:14 +11:00