Commit Graph

227 Commits

Author SHA1 Message Date
Sam Saffron
0fb497eb23 DEV: use Discourse.cache over Rails.cache
Discourse.cache is a more consistent method to use and offers clean fallback
if you are skipping redis

This is part of a larger change that both optimizes Discoruse.cache and omits
use of setex on $redis in favor of consistently using discourse cache

Bench does reveal that use of Rails.cache and Discourse.cache is 1.25x slower
than redis.setex / get so a re-implementation will follow prior to porting
2019-11-27 12:36:19 +11:00
Martin Brennan
e7226a8c84
FEATURE: Allow scoping search to tag (#8345)
* When viewing a tag, the search widget will now show a checkbox to scope the search by tag, which will limit search results to that tag on desktop and mobile
2019-11-14 10:40:26 +10:00
Daniel Waterworth
55a1394342 DEV: pluck_first
Doing .pluck(:column).first is a very common pattern in Discourse and in
most cases, a limit cause isn't being added. Instead of adding a limit
clause to all these callsites, this commit adds two new methods to
ActiveRecord::Relation:

pluck_first, equivalent to limit(1).pluck(*columns).first

and pluck_first! which, like other finder methods, raises an exception
when no record is found
2019-10-21 12:08:20 +01:00
Krzysztof Kotlarek
427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
OsamaSayegh
f364317625 PERF: Improve query speed when looking up direct PMs
Follow up to 5fc5a7f5ae
2019-07-23 03:52:52 +00:00
Osama Sayegh
5fc5a7f5ae FEATURE: Add search operator to see all direct messages from a user (#7913)
* FEATURE: Add search operator to see all direct messages from a user

* Only show message if related messages >= 5

* Make "all messages" the hyperlink

* Review
2019-07-22 10:55:49 -04:00
Blake Erickson
c76732722a FIX: Turn off search logging when read-only (#7877)
If `SiteSetting.log_search_queries` is enabled 500 errors will occur
when searching if the master db is down. This fix allows searching to
still work under these conditions.
2019-07-10 17:05:31 -07:00
Josh Moore
6c5689984f FEATURE: in:tagged search (srv side) (#7822)
* FEATURE: in:tagged and in:untagged advanced search filters

Similar to in:solved or in:unsolved, the filters check for an
existence of the topic_id in the topic_tags table.

see: https://meta.discourse.org/t/how-to-search-filter-untagged-topics/119641/2
2019-06-28 18:19:57 +10:00
Sam Saffron
8f7a387aa7 FEATURE: add support for tag group search
The behaviour of #TERM in search has been amended

1. We try category or subcategory slugs
2. We try tags
3. We try tag-groups

The term `hello #my-group` will search for all posts tagged with any of
the tags in the tag group `My Group`

Future work may be introducing a slug cache here or caching it in the table
but the assumption is that the number of tag groups will not be huge
2019-06-27 17:53:26 +10:00
Penar Musaraj
f51f37eddf FEATURE: apply a small penalty to closed topics when searching (#7782) 2019-06-21 12:03:45 +10:00
Dan Ungureanu
6bd082feab
FIX: Update mapping between locales and Postgres dictionaries. (#7606) 2019-05-27 16:52:09 +03:00
Sam Saffron
30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Sam Saffron
e2bcf55077 DEV: move send => public_send in lib folder
This handles most of the cases in `lib` where we were using send instead
of public_send
2019-05-07 12:25:44 +10:00
Vinoth Kannan
7869a10d18 Revert "FEATURE: Added unlisted topics option to advanced search (#7447)"
This reverts commit 539723f8ff since it is failing the build.
2019-05-01 21:06:20 +05:30
Tim Lange
539723f8ff FEATURE: Added unlisted topics option to advanced search (#7447) 2019-05-01 12:31:13 +10:00
Guo Xiang Tan
e8a4d72281 FIX: Avoid penalizing long documents too much in search.
This is a follow up to e87ca59401.
2019-04-03 14:09:57 +08:00
Guo Xiang Tan
4b0ac91bfb DEV: Remove duplicated scope.
`.joins(:topic)` will automatically add the `deleted_at IS NULL` scope.
2019-04-02 10:48:17 +08:00
Guo Xiang Tan
d8704c11ca PERF: Better use of index when queueing a topci for search reindex.
Also move `Search::INDEX_VERSION` to `SearchIndexer` which is where the
version is actually being used.
2019-04-02 09:53:37 +08:00
Guo Xiang Tan
e87ca59401 FIX: Relevance search will now consider document length in ranking.
The default ranking options ranks by the number of matches which is
highly problematic when posts are stuffed with a keyword. The ranking
will now be divided by the document length which is a much fairer way to
rank.
2019-04-01 14:37:45 +08:00
Guo Xiang Tan
dae0bb4c67 FIX: Post blurb incorrect when search contains a phrase match.
If the blurb generated is not around the search term, we will not be
able to highlight it on the client side.
2019-03-26 17:01:52 +08:00
Guo Xiang Tan
ac661e856a
FEATURE: Allow categories to be prioritized/deprioritized in search. (#7209) 2019-03-25 10:59:55 +08:00
Guo Xiang Tan
54d3648c55 FIX: Use same weights for calculating rank and searching for posts.
* Reduce an extra db query as well when searching for posts ordered by
relevance.
2019-03-20 15:36:31 +08:00
Guo Xiang Tan
64f20e7e7a FIX: Don't ignore category in search when using category filters. 2019-03-19 11:23:14 +08:00
Guo Xiang Tan
5e410dc5e0
FEATURE: Ability to exclude category from search results. (#7194)
This commit also adds `Category#search_priority` which sets the ground
work to enable prioritizing of posts for certain categories when searching.
2019-03-18 15:25:45 +08:00
Vinoth Kannan
daf5a268a7 DEV: Option to preload topic custom fields in Search class 2019-03-17 23:16:09 +05:30
Guo Xiang Tan
684eef71c7 REFACTOR: Better variable name. 2019-03-13 15:23:01 +08:00
Guo Xiang Tan
da941840d4 FIX: Advanced search category term should be case insensitive. 2019-03-12 14:11:21 +08:00
Joffrey JAFFEUX
3acf8a95f3
UX: various tweaks to search-menu (#7114) 2019-03-08 09:23:44 +01:00
Joffrey JAFFEUX
dc4001370c
FEATURE: displays groups in menu search (#7090) 2019-03-04 10:30:09 +01:00
Sam
0a357299b7 FEATURE: add f and t search shortcuts for first post / title
Previously with had `in:title` and `in:first` search shortcuts for
searching in first post or title only. They are a bit of handful to type.

This add 2 shortcuts (t and f) for searching titles of first posts.

This commit also cleans up all advanced filters, they were not properly
regex terminated allowing for weird clauses like `in:firstinator` acting
the same as `in:first`
2019-02-25 10:55:24 +11:00
Bianca Nenciu
4f3ee86bbd FIX: in:title should work irrespective of the order. (#6968) 2019-02-05 10:54:52 +01:00
Arpit Jalan
a121d40771
FIX: do not show PM topics when moving posts to an existing public topic (#6876) 2019-01-14 15:00:45 +05:30
Gerhard Schlager
bf27aecce2 REFACTOR: compact! works since the array can't contain empty strings 2018-11-22 13:27:34 +01:00
Gerhard Schlager
c376670bd2 FIX: a search term containing '& could lead to errors
This also makes sure that the search term in front or after special characters isn't ignored.
2018-11-21 22:07:56 +01:00
Sam
06b9d8223a FIX: search within topic not working correctly in CJK
We were splitting the term prior to search causing everything to miss
2018-11-07 09:41:55 +11:00
Daniel Hollas
cee51672c9 FIX: Strip accents from search query
4481836 introduced accent stipping in search_indexer,
but we need to strip it from the query itself as well

TODO in search with diacritics:
 - Still need to fix excerpts on search page
 - need to support accent stripping in in_topic search
 - need to make sure that in:title works correctly
 - need to fix "word boldening" in titles
2018-10-23 12:10:33 +11:00
David Taylor
9bf522f227
FEATURE: Mixed case tagging (#6454)
- By default, behaviour is not changed: tags are made lowercase upon creation and edit.

- If force_lowercase_tags is disabled, then mixed case tags are allowed.

- Tags must remain case-insensitively unique. This is enforced by ActiveRecord and Postgres.

- A migration is added to provide a `UNIQUE` index on `lower(name)`. Migration includes a safety to correct any current tags that do not meet the criteria.

- A `where_name` scope is added to `models/tag.rb`, to allow easy case-insensitive lookups. This is used instead of `Tag.where(name: "blah")`.

- URLs remain lowercase. Mixed case URLs are functional, but have the lowercase equivalent as the canonical.
2018-10-05 10:23:52 +01:00
Penar Musaraj
70d74f8fc1 FIX: advanced search ordering broken when using tags 2018-09-28 17:27:08 +08:00
Sam
9b7cab589a FIX: revert diacritic stripping
See more details in test case and at: https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/16?u=sam
2018-08-31 11:46:55 +10:00
Maja Komel
020eba4623 FIX: find tags with non-latin names (#6312) 2018-08-27 11:05:28 +10:00
Sam
ac11f8df52 correct regression searching with diacritics 2018-08-24 10:00:51 +10:00
Régis Hanol
0cd9e2acb9 fix build 2018-08-04 01:56:26 +02:00
Kasia Bułat
b71cf6d422 FEATURE: Add search not operator for tags. 2018-07-03 15:57:34 +08:00
Sam
5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
Guo Xiang Tan
ad5082d969 Make rubocop happy again. 2018-06-07 13:28:18 +08:00
Sam
e501936405 FIX: search server side error in rare condition 2018-05-28 15:28:18 +10:00
Sam
c677877e4f FIX: Korean needs no word segmentation 2018-05-28 09:37:57 +10:00
Sam
858a266031 FIX: exact matching should also match on title 2018-05-08 15:59:03 +10:00
Régis Hanol
a98aae3bcd FIX: topic search wasn't working for unlisted topics 2018-05-07 11:43:55 +02:00
Guo Xiang Tan
0d74c30fa7 Remove more uses of rescue nil. 2018-05-07 11:38:23 +08:00