From Grafana charts, GetCountbyStatusandType and
GetNewestJobByStatusesAndType were the two top queries.
Overall, a through look into all job methods leads to
the conclusion of 2 indexes - one on CreateAt, another
a compound index of Status+Type. I have just gone ahead
with the compound index for now. Once the job cleaner
is implemented, I want to take a second look to decide
whether to add the second index or not.
Here is the before-after of the queries:
Query 1:
```
explain analyze select count(*) from jobs where status='error' and type='migrations';
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------
Aggregate (cost=103.42..103.43 rows=1 width=8) (actual time=0.032..0.034 rows=1 loops=1)
-> Bitmap Heap Scan on jobs (cost=4.54..103.42 rows=1 width=0) (actual time=0.027..0.028 rows=0 loops=1)
Recheck Cond: ((type)::text = 'migrations'::text)
Filter: ((status)::text = 'error'::text)
-> Bitmap Index Scan on idx_jobs_type (cost=0.00..4.54 rows=34 width=0) (actual time=0.018..0.019 rows=0 loops=1)
Index Cond: ((type)::text = 'migrations'::text)
explain analyze select count(*) from jobs where status='error' and type='migrations';
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------
Aggregate (cost=8.31..8.32 rows=1 width=8) (actual time=0.079..0.080 rows=1 loops=1)
-> Index Only Scan using jobs_multi on jobs (cost=0.29..8.30 rows=1 width=0) (actual time=0.072..0.073 rows=0 loops=1)
Index Cond: ((status = 'error'::text) AND (type = 'migrations'::text))
Heap Fetches: 0
explain analyze select * from jobs where status='error' and type='migrations' order by createat desc limit 1;
```
Query 2:
```
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------
Limit (cost=8.31..8.32 rows=1 width=187) (actual time=0.037..0.039 rows=0 loops=1)
-> Sort (cost=8.31..8.32 rows=1 width=187) (actual time=0.035..0.036 rows=0 loops=1)
Sort Key: createat DESC
Sort Method: quicksort Memory: 25kB
-> Index Scan using idx_jobs_type on jobs (cost=0.29..8.30 rows=1 width=187) (actual time=0.027..0.027 rows=0 loops=1)
Index Cond: ((type)::text = 'migrations'::text)
Filter: ((status)::text = 'error'::text)
explain analyze select * from jobs where status='error' and type='migrations' order by createat desc limit 1;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------
Limit (cost=8.31..8.32 rows=1 width=187) (actual time=0.065..0.067 rows=0 loops=1)
-> Sort (cost=8.31..8.32 rows=1 width=187) (actual time=0.063..0.064 rows=0 loops=1)
Sort Key: createat DESC
Sort Method: quicksort Memory: 25kB
-> Index Scan using jobs_multi on jobs (cost=0.29..8.30 rows=1 width=187) (actual time=0.021..0.022 rows=0 loops=1)
Index Cond: (((status)::text = 'error'::text) AND ((type)::text = 'migrations'::text))
```
```release-note
NONE
```
* Migrate from gorp to sqlx in store/sqlstore/command_webhook_store.go
* fixing small things
change `CreatedAt` to `CreateAt` and change GetReplicaX().Select to GetReplicaX().Get
Co-authored-by: Saul Pinales <saul021002@gmai.com>
* Add API endpoint and adapt search to allow multi-team search
* Refactor handler, refactor sql query to use squirrel, rename app and store functions and add tests
* Fix lint
* Fix search engines and remove unneeded comments
* Fix test
* Remove user from channel after test
Our store upgrade methods used mlog.Critical and then slept
for a second and triggered os.Exit.
First of all, with the new mlog refactor Critical wasn't a standard
log level so it was never to be logged.
And secondly, this completely missed the fact that there already
existed an API method Fatal which did exactly this.
We mark the Critical API as deprecated, and use Fatal for all
occurences.
The choice should be between either to use Error or Fatal.
While here, we also remove the non-standard exit codes which
just confused things further while debugging.
https://mattermost.atlassian.net/browse/MM-38712
```release-note
NONE
```
* MM-35017: order and inclusion of participants
From now on root poster is not considered a thread participant,
only repliers are participants.
Ordering: A second reply to the thread from the same user should place
the user last at the participants list.
* Fixes shadow
* Fixes tests
* Order By CreateAt for participants of old threads
* Removes unnecessary check on root id
* Removes GetParticipantProfilesByIds it's not needed
At first GetParticipantProfilesByIds was created because GetProfileByIds
gets users by order of username.
This was not needed, we can order by user ids in Go.
This commit removes GetParticipantProfilesByIds and replaces it by
GetProfileByIds as it was done before.
* Fixes participants order for pre CRT threads
Older threads (before CRT) are being converted to CRT threads upon
reply.
This commit considers the order of participants to be inserted to
the new thread.
* Adds tests for thread participants
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
We properly truncate the date part from the createAt time.
The SQL query is improved now to directly return the bool.
Fixed the test to properly test the feature.
https://mattermost.atlassian.net/browse/MM-38132
```release-note
NONE
```
* Added a sqlXPassThrough layer
This layer embeds the original sqlx object and wraps some
key methods to avoid some boilerplate.
- Add timeout to all methods from the timeout in settings.
- Added trace functionality.
- Added auto-rebind capabilities so that each store method
doesn't need to call rebind every time.
- Added auto-lowering of query string to support NamedExec
for both MySQL and Postgres
All methods are not wrapped as all are likely not to be called
from our code. As and when they are called, support will be added.
```release-note
NONE
```
* fix lint issues
```release-note
NONE
```
* address review comments
```release-note
NONE
```
* incorporate suggestions
```release-note
NONE
```
* Prepare support for sqlx
Just setting up some basic skeleton to start
replacing gorp queries with sqlx.
There is still an issue with efficiently writing
cross-platform named queries, because :FieldName
needs to be :fieldname for Postgres.
The positional params can be rebound depending on
driver name. But named queries can't from my
investigation. Will look into this.
```release-note
NONE
```
* Refactor mapper
```release-note
NONE
```
* Forgot to init slice
```release-note
NONE
```
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* CRT: desktop thread notifications
* Fixes go lint
* Adds default for desktop CRT notifications
* Adds email and push notifications for CRT threads
Adds user ids of thread followers with CRT to crtMentions so they will get
notified appropriately.
* Minor change
* Refactor a bit
CRTMentions.addMention had a bug on the return and de-duplication.
This commit fixes duplicate notifications by looking up if the user is to be
notified on CRT on both email and push notifications.
* Minor refactor
* Changes according to review comments
- Fixes adding to followers a user that had explicitly unfollowed a
thread.
- Simplified send email according to email_threads option
- Send mentions and followers in separate arrays via the websocket
- Fixes push notifications message for push_threads
* Adds a comment on a buggy use case
* Updates comment to correct ticket link
* Fixes when user notifications is set to all
There was a bug where if user had set notifications to all
then they would receive desktop notifications even for non following threads.
A similar bug existed in push notifications, where if a user has set it
to all the threads setting would still be considered.
This commit fixes that by adding users to notificationsForCRT
StringArray when they have the non thread setting to 'all'.
* Fixes notifications to users unfollowing threads
Users which had previously explicitly unfollowed a thread
should not receive notifications about those threads.
* Update store mocks
* Fixes push notifications for CRT
Push notification about replies for CRT users should have a title of
"Reply to Thread".
CRT users with global user setting to 'UserNotifyAll' should not get
notifications for unfollowed threads.
This commit fixes those issues.
* Fixes i18n error
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
- We move logging statements to the upper layer.
Store functions are low-level methods and should return error
upwards rather than logging.
- Used IN instead of any (array ()) which is equivalent.
- Made the delay to be of type time.Duration and un-exported it.
- Unexported the batch size constant.
```release-note
NONE
```
Summary
If a user has unfollowed a thread, another user's reply in the thread should not cause the first user to re-follow the thread. The first user will be re-followed if they are mentioned in the thread.
Simplify and make flexible the logic surrounding following and remove some duplicate code.
Ticket Link
https://mattermost.atlassian.net/browse/MM-36544https://mattermost.atlassian.net/browse/MM-37439
* MM-25516: Changed to byte slice instead of string for cluster messages
https://mattermost.atlassian.net/browse/MM-25116
Testing:
Manually tested.
Load-tested with Cluster Controller.
I looked into changing the serialization method to use msgpack,
but the ClusterMessage struct was mainly used for only 3 fields
which didn't lead to much of a CPU time improvement, whereas
actually led to more allocations using msgpack. Hence, I chose
to remain with JSON.
```
name old time/op new time/op delta
ClusterMarshal-8 3.51µs ± 1% 3.10µs ± 2% -11.59% (p=0.000 n=9+10)
name old alloc/op new alloc/op delta
ClusterMarshal-8 776B ± 0% 1000B ± 0% +28.87% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
ClusterMarshal-8 12.0 ± 0% 13.0 ± 0% +8.33% (p=0.000 n=10+10)
```
```release-note
Changed the field type of Data in model.ClusterMessage to []byte from string.
```
* Trigger CI
```release-note
NONE
```
