Setting a `nil` value should actually delete the row instead of preserve it with a `nil` value.
The existing tests passed because they only checked the return value of `nil`. When running the tests on Postgres, this value is returned as an empty `[]byte{}` instead of `nil`, so update the test to reflect this semantically equivalent behaviour.
* MM-21946 Ensure deleted groups are not returned from the groups API
* MM-21946 Only append to query at the end
* MM-21946 Add DeleteAt check to the top of the GetGroups function and clean up the formatting a bit
* MM-21946 Move the From statement into the next block
* MM-20644: Add users to teams as a SchemeAdmin based on a new configuration field on GroupTeams and GroupChannels records.
* MM-20644: Adds SchemeAdmin to mapping of the GroupSyncable struct fields.
* MM-2064: Adds test to ensure SchemeAdmin field value is mapped.
* MM-20644: Adds missing index creation for fresh DBs.
* MM-20644: Duplicates UpdateMembersRole across Team and Channel stores. Adds tests.
* MM-20644: Fixes some old method name references.
* MM-20644: Moves variable declaration; removes Println statement.
* MM-21157: Triggers role sync when a syncable is linked, unlinked, or updated.
* MM-20644: Use a SQL query instead of two to update Team and Channel members.
* MM-20644: Fixes tests; updates query.
* MM-21157: Removes second invocation of function because of refactor that performs it all in a single query.
* MM-21157: Switches a few queries to Squirrel.
* MM-21157: SQL-formats some strings.
* MM-21157: Select with list.
* MM-21157: Converts some more sql to squirrel.
* MM-21157: Fix incorrect conflict resolutions.
* MM-21157: Fix incorrect conflict resolutions.
* MM-21157: Adds missing mocks.
* MM-21157: Clears cache upon syncing roles.
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
`KVCompareAndSet(key, sameValue, sameValue)` can fail spuriously on MySQL if the underlying `UPDATE` requires no actual changes. As per the [MySQL documentation](https://dev.mysql.com/doc/refman/8.0/en/mysql-affected-rows.html), we can't rely on rows affected in this case:
> For UPDATE statements, the affected-rows value by default is the number of rows actually changed. If you specify the CLIENT_FOUND_ROWS flag to mysql_real_connect() when connecting to mysqld, the affected-rows value is the number of rows “found”; that is, matched by the WHERE clause.
It's not tenable to change `CLIENT_FOUND_ROWS` for the all connection, so handle this case in the code instead by running a `SELECT` after the fact. Note that `KVCompareAndSet` has no guarantee of atomicity in this case, but neither would `CompareAndSwap` on which this is method was inspired.
Finally, note that no changes are required for Postgres, which has sane semantics as the default.
Fixes: https://mattermost.atlassian.net/browse/MM-21328
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-20644: Add users to teams as a SchemeAdmin based on a new configuration field on GroupTeams and GroupChannels records.
* MM-20644: Adds SchemeAdmin to mapping of the GroupSyncable struct fields.
* MM-2064: Adds test to ensure SchemeAdmin field value is mapped.
* MM-20644: Adds missing index creation for fresh DBs.
* MM-20644: Duplicates UpdateMembersRole across Team and Channel stores. Adds tests.
* MM-20644: Fixes some old method name references.
* MM-20644: Moves variable declaration; removes Println statement.
* MM-20644: Use a SQL query instead of two to update Team and Channel members.
* MM-20644: Fixes tests; updates query.
* MM-20644: Fix permission check for patching a group syncable.
* MM-20644: Fixes test for change of permissions verification in group patch API request.
* MM-20644: Fix for ORM select vs insert.
* MM-20644: Linting fixes.
* MM-20644: Fixes some tests.
* MM-20644: Skips changing the role of guests.
* MM-20644: Added syncableID filtering
* MM-21211: Optimize Channelstore.UpdateLastViewedAt
For postgres, we use CTEs to do both queries at one
and return the data.
For mysql, CTEs aren't available until version 8. So we just optimize the
existing query.
- We use IN instead of multiple OR conditions for the channel membership check.
According to mysql https://dev.mysql.com/doc/refman/5.7/en/comparison-operators.html#operator_in:
If no type conversion is needed for the values in the IN() list,
they are all constants of the same type, and expr can be compared to each of them
as a value of the same type (possibly after type conversion), an optimization takes place.
The values the list are sorted and the search for expr is done using a binary search,
which makes the IN() operation very quick.
- We use the previous column LastViewedAt to replace the value of LastUpdateAt
instead of doing a switch case again.
https://dev.mysql.com/doc/refman/5.6/en/ansi-diff-update.html
If you access a column from the table to be updated in an expression, UPDATE uses the
current value of the column.
Results (ran with getchannel profile):
Postgres:
| Metric | Baseline | Actual | Delta | Delta % |
| --- | --- | --- | --- | --- |
| Hits | 1594 | 1574 | -20 | -1.25%
| Error Rate | 0.00% | 0.00% | 0% | 0% |
| Mean Response Time | 7.08ms | 6.36ms | -0.72ms | -10.20% |
| Median Response Time | 5.00ms | 4.00ms | -1.00ms | -20.00% |
| 95th Percentile | 9.00ms | 8.00ms | -1.00ms | -11.11% |
Mysql:
| Metric | Baseline | Actual | Delta | Delta % |
| --- | --- | --- | --- | --- |
| Hits | 1600 | 1566 | -34 | -2.12%
| Error Rate | 0.00% | 0.00% | 0% | 0% |
| Mean Response Time | 7.39ms | 7.02ms | -0.37ms | -4.94% |
| Median Response Time | 6.00ms | 6.00ms | 0ms | 0% |
| 95th Percentile | 10.00ms | 10.00ms | 0ms | 0% |
* Add comment on using CTEs for mysql8
* incorporate review comments
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* Refactor to use structured logging
* Properly formatted with gofmt
* created interface Cache, but construction of cache is still coupled.
* Implementing cache factories to build caches
* Simple redis implementation without error handling. Keys and values by default are string
* refactor NewLocalCacheLayer to inject cache factory
* Removed redis impl to focus on cache abstraction
* CacheFactory injected on sqlsupplier and saved in Store struct
* remove useless private method
* replace concrete declaration of lru cache to cache abstraction
* discard spaces
* Renamed factory to provider because concrete implementations of factories may hold an state (such as a redis client for example)
* refactor to include all caches in the same package cache and subpackages
* method close cache. This method will be used for concrete implementations that need to release resources (close a connection, etc)
* closing cacheprovider and releasing resources while closing sql store
* fixed merge conflict fail
* gofmt files
* remove unused property from post_store
* naming refactor to avoid stutter. Added godocs on interface
* Store doesnt know anything about the cache and provider. Cache provider will be built after loading config and injected in localCacheLayer
* fixed broken test
* cache provider initialized before RunOldAppInitialization which initializes the localcachelayer
* move statusCache to server to initialize it with the new cache provider
* update terms_service and channel_layer to have new cacheProvider
* gofmt
* Add Connect method to the cache provider
* mock cacheprovider in user_layer_test
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
Display LHS bot Icon in web app. As part of mentioned task, Added LastIconUpdate variable in model.bot to store last update time of icon. Also added code to update/delete value of the mentioned variable when setting/deleting bot icon.
* migrated termsOfServiceCache from sqlstore to localcachelayer, and mocked store tests
* revert gitignore local change
* fixed caching in termsOfService Get, and added basic tests for termsOfServiceCache
* added a test for cache save, and fixed call to Store tests
* fixed GeLatest termsOfService from cache test
* added license headers to terms_of_service cache files
* using doStandardReadFromCache and doStandardAddToCache when reading and writing to cache
* removed unused variable, termsOfServiceCacheName
* added special key for the latest termsOfService value in termsOfServiceCache
* updated license information on localcachelayer termsOfServiceCache files
* fixed not updating latest termsOfServiceCache on Get by ID, and invalidating cache cluster on termsOfServiceCache save
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
The `channelByNameCache` in the `GetByNames` query checks each teamId+channelName
combination in the cache and if any of the keys is not present, it queries the DB.
In this case, this is effectively a cache miss from the point of view of user.
But since the cache is at a teamId+channelName level, there may be several hits and some misses.
This is misleading as it does not generate proper metrics and might lead to a
false assumption of how effective the cache is.
We create separate metrics which account for all or none cache hits/misses.
This removes the old metrics which are used by other queries. So they are now separated,
and it will improve both the old and new metric. Because the old metric is not contaminated,
and the new metric is also separate.
* Migrate channelCache cache from store/sqlstore/channel_store.go to the new store/localcachelayer
* fix import for new path
* remove unnecessary code, refactor cache constant
* resolve conflicts
* fix missing variable and import after merging
* refactor code, fix current issue
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-14675: Upserts GroupTeam when GroupChannel is linked.
MM-14675: Delete all associated GroupChannels when deleting a GroupTeam.
MM-14675: Using replica DB where possible.
MM-14675: Updates create method used in tests.
* MM-14675: Removes unnecessary DB retrieval of GroupSyncable record.
* MM-21118: Use querybuilder instead of fmt.Sprintf in channelStore
We use the querybuilder API instead of just building the query string
manually using fmt.Sprintf
* Replaced comma with colon
* Add MLOG for sql store
* Update printf function
* Update store/sqlstore/supplier.go
Co-Authored-By: Martin Kraft <martin@upspin.org>
* Update go mod
* update go mod and tidy it up
* update vendor folder
* MM-21116: Improve channelSearch SQL query generation
The likeTerm from buildFulltextClause was being built inefficient using strings.
We change it to use squirrel to build the query.
And while at it, we also change the len check with checking with
an empty string which is more idiomatic. Both compile to the same
code, so there is no difference performance wise.
* Use a better variable name
* Consistent license message for all the go files
* Fixing the last set of unconsistencies with the license headers
* Addressing PR review comments
* Fixing busy.go and busy_test.go license header
* Migrate channelPinnedPostCountsCache cache from store/sqlstore/channel_store.go to the new store/localcachelayer
* Remove GetPinnedPostCountFromCache
* Rearrange test to avoid undeclared variable
