* MM-34002: Improve AddUserToChannel
When we would add a user to a channel, we would
check whether the user is removed from that team or not.
During LDAP sync, this check is not required because the
team member would have just been created. Hence, we
pass a boolean flag to bypass the check.
And with that done, we can freely query the replica.
https://mattermost.atlassian.net/browse/MM-34002
```release-note
NONE
```
* Refactor code
* Rename a struct field
* fix double negative
During LDAP sync, we would call AddTeamMember which had a read-after-write issue
where we would create a team member but then immediately after that
query the team member.
The same pattern was found in:
AddTeamMember
AddTeamMembers
AddTeamMemberByToken
To fix this, we just return the inserted team member from AddUserToTeam and use that
instead of query GetTeamMember again.
```release-note
NONE
```
https://mattermost.atlassian.net/browse/MM-33913
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-31094: Adds tooling to develop and test using a MySQL instance with replication lag. Adds some lazy lookups to fallback to master if results are not found.
* MM-31094: Removes mysql-read-replica from default docker services.
* MM-31094: Switches (store..SessionStore).Get and (store.TeamStore).GetMember to using context.Context.
* MM-31094: Updates (store.UsersStore).Get to use context.
* MM-31094: Updates (store.PostStore).Get to use context.
* MM-31094: Removes feature flag and config setting.
* MM-31094: Rolls back some master reads.
* MM-31094: Rolls a non-cache read.
* MM-31094: Removes feature flag from the store.
* MM-31094: Removes unused constant and struct field.
* MM-31094: Removes some old feature flag references.
* MM-31094: Fixes some tests.
* MM-31094: App layers fix.
* MM-31094: Fixes mocks.
* MM-31094: Don't reparse flag.
* MM-31094: No reparse.
* MM-31094: Removed unused FeatureFlags field.
* MM-31094: Removes unnecessary feature flags variable declarations.
* MM-31094: Fixes copy-paste error.
* MM-31094: Fixes logical error.
* MM-30194: Removes test method from store.
* Revert "MM-30194: Removes test method from store."
This reverts commit d5a6e8529b.
* MM-31094: Conforming to make's strange syntax.
* MM-31094: Configures helper for read replica with option.
* MM-31094: Adds some missing ctx's.
* MM-31094: WIP
* MM-31094: Updates test names.
* MM-31094: WIP
* MM-31094: Removes unnecessary master reads.
* MM-31094: ID case changes out of scope.
* MM-31094: Removes unused context.
* MM-31094: Switches to a helper. Removes some var naming changes. Fixes a merge error.
* MM-31094: Removes SQLITE db driver ref.
* MM-31094: Layer generate fix.
* MM-31094: Removes unnecessary changes.
* MM-31094: Moves test method.
* MM-31094: Re-add previous fix.
* MM-31094: Removes make command for dev.
* MM-31094: Fix for login.
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w`
* added goimports lint check to .golangci.yml
* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w` for a corner case
* make app-layers, *-mocks and store-layers for ci check
Co-authored-by: Mahmudul Haque <mahmudulhaque@protonmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Installing plugins specified by feature flags using the marketplace.
* Switch back to using getplugins client.
* Respect disabling automatic installation of pluings.
* pluginid -> plugin_id
* Debug logs for enable plugin error
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Removing supplier concept from the sql store
* Removing other metions to supplier
* Fixing gofmt
* Fixing gofmt
* Renaming NewSqlStore to New
* Fixing tests
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
The code was doing `go fakeApp.UpdateProductNotices()`
due to which the Store object was accessed in a racy manner
when it was reset if the localcachelayer flag was set.
We fix this by using the StoreOverride functionality
to pre-define what store to use while initializing the server
itself so that there isn't a need to reset the store _after_
the server is set.
We also apply the same fix in the app and api4 layers too
to prevent similar things from happening there.
https://mattermost.atlassian.net/browse/MM-29487
```release-note
NONE
```
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-28067: Optimize app server startup in tests
For every test, we would wipe out the database and then start
the server again. This would run through all the migrations
and create new rows in Systems and Roles table every time.
As a result, this was consuming a lot of setup time for every test.
We optimize this by preloading the DB with dummy data in the roles
and systems table so that the server code just skips over those migrations.
This is completely forward compatible and adding new migrations does not
need to generate the sql files again. Only in case of schema changes to the
roles or systems table, this would need to be done.
It is unlikely the Systems table schema will get changed. The Roles table
might change in future but it's a comparatively rare event. Given the reduction
in CI runtime we are seeing, it's a worthy optimization.
We also apply some more optimizations:
- Coalesce multiple UpdateConfig calls into a single one. Each UpdateConfig call
has to do a json marshal which would take precious CPU cycles. It's much more
efficient to do everything in a single call.
- Remove unnecessary debug.FreeOSMemory in reload config. This was an artifact
from old days and is no longer required.
Numbers:
Results show a full **2 minutes** shaved off the test runtime. Earlier, tests
would take around 16 minutes. Now they take 14 minutes.
```release-note
NONE
```
* fix app package
* MM-30041: Return correct error message for user save
We were collapsing all types of user conflict into a single
error message. Fixed it by inspecting the field of the invalidError type
and returning the correct message.
https://mattermost.atlassian.net/browse/MM-30041
```release-note
NONE
```
* Fix test errors
* Fix wrong message in test when comparing error messages
Co-authored-by: Rodrigo Villablanca <villa061004@gmail.com>
* MM-27149: optimize initBasic
Mostly, all tests just needed the user initialization part and not
the channel and group creation. So we move the user initialization inside
the Setup call. This avoids unnecessary DB calls which take around 250-300ms
on average.
And we make the login requests concurrently to shave off a few more ms.
According to my tests, the 2 login calls take 140 ms on average, which
shaves off 70ms.
So approximately, we shave off 350ms per test. And there are 114 occurences
of these. So around 39 seconds.
* make initlogin only for Setup/SetupEnterprise
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Removing some other fake apps
* More FakeApp removed
* Removing entirely FakeApp
* Fixing some tests
* Fixing get Cluster id from get plugin status
* Fixing failing tests
* Fixing tests
* Fixing test initialization for web
* Fixing InitServer for server tests
* Fixing InitServer for server tests
* Reverting go.sum and go.mod
* Removing unneded HTMLTemplates function in App layer
* Moving back some functions to its old place to easy the review
* Moving back some functions to its old place to easy the review
* Using the last struct2interface version
* Generating store layers
* Fixing merge problems
* Addressing PR comments
* Small fix
* Fixing app tests build
* Fixing tests
* fixing tests
* Fix tests
* Fixing tests
* Fixing tests
* Fixing tests
* Moving license to server struct
* Adding some fixes to the test compilation
* Fixing cluster and some jobs initialization
* Fixing some license tests compilation problems
* Fixing recursive cache invalidation
* Regenerating app layers
* Fix test compilation
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* make Group.Name a pointer to allow null
* fix unit tests
* fix build error
* fix unit test
* ensure Name field not nil
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* Convert bool string comparisons to strconv.ParseBool for REST parameters
* Log failed bool conversions
* Rename errors, changed log levels
* drop strconv.ParseBool error handling
If the query string parameter is omitted, strconv.ParseBool returns an error for the empty strings, which spams the logs. Instead, just assume the default semantics of a `false` return value if an error occurs.
* allow randomized Client4 booleans
It's hard to test api4's handling of the various boolean input values
accepted. Extend Client4 with support for overriding how it builds those
strings, and pick a random value on test startup.
* gofmt -s
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
* [MM-24146] Add unix socket listener for mmctl local mode (#14296)
* add unix socket listener for mmctl local mode
* add a constant for local-mode socket path
* reflect review comments
* [MM-24401] Base approach for Local Mode (#14333)
* add unix socket listener for mmctl local mode
* First working PoC
* Adds the channel list endpoint
* Add team list endpoint
* Add a LocalClient to the api test helper and start local mode
* Add helper to test with both SystemAdmin and Local clients
* Add some docs
* Adds TestForAllClients test helper
* Incorporating @ashishbhate's proposal for adding test names to the helpers
* Fix init errors after merge
* Adds create channel tests
* Always init local mode to allow for enabling-disabling it via config
* Check the RemoteAddr of the request before marking session as local
* Mark the request as errored if it's local and the origin is remote
* Set the socket permissions to read/write when initialising
* Fix linter
* Replace RemoteAddr check to ditch connections with the IP:PORT shape
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Fix translations order
* [MM-24832] Migrate plugin endpoints to local mode (#14543)
* [MM-24832] Migrate plugin endpoints to local mode
* Fix client reference in helper
* [MM-24776] Migrate config endpoints to local mode (#14544)
* [MM-24776] Migrate get config endpoint to local mode
* [MM-24777] Migrate update config endpoint to local mode
* Fix update config to bypass RestrictSystemAdmin flag
* Add patchConfig endpoint
* MM-24774/MM-24755: local mode for addLicense and removeLicense (#14491)
Automatic Merge
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Ashish Bhate <bhate.ashish@gmail.com>
* WIP
* Adding bleve to go modules
* WIP
* Adding missing files from searchengine implementation
* WIP
* WIP
* WIP
* WIP
* WIP
* WIP
* User and channel indexing and searches implemented
* Make bleve tests run with in-memory indexes
* Implement post index and deletion tests
* Initial commits for the search layer
* Removing unnecesary indexing
* WIP
* WIP
* More fixes for tests
* Adding the search layer
* Finishing the migration of searchers to the layer
* Removing unnecesary code
* Allowing multiple engines active at the same time
* WIP
* Add simple post search
* Print information when using bleve
* Adding some debugging to understand better how the searches are working
* Making more dynamic config of search engines
* Add post search basics
* Adding the Purge API endpoint
* Fixing bleve config updates
* Adding missed file
* Regenerating search engine mocks
* Adding missed v5 to modules imports
* fixing i18n
* Fixing some test around search engine
* Removing all bleve traces
* Cleaning up the vendors directory and go.mod/go.sum files
* Regenerating timer layer
* Adding properly the license
* Fixing govet shadow error
* Fixing some tests
* Fixing TestSearchPostsFromUser
* Fixing another test
* Fixing more tests
* Fixing more tests
* Removing SearchEngine redundant text from searchengine module code
* Fixing some reindexing problems in members updates
* Fixing tests
* Addressing PR comments
* Reverting go.mod and go.sum
* Addressing PR comments
* Fixing tests compilation
* Fixing govet
* Adding search engine stop method
* Being more explicit on where we use includeDeleted
* Adding GetSqlSupplier test helper method
* Mocking elasticsearch start function
* Fixing tests
* Search tests
* Fix tests
* Fix mod
* Fixing searchEngine for test helpers with store mocks
* Remove loglines
* Fix i18n strings
* Migrate search posts tests
* Fix linter
* Do not run search tests if -short flag is enabled
* Migrate back store tests that didn't belong to the searchlayer
* Fix scopelint issues
Co-authored-by: Jesús Espino <jespinog@gmail.com>
* Remove unncessary recover() calls
A recover which is not called inside a defer will always return nil.
So there's no use of calling it.
And even if we did call it inside a defer, recovering and immediately
panicking does not make sense either. So we just remove it.
* Fixed another instance
* MM-22205 Add get channel moderations endpoint
* MM-22206 Add patch channel moderations endpoint
Add api tests for patch channel moderations
* MM-22205 Ensure ordered permissions returned and create struct ChannelModeratedRoles
* MM-22206 Use structs instead of map
* MM-22206 Add test cases for GetChannelModeratedPermissions
* MM-22206 Add tests for ChannelModeratedPermissionsChangedByPatch
* MM-22206 Use NewBool instead of defining booleans
* MM-22206 Tie Channel Mentions to Create Posts when building Channel Moderations
* Revert "MM-22206 Tie Channel Mentions to Create Posts when building Channel Moderations"
This reverts commit a0bfc95f17.
* MM-22206 Review changes
Modify GetSchemeRolesForChannel to return named variables
Change calls to SessionHasPermissionToChannel to SessionHasPermissionTo
Add a CreateChannelScheme method
Add a DeleteChannelScheme method
Move GetChannelModeratedPermissions to Role model
* Fix lint
* Add ChannelModeration methods to App interface
* MM-22206 Rename method to GetTeamSchemeChannelRoles
* MM-22206 Check CHANNEL_MODERATED_PERMISSIONS_MAP for existing permission before iterating through it
* Modify wording to higherScoped to match #13813
* MM-22206 Delete channel scheme between tests
* MM-22206 Fix tests
* Actually patch role
* MM-22206 Shadow declaration of err
* Cache test users to speed up testing
Users were recreated for many tests and hashing passwords takes a lot of
time. Testing /api4 takes around 120s less and /app 60s less.
* fix build
* simplifying a bit the approach
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Jesús Espino <jespinog@gmail.com>
* Introducing unit (not integration) tests for the app layer
* Initial support for unit tests at the API
* Adding unit tests support to the store layer
* Add unit tests support in commands
* Adding last tests needed for run unit tests properly
* Fixing govet
* Removing some duplication
* Fixing tests
* Fixing tests
* Not compiling test helpers with the main module for api
* Revert "Not compiling test helpers with the main module for api"
This reverts commit 36a199bbe0.
* Fixing tests
* Fixing unit tests
* More consistency between api4/apiteslib.go and app/helper_test.go
* Renaming things to make more obvious the new Setup functions purpose
* Reverting change in go.sum
* Start with empty mock for app layer
* Start with empty mock for api layer
* Start with empty mock for web layer
* Renaming SetupWithStoreMockConfig to SetupConfigWithStoreMock
* Fixing tests on web package
* Removing unnecesary function
* MM-21793: Allow bots to be removed and added from group synced teams and channels
* MM-21793: Add tests for adding and removing a team and channel members
* MM-21793 Add punctuation to comments and remove unnecessary variable
* Consistent license message for all the go files
* Fixing the last set of unconsistencies with the license headers
* Addressing PR review comments
* Fixing busy.go and busy_test.go license header
* MM-8607: add ability to turn off non-critical services under load
* server busy invalid param unit tests
* MM-8607: rename server busy endpoints
* MM-8607: handle case where App not initialized
* MM-8607: additional unit test cases per feedback.
* MM-8607: use decorator to check isbusy when adding endpoint route
* MM-8607: rename endpoints, use struct for json
* Update api4/system.go
Fix misspelled log output
Co-Authored-By: Saturnino Abril <saturnino.abril@gmail.com>
* MM-8607: fix i18n order; max seconds for server busy expiry
* migrate tests to use testify
* check resp error
* require empty auth
* fix based on comment
* change to require
* change all assert to require
* remove line
