* MM-25404: Add GetHealthScore metric for the cluster
This PR adds the necessary function to the cluster interface
so that they can be called from App code.
* Add mocks
* Remove fakeapp
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* [MM-23535] Add disable database search flag and return empty posts results if set
* Add UpdateConfig function for the SearchStore and hook it into the app lifecycle
* Add the config listener in the server instance instead of using FakeApp
* Instantiate searchlayer as a pointer to avoid passing around copies of it
The current code path for `CreatePostAsUser` tries to update the `LastViewedAt` for bots, which logs a warning message if the bot isn't actually in the channel.
This pull request changes the semantics of bot posting to not update the bot's `LastViewedAt` timestamp, avoiding this log altogether. It matches the semantics of `from_webhook`, but notably makes bots slightly less like users in that they no longer "read" channels when they post. This seems reasonable, but I'm both looking for validation of this semantic change in addition to the code review.
Fixes: https://mattermost.atlassian.net/browse/MM-23926
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-22962 Create use_group_mentions permission and give to all non guest roles that can create post
* Add use_group_mentions to team admin role for test
* Trigger CI
* MM-22962 Remove old migration keys
* explicitly assert panic as error log
* Revert "[MM-18150] plugin panic trace should not be lost (#13559)"
This reverts commit 5d928b4f94, while leaving the unit tests intact
and now asserting debug logs instead.
* missing license header
* WIP
* Adding bleve to go modules
* WIP
* Adding missing files from searchengine implementation
* WIP
* WIP
* WIP
* WIP
* WIP
* WIP
* User and channel indexing and searches implemented
* Make bleve tests run with in-memory indexes
* Implement post index and deletion tests
* Initial commits for the search layer
* Removing unnecesary indexing
* WIP
* WIP
* More fixes for tests
* Adding the search layer
* Finishing the migration of searchers to the layer
* Removing unnecesary code
* Allowing multiple engines active at the same time
* WIP
* Add simple post search
* Print information when using bleve
* Adding some debugging to understand better how the searches are working
* Making more dynamic config of search engines
* Add post search basics
* Adding the Purge API endpoint
* Fixing bleve config updates
* Adding missed file
* Regenerating search engine mocks
* Adding missed v5 to modules imports
* fixing i18n
* Fixing some test around search engine
* Removing all bleve traces
* Cleaning up the vendors directory and go.mod/go.sum files
* Regenerating timer layer
* Adding properly the license
* Fixing govet shadow error
* Fixing some tests
* Fixing TestSearchPostsFromUser
* Fixing another test
* Fixing more tests
* Fixing more tests
* Removing SearchEngine redundant text from searchengine module code
* Fixing some reindexing problems in members updates
* Fixing tests
* Addressing PR comments
* Reverting go.mod and go.sum
* Addressing PR comments
* Fixing tests compilation
* Fixing govet
* Adding search engine stop method
* Being more explicit on where we use includeDeleted
* Adding GetSqlSupplier test helper method
* Mocking elasticsearch start function
* Fixing tests
* Search tests
* Fix tests
* Fix mod
* Fixing searchEngine for test helpers with store mocks
* Remove loglines
* Fix i18n strings
* Migrate search posts tests
* Fix linter
* Do not run search tests if -short flag is enabled
* Migrate back store tests that didn't belong to the searchlayer
* Fix scopelint issues
Co-authored-by: Jesús Espino <jespinog@gmail.com>
* MM-22212: Read non-moderated permissions from higher-scoped scheme.
* MM-2212: Corrects test count in comment.
* MM-22212: Adds godoc comment.
* MM-2212: Switches to the channel roles check in a few more places.
* MM-22212: Refactors and fixes.
* MM-22212: Reverts change, no longer required.
* MM-22212: Removes translation.
* MM-22212: Un-comments merged new permission.
* MM-22212: Un-comments merged new permission.
* MM-22212: Performance tweak.
* MM-22212: Fixes some fmting.
* MM-22212: Add unit test for newly-added store methods.
* MM-22212: Renames app method.
* MM-22212: Re-uses existing function to find string in slice.
* MM-22212: Keeps 'higher-scoped' terminology for consistency.
* MM-22212: Refactors based on PR feedback.
* MM-22212: Fix for some bad merging.
* MM-22212: Renamed some things.
* MM-22212: Use an 'else' instead of a 'continue' for readability.
* MM-22212: Caches (*SqlRoleStore).ChannelRolesUnderTeamRole.
* MM-22212: Adds mock to new cache store.
* MM-22212: Adds missing open tracing app layer methods.
* MM-22212: Adds migration to add moderated permissions to channel_admin if present on channel_user.
* MM-22212: Migrates team schemes. Removes unused AppError.
* MM-22212: Fix for for if.
* MM-22212: Fixes iterator.
* MM-22212: Updates open tracing generated methods.
* MM-22212: Fix mocks.
* MM-22212: Change migration key name.
* MM-22212: Switched to data structure from other branch.
* MM-22212: Fixes tests after adding 'use_channel_mentions' to the channel_admin role.
* MM-22212: Adds tracking of channel moderation.
* Revert "MM-22212: Adds tracking of channel moderation."
This reverts commit 23689efa22.
* MM-22212: Switch some functions to methods and vice versa.
* MM-22212: Fix for refactor bug not notifiying websocket about changed role.
* MM-22212: Adds test for public/private 'manage_members' handling.
* MM-22122 Fix manage channel members edge case for public and private channels (#14049)
* MM-22212: Adds moderated permission to team_admin.
* MM-22212: Updates migration.
* MM-22212: Revert unnecessary update to default roles.
* Add channel scheme updated event when channel scheme is deleted or created (#14057)
* MM-22212: Adds newline.
* MM-22212: Migration fix.
* MM-22212: Fix for migration.
* MM-22212: Test fix.
Co-authored-by: Farhan Munshi <3207297+fm2munsh@users.noreply.github.com>
* WIP
* Adding bleve to go modules
* WIP
* Adding missing files from searchengine implementation
* WIP
* WIP
* WIP
* WIP
* WIP
* WIP
* User and channel indexing and searches implemented
* Make bleve tests run with in-memory indexes
* Implement post index and deletion tests
* Initial commits for the search layer
* Removing unnecesary indexing
* WIP
* WIP
* More fixes for tests
* Adding the search layer
* Finishing the migration of searchers to the layer
* Removing unnecesary code
* Allowing multiple engines active at the same time
* WIP
* Add simple post search
* Print information when using bleve
* Adding some debugging to understand better how the searches are working
* Making more dynamic config of search engines
* Add post search basics
* Adding the Purge API endpoint
* Fixing bleve config updates
* Adding missed file
* Regenerating search engine mocks
* Adding missed v5 to modules imports
* fixing i18n
* Fixing some test around search engine
* Removing all bleve traces
* Cleaning up the vendors directory and go.mod/go.sum files
* Regenerating timer layer
* Adding properly the license
* Fixing govet shadow error
* Fixing some tests
* Fixing TestSearchPostsFromUser
* Fixing another test
* Fixing more tests
* Fixing more tests
* Removing SearchEngine redundant text from searchengine module code
* Fixing some reindexing problems in members updates
* Fixing tests
* Addressing PR comments
* Reverting go.mod and go.sum
* Addressing PR comments
* Fixing tests compilation
* Fixing govet
* Adding search engine stop method
* Being more explicit on where we use includeDeleted
* Adding GetSqlSupplier test helper method
* Mocking elasticsearch start function
* Fixing tests
Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* Introducing unit (not integration) tests for the app layer
* Initial support for unit tests at the API
* Adding unit tests support to the store layer
* Add unit tests support in commands
* Adding last tests needed for run unit tests properly
* Fixing govet
* Removing some duplication
* Fixing tests
* Fixing tests
* Not compiling test helpers with the main module for api
* Revert "Not compiling test helpers with the main module for api"
This reverts commit 36a199bbe0.
* Fixing tests
* Fixing unit tests
* More consistency between api4/apiteslib.go and app/helper_test.go
* Renaming things to make more obvious the new Setup functions purpose
* Reverting change in go.sum
* Start with empty mock for app layer
* Start with empty mock for api layer
* Start with empty mock for web layer
* Renaming SetupWithStoreMockConfig to SetupConfigWithStoreMock
* Fixing tests on web package
* Removing unnecesary function
* Run all tests fasts in postgres
* Moving postgres config to a file
* Addressing PR review comments
* Testing against mysql in the CI to keep things less changed
* Revert "Testing against mysql in the CI to keep things less changed"
This reverts commit fc940c111d.
* Fixing a test broken in postgres but working in mysql
* Fixing some tests
* Fixing some config tests with postgres
* Fixing the rest of config tests
* Fixing govet error
* Fixing search tests for postgres
* Consistent license message for all the go files
* Fixing the last set of unconsistencies with the license headers
* Addressing PR review comments
* Fixing busy.go and busy_test.go license header
* enable non-GOPATH testing
Tweak the testlib package to support finding the root without assuming the folder name of the repository.
Fixes: MM-19729
* fix web tests
* Migrating roles and schemes to new Cache Layer
* Adding missed license headers
* Updating cache tests
* Adding the cache layer to the testlib helper
* Fixing cyclic dependency
* fixing a bit of not-idiomatic error handling
* Another small fix arrount idiomatic error handling
* MM-16272 - Synchronize plugins across cluster (#11611)
* MM-16272 - Synchronize plugins across cluster
* Adding a test
* MM-16272 - Fixed tests
* MM-16272 - PR feedback
* MM-16270 - Plugin Sync (#11615)
* Initial implementation for plugin synch with file store. WIP
* Removed ListAll implementation. Used ListDirectory and change localstore to be consistent and return all items (files and folders) from directory
* Refactored plugin filestore operations out of main install/remove plugin
* Fixing error handling details
* Changes to use structured logging
* More logging fixes
* Wording and comments improvements
* Error handling and control flow improvements
* Changed managed flag check to use os.stat
* Added file store plugin dir and filename consts
* Replaced FileRead to use a the FileReader in PluginSync
* Minor styling and PR feedback changes
* Minor error handling improvements
* Added unit test for SyncPlugins. Changed SyncPlugins to use plugins environment to list available plugins
* PR Feedback improvements
* Minor err handling fix
* Removing FileStorePath from PluginEventData (#11644)
* Fix plugin path (#11654)
* tweak path, logging
Fix an issue not finding the plugins folder in S3. Tweak logging messages to add additional clarity.
* Removing FileExists check when Syncing plugins. Updated localstore to not return an error when directory does not exist
* PR Feedback
* Install prepackaged plugins locally only (#11656)
* s/uninstall/remove
* Updated ClusterMessage comment
* Updated PluginSync to test against s3 + local storage
* MM-16506: conditionally build go.mod for plugins
Unless `GO111MODULE=off`, generate a `go.mod` that points at the local copy of `mattermost-server` to ensure plugin tests that compile source code on demand always test with the local copy of mattermost-server. This also fixes an issue with early adopters of `GO111MODULE=on` on the server failing to find the right version of go-i18n dependencies.
* plugin: enable testlib resource management
* customize fileutils.Find* for testlib
* create/update config.json using go generate
* added default config generator
added config-reset to Jenkins and make package, updated defaults to consider 'isNew' flag
* corrections after code review
* fixed Config.isValid to handle empty encryption keys
* fixed Config.isValid to handle empty encryption keys
* fixed Config.isValid to handle empty encryption keys
* isUpdate now only checks for nil
* Addressed review comments, added unit testing for default config generator
* err shadowing
* license
* provide output file for config generator via ENV variable, since go generate doesn't support arguments and we need two output paths (config-reset and package)
* cleanup
* proper defaults for PushNotificationServer and SendPushNotifications
* corrected generating defaults for TrustedProxyIPHeader to be consistent with default.json in master
* Check for empty SiteURL as well as nil
* corrected SiteURL settings and checking
* crazy typos fixed
* corrected tests to newly expected values
* relaxed the checks
* fixed formatting
* [MM-13828] Running tests from a new temp folder with all test resources
Possible fix for #10132
All packages which have a TestMain and use testlib.MainHelper will have a new current working directory which will have all the test
resources copied.
Note: default.json is copied as config.json as well to make sure tests don't have any impact due to changes in config by devs
* [MM-13828] Added TestMain to remaining packages to use testlib.MainHelper
This makes sure tests from all packages run with same test resources, setup in a new temp folder for each package
* Updated Jenkins file to not not config/default.json
This makes sure CI has same config files as a dev's machine
* [MM-13828] Changes requested from code review
Added accessor methods to testlib.MainHelper for accessing members
Fixed some broken tests due to change in cwd while tests run
Some other code refactoring and improvements
* [MM-13828] Added new factory method with options for creating test main helper and some code refactoring
testlib.NewMainHelperWithOptions supports options to turn on/off test dependencies and environment setup
Some other code refactoring
* Exporting members of testlib.MainHelper to make enterprise tests work
* Fixed gofmt error
* [MM-13828] removed unwanted dependency on plugins directory while setting up test resources
* [MM-13828] Fixed some tests failing due to them being running from temp folder
* [MM-13828] Some code changes suggested in PR review
* Fixed gofmt error
* refactor utils/config* to config/
* pull validateLdapFilter into app
* clean up Config/GetConfig/GetSanitizedConfig usage
Eliminate app.GetConfig() in favour of just using app.Config() directly,
but expose app.GetSanitizedConfig() for when the old behaviour was
required.
* web: isolate config setup
* TestInvitePeopleProvider: make config explicit
* regenerateClientConfig: avoid racey map access
* integrate watch flag into app.ConfigFile option
* make app.Option return an error
* release.mk: only cp static files from config/
* release.mk: fix cp static files from config/
* api4: TestPlugin cleanup
* s/c/cfg/ for clarity
* fix merge conflict
* testlib: allow customization of testlib driver name
* MM-12083: unittest using externally managed database
* cherry-pick Makefile changes from @cpanato
* Jenkins changes (#9915)
* add docker compose
* udpate
* when using minio dont need to set the region
* update
* add wait for it script
* using old minio
* add new jenkins file
* update makefile
* add dockerfile
* rename the docker-compose proj to avoid colision (#9917)
* rename the docker-compose proj to avoid colision
* enable debug
* enable debug to double checkt the branchs and fix docker-compose name (#9919)
* add ee hash to check (#9920)
* fix name (#9921)
* update jenkins file to push from branch and prs
* if a new push comes in stop the running build
* split mysql and postgres variables
* add script to run jenkins-like env in local dev env
* update docker-compose project name to use uuid to make it more randon
* fix DCNAME definition
* update elasticsearch docker image
* revert test
* tidy up stages, and wait for mysql differently
* update docker image and add check for postgres
* checking if is ready
* update docker compose to have a wait for deps
* add readme and rename dockerfile
* fix -unittest setup
* using mm docker image
* restore parallel unit tests at the package level
Spin up a dedicated database for each package under test to avoid races in accessing the same tables.
Simplify the interface for configuring the test database to just a DSN instead of multiple exports for each field.
* try to work around root mysql access in CI
* update local-test-env.sh too
* MYSQL_ROOT_HOST: %
* fix missing quotes
* setting some memory limits for mysql
* revert memory docker compose does not support
* fix env name for postgres
* expose errors in app/export_test.go
* fix test label, better error checking on teardown
* increase query timeout for tests
* fix export_test
* update local dev script
* add configurable mysql root passwd
* testlib: introduce and leverage
This doesn't yet factor out the individual test helpers: many packages
still rely on `api4` directly to do this, but now wire up the test store
setup through this package. `app` and `store`, in particular, don't use
`testlib` because of circular dependencies at the moment.
* cmd: command_test.go: use api4 testlib
* cmd: plugin_test.go: remove dependence on test-config.json
* cmd: config_test.go use configured database settings
* ensure test-(te|ee) exit with status code
* test-server: run all tests, deprecating test-te/test-ee
* cmd/mattermost/commands: fix unit tests
Instead of relying on (and modifying) a config.json found in the current path, explicitly create a temporary one from defaults for each test. This was likely the source of various bugs over time, but specifically allows us to override the SqlSettings to point at the configured test database for all tests simultaneously.
* wrap run/check into a test helper
It was insufficient to set a config for each invocation of CheckCommand or RunCommand: some tests relied on the config having changed in a subsequent assertion. Instead, create a new test helper embedding api4.TestHelper. This has the nice advantage of cleaning up all the teardown.
* additional TestConfigGet granularity
* customized config path to avoid default location
* be explicit if the storetest initialization fails
* generate safe coverprofile names in the presence of subtests
* additional TestConfigShow granularity
* fix permission_test.go typo
* fix webhook tests
* actually flag.Parse() to skip database setup on os.Execed tests
* fix recent regression in #9962, not caught by unit tests
