* add request context
* move initialialization to server
* use app interface instead of global app functions
* remove app context from webconn
* cleanup
* remove duplicated services
* move context to separate package
* remove finalize init method and move content to NewServer function
* restart workers and schedulers after adding license for tests
* reflect review comments
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Replacing require.nil in app layer
* Fixing some tests
* Fixing tests
* Reverting the error correction, defering it for another PR
* Fixing golangci-lint errors
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Reducing the filestore dependencies from the rest of the source code
* Making more generic config conversion to FileBackendSettings
* Fixing usage of the NewFileBackend function
* Fixing more usages of the NewFileBackend function
* Fix some linter errors
* Fix more linter errors
* Fixing some unit tests
* Fixing linter problem
* Addressing PR review comments
* Simplifing the CopyFile for tests
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
We shutdown all plugins prior to checking the log buffer
so that there are no active plugins writing to the buffer
when we read the contents.
https://mattermost.atlassian.net/browse/MM-31324
```release-note
NONE
```
* Start plugins concurrently
* Determine which plugins need to be activated/deactivated before changing their state
* Test activation/deactivation of plugins in TestSyncPluginsActiveState
* Change test comments
* remove unneeded temp var
th.InitBasic initializes users, channels, and teams. But a lot of
tests don't use it at all, causing unncessary delays. A single InitBasic
call takes around 400ms. This PR removes 71 of them, approximately shaving
off 28 seconds. Arguably, this would be more because some tests run in loops.
* Implemented AmazonS3PathPrefix
* Remove unecessary method
* fix for test
* fix for test which are failing
* fix for test which are failing
* fix for test
Co-authored-by: Dusan Panic <dusan@salestrekker.com>
* MM-22706: pass along set_online flag in websocket response
To let the client know whether a user has created a post without
being online or not, we get the set_online query param and
pass it down to the websocket event being passed down to the client.
With this PR, the "data" field of the `posted` event will contain
a `set_online` boolean field set to true/false depending on the
query_param set_online value set in the createPost call.
* Setting to false for auto responder
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* explicitly assert panic as error log
* Revert "[MM-18150] plugin panic trace should not be lost (#13559)"
This reverts commit 5d928b4f94, while leaving the unit tests intact
and now asserting debug logs instead.
* missing license header
* Transit panic from debug to error
* Parse plugin's StdErr and output panic to the mlog.Error
* Add unit tests
* Change log test
* Remove buffer from logger
* Remove 'panic' string filter
* Change *Buffer to io.Writer
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
Setting a `nil` value should actually delete the row instead of preserve it with a `nil` value.
The existing tests passed because they only checked the return value of `nil`. When running the tests on Postgres, this value is returned as an empty `[]byte{}` instead of `nil`, so update the test to reflect this semantically equivalent behaviour.
* MM-21103: change plugin signature path
Save as `<plugin_id>.tar.gz.sig` instead of `<plugin_id>.sig`. The latter was a relic of the previous design to support multiple plugin signatures, but now creates an inconsistency with how the original source files were supplied as `<some_name>.tar.gz` and `<some_name>.tar.gz.sig`.
Fixes: https://mattermost.atlassian.net/browse/MM-21103
* relax signature matches to avoid assuming signatures always exist
* Consistent license message for all the go files
* Fixing the last set of unconsistencies with the license headers
* Addressing PR review comments
* Fixing busy.go and busy_test.go license header
* MM-17549: use StatusNotFound when deleting plugin (#12983)
Trying to delete a plugin that does not exist should fail with a 404, not a 400.
Relates-to: https://mattermost.atlassian.net/browse/MM-17549
* MM-19630: marketplace: model ReleaseNotesURL (#13083)
Automatic Merge
* MM-20065: allow prepackaged plugin upgrade (#13076)
* MM-20065: allow prepackaged plugin upgrade
When locally installing prepackaged plugins, skip if a plugin exists with the same id and is the same or a newer version.
This is effectively a "poor man's" rework of prepackaged plugins to allow upgrade of prepackaged plugins via the marketplace. The larger plan to rework prepackaged plugins was deferred from v5.18.
Fixes: https://mattermost.atlassian.net/browse/MM-20065
* eliminate unnecessary installPlugin
* fix TestPluginSync defaults to match minio
* cleanExistingBundles
* close prepackaged filereader
* simplify
* Add SetWithOptions
* Avoid passing two structs to the functions
* Rename ExpiryInSeconds -> ExpireInSeconds
* Use t.Run for the tests
* Fix build
* Address feedback
* Update log message
* Update docs and use KVSetWithOptions in KVCompareAndSetJSON
* Improve code style
* Use struct instead of pointer to struct
* Fix minimum server versions
* Update documentation
* Address feedback
* Revert new implemention of kv helpers
* Adress feedback
* MM-16272 - Synchronize plugins across cluster (#11611)
* MM-16272 - Synchronize plugins across cluster
* Adding a test
* MM-16272 - Fixed tests
* MM-16272 - PR feedback
* MM-16270 - Plugin Sync (#11615)
* Initial implementation for plugin synch with file store. WIP
* Removed ListAll implementation. Used ListDirectory and change localstore to be consistent and return all items (files and folders) from directory
* Refactored plugin filestore operations out of main install/remove plugin
* Fixing error handling details
* Changes to use structured logging
* More logging fixes
* Wording and comments improvements
* Error handling and control flow improvements
* Changed managed flag check to use os.stat
* Added file store plugin dir and filename consts
* Replaced FileRead to use a the FileReader in PluginSync
* Minor styling and PR feedback changes
* Minor error handling improvements
* Added unit test for SyncPlugins. Changed SyncPlugins to use plugins environment to list available plugins
* PR Feedback improvements
* Minor err handling fix
* Removing FileStorePath from PluginEventData (#11644)
* Fix plugin path (#11654)
* tweak path, logging
Fix an issue not finding the plugins folder in S3. Tweak logging messages to add additional clarity.
* Removing FileExists check when Syncing plugins. Updated localstore to not return an error when directory does not exist
* PR Feedback
* Install prepackaged plugins locally only (#11656)
* s/uninstall/remove
* Updated ClusterMessage comment
* Updated PluginSync to test against s3 + local storage
* MM-14246 - Plugin framework: support transactional semantics with KV Store
Rename old, new variable names
Moving New function to the bottom
* Made CompareAndUpdate sync, updated tests
* Removed going through channel in CompareAndSetPluginKey
* Inserting new key when oldValue is nil to KVCompareAndSet
* Updated error text to include CompareAndSet
* Rerouted the unit test logs through t.Log
* resolving merge confilvts
* Update testing.go
* Update helper_test.go
* Added godocs for NewTestingLogger
* Added go docs for NewTestingLogger
* Resolving conflicts
* Stop inserting hashed plugin keys
* Address comments
* Remove else statement
* Address comments
* Add comment
* Update as per comments
* Refactor as per comments
* Update plugin_key_value_store.go
minor comment tweaks
* fail on non StatusNotFound for original key query
* idiomatic error handling, and do not clean up if insert fails
* guard app plugins with mutex
Shutting down the app could race with a goroutine that uses the plugins environment, since we shut down the plugins first before cleaning up goroutines.
* fix go vet issues
* Adding blank request context to plugin hooks for future use.
* Rename RequestContext to Context
* Adding context to ServeHTTP and ExecuteCommand
* Fixing import cycle in test.
* allow `Wait()`ing on the supervisor
In the event the plugin supervisor shuts down a plugin for crashing too
many times, the new `Wait()` interface allows the `ActivatePlugin` to
accept a callback function to trigger when `supervisor.Wait()` returns.
If the supervisor shuts down normally, this callback is invoked with
a nil error, otherwise any error reported by the supervisor is passed
along.
* improve plugin activation/deactivation logic
Avoid triggering activation of previously failed-to-start plugins just
becase something in the configuration changed. Now, intelligently
compare the global enable bit as well as the each individual plugin's
enabled bit.
* expose store to manipulate PluginStatuses
* expose API to fetch plugin statuses
* keep track of whether or not plugin sandboxing is supported
* transition plugin statuses
* restore error on plugin activation if already active
* don't initialize test plugins until successfully loaded
* emit websocket events when plugin statuses change
* skip pruning if already initialized
* MM-8622: maintain plugin statuses in memory
Switch away from persisting plugin statuses to the database, and
maintain in memory instead. This will be followed by a cluster interface
to query the in-memory status of plugin statuses from all cluster nodes.
At the same time, rename `cluster_discovery_id` on the `PluginStatus`
model object to `cluster_id`.
* MM-8622: aggregate plugin statuses across cluster
* fetch cluster plugin statuses when emitting websocket notification
* address unit test fixes after rebasing
* relax (poor) racey unit test re: supervisor.Wait()
* make store-mocks
