Commit Graph

22 Commits

Author SHA1 Message Date
Jesse Hallam
9726a917cb MM-21915: improve tar gz path traversal (#13675)
Automatic Merge
2020-01-27 11:33:11 -05:00
Shobhit Gupta
7d99d8fba7 Add IsValid method to *Manifest struct (#13609) 2020-01-17 21:08:55 +01:00
Maria A Nunez
87eb7697f9 MM-19606- Rework Prepackaged Plugins (#13449)
* MM-19609 - Add new prepackage configuration settings (#13062)

* Add signatures to the prepackaged plugins (#13138)

* MM-19612 - Support querying local plugin marketplace when upst… (#13250)

* MM-19612 - Support querying local plugin marketplace when upstream unavailable or disabled

* Update translations file

* Fixed comment

* Updated to check EnableRemoteMarketplace setting and LocalOnly to get marketplace plugins

* Fixed unit tests

* Tests cleanup code

* Removed unused error message

* Updated tests

* MM-19614- Updated Marketplace Service error id (#13388)

* [MM-19610] Consume prepackaged plugins (#13005)

* consume prepackaged plugins into memory

* missing i18n

* remove spurious .gitignore changes

* return on failure to install prepackged plugins

* cleanup

* s/plugins/availablePlugins

* whitespace

* don't return extractDir when not needed

* s/plug/plugin

* error on icon, cleanup

* update armored version of testplugin signature

* honour AutomaticPrepackagedPlugins

* document getPrepackagedPlugin

* MM-19613 - Include prepackaged plugins in marketplace results (#13433)

* Added prepackaged plugins to marketplace results

* PR Feedback

* PR Feedback

* Update error where definition

* Removing unnecessary var declaration

* Updated comments

* MM-21263 - Use EnableRemoteMarketplace in marketplace install… (#13438)

* MM-21263 - Use EnableRemoteMarketplace in marketplace install endpoint

* Call updateConfig before calling NewServer in TestHelper

* Added translations

* PR feedback

* Translations

* Feedback

* s/helpers.go/download.go

* Converging env.PrepackagedPlugins

* Initial PR feedback

* Ordered imports properly

* Updated DownloadURL to return slice of bytes

* Fixed method typo

* Fixed logging

* Added read lock for prepackaged plugins list

* PR Feedback

* Added condition to only install prepackaged plugin if it was previously enabled

* Linting

* Updated to check plugin state in config

* Closing filereader

* Only add local label if remote marketplace is enabled

* Updated local tag description

* Fixed tests

Co-authored-by: Ali Farooq <ali.farooq0@pm.me>
Co-authored-by: Shota Gvinepadze <wineson@gmail.com>
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
2020-01-15 13:38:55 -05:00
Jesse Hallam
89c0b61bc3 MM-21103: change plugin signature path (#13360)
* MM-21103: change plugin signature path

Save as `<plugin_id>.tar.gz.sig` instead of `<plugin_id>.sig`. The latter was a relic of the previous design to support multiple plugin signatures, but now creates an inconsistency with how the original source files were supplied as `<some_name>.tar.gz` and `<some_name>.tar.gz.sig`.

Fixes: https://mattermost.atlassian.net/browse/MM-21103

* relax signature matches to avoid assuming signatures always exist
2019-12-12 13:45:55 -04:00
Miguel de la Cruz
2259b7f2a8 [MM-19948] Set version on module file and internal paths (#13186)
* [MM-19948] Set version on module file and internal paths

* Fixes after merge

* Fix i18n checker error
2019-11-28 14:39:38 +01:00
Ali Farooq
a6e992ae74 MM-16368 - Plugin Signing (#13017)
* [MM-18757] POST handler for `/plugins/marketplace` (#12372)

* Implement installMarketplacePlugin

* Add InstallMarketplacePlugin endpoint

* Fix go.mod

* merge with master

* Fix go.mod

* Fix plugin tests

* Move get plugin to marketplace client

* Fix stylistic concerns

* Add trailing newline to the go.mod

* [MM-16586] Add plugin signature settings (#12390)

* MM-17149 - Extend config.json for marketplace settings (#11933)

* MM-17149 - Extend config.json for marketplace settings

* Renamed MarketplaceUrl, tracking default marketplace url

* Added EnableMarketplace to the client config

* Revert "Added EnableMarketplace to the client config"

This reverts commit 0f982c4c66.

* MM-17149 - Added EnableMarketplace to the client config (#11958)

* Added EnableMarketplace to the client config

* Moved EnableMarketplace setting out of limited client configuration

* Add public key settings to the config.json

* Rename PublicKeys to SignaturePublicKeyFiles

* Change filepath.Split to Base

* Remove additional prints

* Force extention of a public key file

* Remove config validation

* Remove error on delete

* Remove config cloning

* Add error messages

* Add plugin public key tests

* Rename extension to PluginSignaturePublicKeyFileExtention

* Remove EnforceVerification

* Change []*PublicKeyDescription to []string

* Change .asc extension to .plugin.asc

* Change ordering of public methods

* Change plugin key commands

* Update examples in the plugin key commands

* Remove forcing extention

* Add verify signature in settings

* Fix tabbing

* Fix naming

* Remove unused text

* Remove unused text

* Update command examples

* Fix unit tests

* Change errors.New to errors.Wrap

* Fix verbose flag

* Change .asc to .gpg

* Fix }

* Change AddPublicKey signature

* Change public.key extension

* Add plugin public key command tests

* Update en.json

* Bootstrap the public keys

* Update en.json

* Fix en.json

* Fix en.json

* Bootstrap hard-coded public key

* Remove unused texts in en.json

* Change file to name

* Add license header

* Update development public key

* Remove writeFile method

* Remove .plugin.asc extension

* Rename publiKey to mattermostPublicKey

* Remove init_public_keys string

* GolangCI

* Closing file handlers

* Fixed test that was installing nps plugin

* [MM-19798] Implement plugin signature verification (#12768)

* MM-17149 - Extend config.json for marketplace settings (#11933)

* MM-17149 - Extend config.json for marketplace settings

* Renamed MarketplaceUrl, tracking default marketplace url

* Added EnableMarketplace to the client config

* Revert "Added EnableMarketplace to the client config"

This reverts commit 0f982c4c66.

* MM-17149 - Added EnableMarketplace to the client config (#11958)

* Added EnableMarketplace to the client config

* Moved EnableMarketplace setting out of limited client configuration

* Add public key settings to the config.json

* Rename PublicKeys to SignaturePublicKeyFiles

* Change filepath.Split to Base

* Remove additional prints

* Force extention of a public key file

* Remove config validation

* Remove error on delete

* Remove config cloning

* Add error messages

* Add plugin public key tests

* Rename extension to PluginSignaturePublicKeyFileExtention

* Remove EnforceVerification

* Change []*PublicKeyDescription to []string

* Change .asc extension to .plugin.asc

* Change ordering of public methods

* Change plugin key commands

* Update examples in the plugin key commands

* Remove forcing extention

* Add verify signature in settings

* Fix tabbing

* Fix naming

* Remove unused text

* Remove unused text

* Update command examples

* Fix unit tests

* Change errors.New to errors.Wrap

* Fix verbose flag

* Change .asc to .gpg

* Fix }

* Change AddPublicKey signature

* Change public.key extension

* Add plugin public key command tests

* Update en.json

* Bootstrap the public keys

* Update en.json

* Fix en.json

* Fix en.json

* Bootstrap hard-coded public key

* Remove unused texts in en.json

* Change file to name

* Add license header

* Implement plugin signature verification

* Remove benburker openpgp

* Update en.json

* Update development public key

* Add support of multiple signatures in filestore

* Update en.json

* Run go mod vendor

* Fix style

* Remove writeFile method

* Remove .plugin.asc extension

* Rename publiKey to mattermostPublicKey

* Verify plugin with mattermost public key

* Remove init_public_keys string

* Add InstallPluginWithSignature method and  Refactor

* Add signature verification on claster notification

* Remove armored signature headers

* Add error strings

* Fix en.json

* Change signatureStorePath

* Implement minor fixes

* Refactor plugin install methods

* Add installPlugin method to uploadPlugin

* Update en.json

* Refactor installPlugin

* Limit number of signatures

* Close signatures

* Fix helper function

* Fix fromReadCloseSeekerToReadSeeker

* Cleaned up ReadCloseSeeker for signatures

* Remove signature truncation on FS

* GolangCI

* Add tests for armored signatures and plugin uploads

* Fix nil slice issue

* Fix TestPluginSync

* Fixed tests

* Return io.ReadSeeker from downloadFromUrl

* Add log for the found plugins in the file store

* Remove logging plugin detection info

* [MM-20134] Consume and store single-signature for each plugin (#13081)

* Consume and store single-signature for each plugin

* Fix en.json

* Remove saveSignature method

* Remove public key hash

* PR Feedback

* refactored config

* PR feedback
2019-11-18 19:02:41 -05:00
Jesse Hallam
2571d97723 MM-18517: Plugin Marketplace (Phase 2) (#13086)
* MM-17549: use StatusNotFound when deleting plugin (#12983)

Trying to delete a plugin that does not exist should fail with a 404, not a 400.

Relates-to: https://mattermost.atlassian.net/browse/MM-17549

* MM-19630: marketplace: model ReleaseNotesURL (#13083)

Automatic Merge

* MM-20065: allow prepackaged plugin upgrade (#13076)

* MM-20065: allow prepackaged plugin upgrade

When locally installing prepackaged plugins, skip if a plugin exists with the same id and is the same or a newer version.

This is effectively a "poor man's" rework of prepackaged plugins to allow upgrade of prepackaged plugins via the marketplace. The larger plan to rework prepackaged plugins was deferred from v5.18.

Fixes: https://mattermost.atlassian.net/browse/MM-20065

* eliminate unnecessary installPlugin

* fix TestPluginSync defaults to match minio

* cleanExistingBundles

* close prepackaged filereader

* simplify
2019-11-18 13:40:49 -04:00
Ali Farooq
72965d864a MM-18881 - Attempting to configure uninstalled plugins gets st… (#12687)
* MM-18881 - Attempting to configure uninstalled plugins gets stuck at Loading... without timeout.

* Added a test

* Closing websocketclient
2019-10-10 15:01:39 -04:00
Ali Farooq
5754ffc320 MM-17087 - Disable plugin on removal (#11779)
* MM-17087 - Disable plugin on removal

* Updated documentation

* Got reid of notifyPluginEvents

* Updated documentation

* Added plugin installation/activatoin flow as a toplevel go doc in plugin_install.go

* Generating webapp bundle on plugin installation

* Fixed shadowing issue

* Updated doc to include unguarded race condition

* Renamed GenerateWebappBundle

* Added a debug log when peers are not ready to notify

* Updated docs

* Removed extra line
2019-08-22 15:17:47 -04:00
Jesse Hallam
98ff5fab32 MM-16261: Synchronize plugins in HA (#11657)
* MM-16272 - Synchronize plugins across cluster (#11611)

* MM-16272 - Synchronize plugins across cluster

* Adding a test

* MM-16272 - Fixed tests

* MM-16272 - PR feedback

* MM-16270 - Plugin Sync (#11615)

* Initial implementation for plugin synch with file store. WIP

* Removed ListAll implementation. Used ListDirectory and change localstore to be consistent and return all items (files and folders) from directory

* Refactored plugin filestore operations out of main install/remove plugin

* Fixing error handling details

* Changes to use structured logging

* More logging fixes

* Wording and comments improvements

* Error handling and control flow improvements

* Changed managed flag check to use os.stat

* Added file store plugin dir and filename consts

* Replaced FileRead to use a the FileReader in PluginSync

* Minor styling and PR feedback changes

* Minor error handling improvements

* Added unit test for SyncPlugins. Changed SyncPlugins to use plugins environment to list available plugins

* PR Feedback improvements

* Minor err handling fix

* Removing FileStorePath from PluginEventData (#11644)

* Fix plugin path (#11654)

* tweak path, logging

Fix an issue not finding the plugins folder in S3. Tweak logging messages to add additional clarity.

* Removing FileExists check when Syncing plugins. Updated localstore to not return an error when directory does not exist

* PR Feedback

* Install prepackaged plugins locally only (#11656)

* s/uninstall/remove

* Updated ClusterMessage comment

* Updated PluginSync to test against s3 + local storage
2019-07-18 15:05:53 -03:00
Maria A Nunez
859c571558 MM-16261 - Store Plugin in File Store (#11511)
* Implemneted saving the plugin bundle on the file store upon plugin upload

* Fixed compilation error

* Fixed compilation issue

* Added deletion from file store upon plugin uninstall

* Added condition to delete from store only when exists. Added case of saving the bundle to the store when uploading from url. Added checks in plugin tests

* Fixed compilation error

* Moved storage of plugin bundle within app/installPlugin

* Moved storing to filestore before enabling the plugin

* Fixed error handling

* Code styling improvements

* Minor styling fix
2019-07-10 17:05:33 -03:00
Michael Kochell
b68194e035 [MM-15831] Improve system for storing status of available plug… (#11185)
* Move State property from activePlugin to PluginHealthStatus. env.activePlugins is now reserved for healthy running plugins.

* Add comments for function declarations

* Combine activePlugins and pluginHealthStatuses into a common structure, registeredPlugins

* Add check to see if plugin is active before deactivating it

* Make `Deactivate` set plugin status

* Add comment explaining the `registeredPlugins` map
* Give responsibility to set plugin disabled status upon deactivation back to `env.Deactivate`

* check if plugin needs to be deactivated before setting status
2019-06-25 17:44:08 -04:00
Hanzei
c8fab6e313 Fix shadowed variables in app package: Part 1 of 3 (#10000)
* Fix shadowed variables: Part 1

* Remove additional variable

* Fix tests

* Refactor JoinDefaultChannels
2019-01-25 17:40:56 +01:00
Lev
829f183bb0 MM-8662: Preserve plugin's enabled state when overwriting (#10005)
If a plugin upload overwrites an exiosting plugin, the new one will be
immediately enabled if the old one was enabled.
2018-12-17 08:35:12 -05:00
Jesse Hallam
c46d8ec892 MM-12974: guard app plugins with mutex (#9818)
* guard app plugins with mutex

Shutting down the app could race with a goroutine that uses the plugins environment, since we shut down the plugins first before cleaning up goroutines.

* fix go vet issues
2018-11-20 08:52:51 -05:00
Christopher Speller
ecade2f1ec MM-12849 Moving all non request scoped items to Server struct (#9806)
* Moving goroutine pool

* Auto refactor

* Moving plugins.

* Auto refactor

* Moving fields to server

* Auto refactor

* Removing siteurl duplication.

* Moving reset of app fields

* Auto refactor

* Formatting

* Moving niling of Server to after last use

* Fixing unit tests.
2018-11-07 10:20:07 -08:00
Jesse Hallam
b843774de8 MM-11905: deactivate plugins before unregistering (#9666)
Plugins that listen for config changes might incorrectly re-register
their commands before being /actually/ deactivated, leaving the new
commands alive thereafter.
2018-10-15 14:08:51 -07:00
Jesse Hallam
e87965f39d MM-11905: delete plugin commands on removal (#9601)
* defer plugin tear down for testing

* test expected plugin command unregistration

* MM-11905: uninstall plugin commands on remove
2018-10-10 10:56:54 -07:00
Jesse Hallam
8948b91d7a unpack prepacked plugins on plugins initialization (#9149)
* unpack prepackaged plugins on plugins initialization

* leverage utils.FindDir
2018-07-26 08:45:26 -04:00
Jesse Hallam
17f211c393 MM-11292: clean up plugins GoDoc (#9109)
* clean up plugins GoDoc:

- eliminate plugin.NewBlankContext() as unnecessary
- export ValidIdRegex as a string vs. the less readable var
- add/update various documentation strings
- hide everything by default, except where used by client plugins or the mattermost-server. The exception to this rule are the `*(Args|Returns)` structs which must be public for go-plugin, but are now prefixed with `Z_` with a warning not to use.
- include a top-level example to get plugin authors started

This is not a breaking change for existing plugins compiled against
plugins-v2.

* remove commented out ServeHTTPResponseWriter

* update examples to match developer docs

* add missing plugin/doc.go license header
2018-07-13 10:29:50 -04:00
Joram Wilander
7bfb5aec26 Add back enable/disable WS events (#9052) 2018-07-05 07:23:55 -07:00
Christopher Speller
1e5c432e10 MM-10702 Moving plugins to use hashicorp go-plugin. (#8978)
* Moving plugins to use hashicorp go-plugin.

* Tweaks from feedback.
2018-06-25 12:33:13 -07:00