Commit Graph

93 Commits

Author SHA1 Message Date
Doug Lauder
5e59b5f70c MM-23935 extend session expiry on user activity (#14275)
* MM-23935  extend session expiry on user activity

- if user types anything before a session expires the session will be extended to now + session length

- ensures new session expiries are not written to DB too frequently

- new session store func for updating session ExpiresAt

- session length defaults for mobile and web/ldap changed from 180 days to 30 days
2020-05-06 15:41:10 -04:00
Jesse Hallam
4582823f87 fix /test command (#14169)
Automatic Merge
2020-04-18 00:08:57 -07:00
Agniva De Sarker
4210ae60ab MM-22056: Fix flaky test TestUploadFiles (#14234)
Automatic Merge
2020-04-07 07:09:04 -07:00
Martin Kraft
4d99aa22ba MM-22212 & MM-22208: Read from the higher-scoped scheme if the permission is non-moderated. (#13813)
* MM-22212: Read non-moderated permissions from higher-scoped scheme.

* MM-2212: Corrects test count in comment.

* MM-22212: Adds godoc comment.

* MM-2212: Switches to the channel roles check in a few more places.

* MM-22212: Refactors and fixes.

* MM-22212: Reverts change, no longer required.

* MM-22212: Removes translation.

* MM-22212: Un-comments merged new permission.

* MM-22212: Un-comments merged new permission.

* MM-22212: Performance tweak.

* MM-22212: Fixes some fmting.

* MM-22212: Add unit test for newly-added store methods.

* MM-22212: Renames app method.

* MM-22212: Re-uses existing function to find string in slice.

* MM-22212: Keeps 'higher-scoped' terminology for consistency.

* MM-22212: Refactors based on PR feedback.

* MM-22212: Fix for some bad merging.

* MM-22212: Renamed some things.

* MM-22212: Use an 'else' instead of a 'continue' for readability.

* MM-22212: Caches (*SqlRoleStore).ChannelRolesUnderTeamRole.

* MM-22212: Adds mock to new cache store.

* MM-22212: Adds missing open tracing app layer methods.

* MM-22212: Adds migration to add moderated permissions to channel_admin if present on channel_user.

* MM-22212: Migrates team schemes. Removes unused AppError.

* MM-22212: Fix for for if.

* MM-22212: Fixes iterator.

* MM-22212: Updates open tracing generated methods.

* MM-22212: Fix mocks.

* MM-22212: Change migration key name.

* MM-22212: Switched to data structure from other branch.

* MM-22212: Fixes tests after adding 'use_channel_mentions' to the channel_admin role.

* MM-22212: Adds tracking of channel moderation.

* Revert "MM-22212: Adds tracking of channel moderation."

This reverts commit 23689efa22.

* MM-22212: Switch some functions to methods and vice versa.

* MM-22212: Fix for refactor bug not notifiying websocket about changed role.

* MM-22212: Adds test for public/private 'manage_members' handling.

* MM-22122 Fix manage channel members edge case for public and private channels (#14049)

* MM-22212: Adds moderated permission to team_admin.

* MM-22212: Updates migration.

* MM-22212: Revert unnecessary update to default roles.

* Add channel scheme updated event when channel scheme is deleted or created (#14057)

* MM-22212: Adds newline.

* MM-22212: Migration fix.

* MM-22212: Fix for migration.

* MM-22212: Test fix.

Co-authored-by: Farhan Munshi <3207297+fm2munsh@users.noreply.github.com>
2020-03-23 13:44:20 -04:00
Gabriel Sagula
a1719e9fcf MM-22707: improved image upload tests and logging (#14018)
* improved image upload tests and error logging message

Signed-off-by: Gabriel Linden Sagula <gsagula@gmail.com>

* removed unecessary logging message

Signed-off-by: Gabriel Linden Sagula <gsagula@gmail.com>

* fixed null ptr issue

Signed-off-by: Gabriel Linden Sagula <gsagula@gmail.com>

* rolled back pointer to the client

Signed-off-by: Gabriel Linden Sagula <gsagula@gmail.com>

* added hexdump of the previewer file

Signed-off-by: Gabriel Linden Sagula <gsagula@gmail.com>

* fixed tests that use the renamed files

Signed-off-by: Gabriel Linden Sagula <gsagula@gmail.com>

* addressed reviewer notes

Signed-off-by: Gabriel Linden Sagula <gsagula@gmail.com>

* addressed reviewer concerns

Signed-off-by: Gabriel Linden Sagula <gsagula@gmail.com>

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-17 10:20:29 +05:30
catalintomai
ed52acd89c MM-18062 - add support for Office365Settings Directory (tenant) Id (#13737)
* MM-18062 add directory id field to O365 settings
2020-02-14 09:55:40 -08:00
Ben Schumacher
cb49d560ab Allow plugins to specify a ReleaseNotesURL via manifest (#13677) 2020-01-28 19:33:21 +01:00
Maria A Nunez
ebfd332161 MM-21374 - Fixed extracting icon for prepackaged and local plugins (#13633)
* Merge upstream

* Added extracting iconData for prepackaged and local plugins

* Fixed tests

* Removed report.xml

Co-authored-by: Jason Paul Deland <9366595+jaydeland@users.noreply.github.com>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-01-20 10:34:18 -05:00
Ben Schumacher
3491c3abb2 Respect HomepageURL in manifest for local plugins (#13595)
* Respect HomepageURL in manifest for local plugins

* Add documentation for updating the plugin signatures
2020-01-17 09:09:58 +01:00
Maria A Nunez
87eb7697f9 MM-19606- Rework Prepackaged Plugins (#13449)
* MM-19609 - Add new prepackage configuration settings (#13062)

* Add signatures to the prepackaged plugins (#13138)

* MM-19612 - Support querying local plugin marketplace when upst… (#13250)

* MM-19612 - Support querying local plugin marketplace when upstream unavailable or disabled

* Update translations file

* Fixed comment

* Updated to check EnableRemoteMarketplace setting and LocalOnly to get marketplace plugins

* Fixed unit tests

* Tests cleanup code

* Removed unused error message

* Updated tests

* MM-19614- Updated Marketplace Service error id (#13388)

* [MM-19610] Consume prepackaged plugins (#13005)

* consume prepackaged plugins into memory

* missing i18n

* remove spurious .gitignore changes

* return on failure to install prepackged plugins

* cleanup

* s/plugins/availablePlugins

* whitespace

* don't return extractDir when not needed

* s/plug/plugin

* error on icon, cleanup

* update armored version of testplugin signature

* honour AutomaticPrepackagedPlugins

* document getPrepackagedPlugin

* MM-19613 - Include prepackaged plugins in marketplace results (#13433)

* Added prepackaged plugins to marketplace results

* PR Feedback

* PR Feedback

* Update error where definition

* Removing unnecessary var declaration

* Updated comments

* MM-21263 - Use EnableRemoteMarketplace in marketplace install… (#13438)

* MM-21263 - Use EnableRemoteMarketplace in marketplace install endpoint

* Call updateConfig before calling NewServer in TestHelper

* Added translations

* PR feedback

* Translations

* Feedback

* s/helpers.go/download.go

* Converging env.PrepackagedPlugins

* Initial PR feedback

* Ordered imports properly

* Updated DownloadURL to return slice of bytes

* Fixed method typo

* Fixed logging

* Added read lock for prepackaged plugins list

* PR Feedback

* Added condition to only install prepackaged plugin if it was previously enabled

* Linting

* Updated to check plugin state in config

* Closing filereader

* Only add local label if remote marketplace is enabled

* Updated local tag description

* Fixed tests

Co-authored-by: Ali Farooq <ali.farooq0@pm.me>
Co-authored-by: Shota Gvinepadze <wineson@gmail.com>
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
2020-01-15 13:38:55 -05:00
catalintomai
37ce413b7d MM-20698: Add server support for getting SAML metadata and updating S… (#13311)
* MM-20698: Add server support for getting SAMl metadata and updating SAMl settings
2020-01-09 21:25:20 -08:00
lindalumitchell
397e83f870 Remove redundant in-line image tests (#13578) 2020-01-09 14:35:47 +08:00
Shota Gvinepadze
9898f91f7e [MM-20309] production public key (#13244)
* Add production public key

* Fix unit tests
2019-12-05 23:41:52 +04:00
Ali Farooq
a6e992ae74 MM-16368 - Plugin Signing (#13017)
* [MM-18757] POST handler for `/plugins/marketplace` (#12372)

* Implement installMarketplacePlugin

* Add InstallMarketplacePlugin endpoint

* Fix go.mod

* merge with master

* Fix go.mod

* Fix plugin tests

* Move get plugin to marketplace client

* Fix stylistic concerns

* Add trailing newline to the go.mod

* [MM-16586] Add plugin signature settings (#12390)

* MM-17149 - Extend config.json for marketplace settings (#11933)

* MM-17149 - Extend config.json for marketplace settings

* Renamed MarketplaceUrl, tracking default marketplace url

* Added EnableMarketplace to the client config

* Revert "Added EnableMarketplace to the client config"

This reverts commit 0f982c4c66.

* MM-17149 - Added EnableMarketplace to the client config (#11958)

* Added EnableMarketplace to the client config

* Moved EnableMarketplace setting out of limited client configuration

* Add public key settings to the config.json

* Rename PublicKeys to SignaturePublicKeyFiles

* Change filepath.Split to Base

* Remove additional prints

* Force extention of a public key file

* Remove config validation

* Remove error on delete

* Remove config cloning

* Add error messages

* Add plugin public key tests

* Rename extension to PluginSignaturePublicKeyFileExtention

* Remove EnforceVerification

* Change []*PublicKeyDescription to []string

* Change .asc extension to .plugin.asc

* Change ordering of public methods

* Change plugin key commands

* Update examples in the plugin key commands

* Remove forcing extention

* Add verify signature in settings

* Fix tabbing

* Fix naming

* Remove unused text

* Remove unused text

* Update command examples

* Fix unit tests

* Change errors.New to errors.Wrap

* Fix verbose flag

* Change .asc to .gpg

* Fix }

* Change AddPublicKey signature

* Change public.key extension

* Add plugin public key command tests

* Update en.json

* Bootstrap the public keys

* Update en.json

* Fix en.json

* Fix en.json

* Bootstrap hard-coded public key

* Remove unused texts in en.json

* Change file to name

* Add license header

* Update development public key

* Remove writeFile method

* Remove .plugin.asc extension

* Rename publiKey to mattermostPublicKey

* Remove init_public_keys string

* GolangCI

* Closing file handlers

* Fixed test that was installing nps plugin

* [MM-19798] Implement plugin signature verification (#12768)

* MM-17149 - Extend config.json for marketplace settings (#11933)

* MM-17149 - Extend config.json for marketplace settings

* Renamed MarketplaceUrl, tracking default marketplace url

* Added EnableMarketplace to the client config

* Revert "Added EnableMarketplace to the client config"

This reverts commit 0f982c4c66.

* MM-17149 - Added EnableMarketplace to the client config (#11958)

* Added EnableMarketplace to the client config

* Moved EnableMarketplace setting out of limited client configuration

* Add public key settings to the config.json

* Rename PublicKeys to SignaturePublicKeyFiles

* Change filepath.Split to Base

* Remove additional prints

* Force extention of a public key file

* Remove config validation

* Remove error on delete

* Remove config cloning

* Add error messages

* Add plugin public key tests

* Rename extension to PluginSignaturePublicKeyFileExtention

* Remove EnforceVerification

* Change []*PublicKeyDescription to []string

* Change .asc extension to .plugin.asc

* Change ordering of public methods

* Change plugin key commands

* Update examples in the plugin key commands

* Remove forcing extention

* Add verify signature in settings

* Fix tabbing

* Fix naming

* Remove unused text

* Remove unused text

* Update command examples

* Fix unit tests

* Change errors.New to errors.Wrap

* Fix verbose flag

* Change .asc to .gpg

* Fix }

* Change AddPublicKey signature

* Change public.key extension

* Add plugin public key command tests

* Update en.json

* Bootstrap the public keys

* Update en.json

* Fix en.json

* Fix en.json

* Bootstrap hard-coded public key

* Remove unused texts in en.json

* Change file to name

* Add license header

* Implement plugin signature verification

* Remove benburker openpgp

* Update en.json

* Update development public key

* Add support of multiple signatures in filestore

* Update en.json

* Run go mod vendor

* Fix style

* Remove writeFile method

* Remove .plugin.asc extension

* Rename publiKey to mattermostPublicKey

* Verify plugin with mattermost public key

* Remove init_public_keys string

* Add InstallPluginWithSignature method and  Refactor

* Add signature verification on claster notification

* Remove armored signature headers

* Add error strings

* Fix en.json

* Change signatureStorePath

* Implement minor fixes

* Refactor plugin install methods

* Add installPlugin method to uploadPlugin

* Update en.json

* Refactor installPlugin

* Limit number of signatures

* Close signatures

* Fix helper function

* Fix fromReadCloseSeekerToReadSeeker

* Cleaned up ReadCloseSeeker for signatures

* Remove signature truncation on FS

* GolangCI

* Add tests for armored signatures and plugin uploads

* Fix nil slice issue

* Fix TestPluginSync

* Fixed tests

* Return io.ReadSeeker from downloadFromUrl

* Add log for the found plugins in the file store

* Remove logging plugin detection info

* [MM-20134] Consume and store single-signature for each plugin (#13081)

* Consume and store single-signature for each plugin

* Fix en.json

* Remove saveSignature method

* Remove public key hash

* PR Feedback

* refactored config

* PR feedback
2019-11-18 19:02:41 -05:00
Martin Kraft
598f7c7255 MM-17767: Remove ExperimentalLdapGroupSync config. (#12388) 2019-10-08 08:03:39 -04:00
Jesse Hallam
4ce7b92283 MM-17023: Plugin Marketplace (#12183)
* MM-17149 - Extend config.json for marketplace settings (#11933)

* MM-17149 - Extend config.json for marketplace settings

* Renamed MarketplaceUrl, tracking default marketplace url

* Added EnableMarketplace to the client config

* Revert "Added EnableMarketplace to the client config"

This reverts commit 0f982c4c66.

* MM-17149 - Added EnableMarketplace to the client config (#11958)

* Added EnableMarketplace to the client config

* Moved EnableMarketplace setting out of limited client configuration

* MM-17150, MM-17545, MM-18100 - Implement GET /api/v4/plugins/m… (#11977)

* MM-17150 - Implement GET /api/v4/plugins/marketplace proxying upstream
MM-17545 - Merge locally installed plugins into GET /api/v4/plugins/marketplace

* Replaced MarketplacePluginState with Installed

* Setting InstalledVersion instead of Installed

* marketplace client setting per_page if non zero

* Creating insecure client for marketplace url

* Fixed trailing slash for default marketplace url

* Adding filtering

* Fixed function names

* Renamed Manifest() to GetManifest(), added godoc for BaseMarketplacePlugin

* Handling plugin.ErrNotFound correctly

* Checking err == nil instead when a plugin is installed

* MM-18450 - Local-only plugin search (#12152)

* MM-17846: plugin icons (#12157)

* MM-17846: add support for plugin icons

Extend the model definitions to support plugin icons from the marketplace.

* s/IconURL/IconData

* MM-18475 - Converge on snake_case responses from the marketplace (#12179)

* MM-18520 - MM-Server should forward server version to marketplace server (#12181)

* Renamed request to filter client4.GetMarketplacePlugins

* Renamed request to filter

* Guarding against bad marketplace server response
2019-09-17 15:02:26 -04:00
scott lee davis
2fa63b10ec MM 7971 dockerhost -> localhost && docker -> docker-compose (#10872)
* replace dockerhost with localhost

* remove uneeded setup-max build step (no more dockerhost)

* changes as recommended by @cpanato

* make clean-docker with docker-compose

* added ports to docker-compose.yml (needed for osx).   ignore error for ldapadd (when already exists)

* add clean-old-docker to legacy.mk

* docker-compose stop instead of down for `make stop-docker`
2019-08-02 11:53:00 -03:00
Jesse Hallam
4beb1226c8 revert 0e534c8, rewriting plugin tests to use actual pugins (#11713)
Revert https://mattermost.atlassian.net/browse/MM-14559. The tests were refactored, but all the hooks were be run as:

    _, ret := hooks.MessageWillBePosted(nil, nil)

and the plugins were panicking, but then nothing was actually failing because this hook doesn't support returning a "proper" error. Good news is that the old tests are still passing, so nothing has regressed in the toolkit, but thinking we should revisit this holistically instead of trying to fix in place.
2019-07-29 10:07:00 -03:00
Lev
52ac87ea32 MM-16997: Simplified TestUploadFiles (#11707)
* MM-16997: Simplified TestUploadFiles

Tickets:
https://mattermost.atlassian.net/browse/MM-16997
https://mattermost.atlassian.net/browse/MM-16760

- The tests now fully buffer the data vefore uploading it. The prior
  code was much more complex because it was intended to eventually
  mature into generic client code; not a priority, definitely not
  valuable for this test.
- Also improved error handling in TestHookFileWillBeUploaded for
  MM-16760

* PR feedback: typo
2019-07-26 09:54:04 -07:00
Sven Hüster
5690df9d95 add support for new slack post format and multiple attachments in slack importer (#11415)
* add support for new slack post format and multiple attachments

* add test for multiple attachments in imported slack posts
2019-07-11 10:47:51 +01:00
Ali Farooq
2ecca12bed MM-15452 - Add ability to override LHS icon for bot accounts (#11423)
* MM-15452 - Add ability to override LHS icon for bot accounts

* MM-15452 - Added translations

* MM-15452 - Updated GetIconImage test to check returned image

* MM-15452 - Added Delete handler for /icon endpoint, invalidating user cache on set/delete

* MM-15452 - Moved /icon routes under bot/, addressed other pr feedback

* MM-15452 - More conflict resolutoin

* MM-15452 Restoring api4/user.go

* MM-15452 - Using require as opposed to t for test assertions

* MM-15452 - Updated as per PR feedback
2019-07-06 08:56:21 +02:00
Sven Hüster
cc6c385d3e Add Slack corporate import ability (#10905)
* add support for slack corporate export data

* changed tempChannels for channel types. concat after parsing channels

* fix typo

* add fixes suggested by @lieut-data
2019-06-04 17:11:29 -04:00
Eli Yukelzon
18612bf771 MM-14874 - Google SSO: Migrate user API endpoint from Google+ API to People API (#10833) 2019-05-19 10:31:29 +03:00
Martin Kraft
25fd962016 MM-14412: Enables force removals of team/channel members base on group constraints. (#10490)
* MM-14412: Enables force removals of team/channel members base on group constraints.

* Renames some existing methods and variables.

* Change return types to ChannelMembers and TeamMembers for some existing methods.

* Adds option to change LDAP_DATA to either 'qa' or 'test' with env variable.

* Adds methods to retrieve ChannelMembers and TeamMembers that, based on group constraints, should be deleted.

* Adds helper functions to create GroupTeams and GroupChannels.

* MM-14412: Switches to helper methods for GroupSyncable creation in test files.

* MM-14412: Style fix.

* MM-14412: Switches remaining GroupSyncable instances to be created with helper functions.

* MM-14412: Typo fix.

* MM-11412: Build fixes.

* MM-14412: Checks if user is team member before re-adding.

* MM-14412: Update for change of GroupConstrained fields type.
2019-04-02 09:22:50 -04:00
Eli Yukelzon
0e534c8a49 MM-14559 Rewrite existing plugin API tests to exclusively test via actual plugin (#10494)
* initial work on rewriting test cases using the raw literal format

* split out the embedded tests into separate file and implement the test using a generic tester function

* initial work on rewriting test cases using the raw literal format

* split out the embedded tests into separate file and implement the test using a generic tester function

* moved tests to separate dir to avoid conflicts

* removed duplicate files

* added license header

* fixed plugin paths

* moved all trivial tests to separate files
created "meta-test" that scans a folder of api tests and runs them

* remove rpc tests from make

* use fileutil.FindDir instead of pwd
clean up the test files
2019-04-01 08:39:45 -07:00
Miguel de la Cruz
44887a0272 Adds elasticsearch to the user and channel autocompletion functions (#10354)
* Adds elasticsearch to the user and channel autocompletion functions

* Implement channel store GetChannelsByIds test

* Style changes and govet fixes

* Add gofmt fixes

* Extract default channel search limit to a const

* Add StringSliceDiff function to the utils package

* Honor USER_SEARCH_MAX_LIMIT on the user autocomplete api handler

* Change the elasticsearch development image
2019-03-15 17:53:53 +00:00
Jesús Espino
921504483b Removing unnused InviteSalt config (#10446) 2019-03-15 07:06:28 +01:00
Yusuke Nemoto
1684f240e0 [MM-4819] Add MinimumHashtagLength config (#10248)
#### Summary
Add `MinimumHashtagLength` config

#### Ticket Link
#9156 
https://mattermost.atlassian.net/browse/MM-4819

#### Checklist
N/A
2019-03-11 14:09:50 +01:00
Jesse Hallam
72448d12a9 deprecate timezones.json (#10311) 2019-02-25 12:07:45 -04:00
Angela Chang
1018ead619 [MM-8421] Fix tiff preview - backend (#10170) 2019-02-21 09:06:59 +01:00
Martin Kraft
afcb224a79 Adds configuration for separate reply-to email header. (#10144)
* Adds configuration for separate reply-to email header.

* Changes config setting name.

* Using a separate variable and value in test.

* Updates for config pointer changes in another PR.

* Adds new key to test config. Adds default value.
2019-02-04 17:01:05 -05:00
Martin Kraft
6d30b21dd2 Ldap groups phase1 (#9752)
* Initial models, API, app, and persistence of groups and group syncing.

* Consistent letter casing in ldif.

* Moves group-specific migrations into func.

* Adds API endpoint to retrieve LDAP groups (and associated MM groups) one tree level at a time.

* Adds mattermost group id to SCIMGroup (if available).

* Splits user and group creation so that memberOf works. Returns users from ldap interface.

* Updates method name.

* Returns users IDs instead of User.

* Removes non-essential group data.

* MM-11807: Add GroupFilter to LDAP config. (#9513)

* MM-11807: Add GroupFilter to LDAP config.

* Add diagnostic.

* Adds new config option for using 'memberOf' overlay.

* Adds API endpoint to link a group.

* Removes debug statements.

* Adds unlink group API endpoint.

* Fix to LDAP API. Adds API method to client4 and app.

* Adds some missing app methods. Renames API unexported func.

* Fixes link/unlink API path to accept valid DNs.

* Allow any character for DN portion of path.

* Switches from DN to objectGUID or entryUUID as the remote identifier linking LDAP groups to MM groups.

* Formatting.

* Formatting.

* Setting group name field to an ID for phase 1.

* Adds an LDAP config field to Setting up configuration for local LDAP.

* Changes to LDAP and GroupStore interfaces.

* Draft of nesting groups in API response.

* Removes unnecessary tree models.

* Updates group membershipt create store method to also restore.

* Adds new config to test config.

* Accept AD format length.

* Switches to SetUniqueTogether method.

* Updates revert.

* Tweaks to syncing queries .

* Updates query for pending team and channel memberships.

* Removes old GroupSyncableScanner usage. Some formatting and renaming.

* Fixes bug setting syncable type in selecting paged.

* Adds tests for syncables populator.

* Only add users to teams and channels that are not deleted.

* Renames method.

* Updates test LDAP setup.

* Removes memberof config stuff.

* Renames.

* Updates test data.

* Fix for gofmt.

* Adds missing license.

* Adds missing teardowns.

* Test fix.

* Adds a cycle to the groups test data.

* Changes API to return flat list.

* Removes some unused interface and app methods.

* Returns empty braces if results are empty.

* Adds more LDAP test data.

* Fix for test data error.

* Adds error.

* Moves test groups.

* Adds OU for load test data.

* Moves load test ou creation to load data.

* Adds a new bool flag to SCIMGroups.

* Removes SCIMGroup completely.

* Removes FULL JOIN because it is not supported in MySQL.

* Adds tests for sync queries; renames constant.

* Bad merge fix.

* Vet fix.

* Returning OK on delete ldap group link

* Removes foreign key constraints.

* Adding total to the ldap getAllGroups api endpoint

* Adds get group members page.

* Removes pagination from groups syncables list API.

* Adding syncable check now that foreign key constraint is removes.

* Joins teams and channels to group syncables.

* Adds group member count.

* Adding GetAllChannels and SearchAllChannels for system admins only

* Fix.

* Test fix from pagination removal.

* Orders groupmembers by createat.

* Fixing search of all channels

* Test fix after removing pagination.

* JSON syntax error fix.

* Changing tests (for now) pending investigation.

* Adding GetAllChannels and SearchAllChannels tests for the store

* Adding GetAllChannels and SearchAllChannels API tests

* Omit empty JSON values of group syncables.

* Fixing GetAllChannels and SearchAllChannels tests

* Fixing GetAllChannels and SearchAllChannels store tests

* Fixing GetAllChannels api tests

* Adds 'LDAP groups' feature flag. (#9861)

* Migrate new client functions to idiomatic error handling

* Test fixes.

* Simplification of groups api (#9860)

* Simplification of groups api

* Fixing RequireSyncableType

* Test fix.

* Update api4/group.go

Co-Authored-By: mkraft <martinkraft@gmail.com>

* Update api4/group.go

Co-Authored-By: mkraft <martinkraft@gmail.com>

* Update api4/group.go

Co-Authored-By: mkraft <martinkraft@gmail.com>

* Update api4/group.go

Co-Authored-By: mkraft <martinkraft@gmail.com>

* Update api4/group.go

Co-Authored-By: mkraft <martinkraft@gmail.com>

* Update api4/group.go

Co-Authored-By: mkraft <martinkraft@gmail.com>

* Update api4/group.go

Co-Authored-By: mkraft <martinkraft@gmail.com>

* Update api4/group.go

Co-Authored-By: mkraft <martinkraft@gmail.com>

* Update api4/group.go

Co-Authored-By: mkraft <martinkraft@gmail.com>

* Update api4/group.go

Co-Authored-By: mkraft <martinkraft@gmail.com>

* Fix copy/paste error.

* Fix copy/paste error.

* Adds missing return, changes to correct HTTP status code.

* Adds missing return, changes status codes.

* Check for license.

* Renames variable for new signature.

* Adds client method to get a group.

* Adds client method and tests for PatchGroup.

* Adds more API tests.

* Adds groups API tests.

* Adds client method and tests for getting group syncables.

* Adds tests for patching group teams and channels.

* Update to translations.

* Removes test.

* Fix incorrect conditional.

* Removes unnecessary nil check.

* Removes unnecessary return.

* Updates comment, removes unused variable.

* Uses consistent JSON unmarshal pattern.

* Uses consistent JSON unmarshal pattern.

* Moves const block.

* Switches 'already linked' from error to success response.

* Removes commented-out code.

* Switched to status ok.

* Add parens for readability.

* Fix copy/paste error.

* Unexport some structs.

* Removes repeated validity check.

* Return without attempting commit if there's a rollback.

* Fix incorrect HTTP status code.

* Update store/sqlstore/group_supplier.go

Co-Authored-By: mkraft <martinkraft@gmail.com>

* Adds utility methods for going from groupsyncable to groupteam and groupchannel.

* Fixing george suggestions (#9911)

* Test fix.

* Adds QA data to VC with visualization.

* Fixes typo in graph image.

* Update display name when re-linking in case it has changed in LDAP.

* Adds ability to configure group display name and unique identifier. (#9923)

* Adds ability to configure group display name and unique identifier.

* Adds some configs to confi-ldap make command.

* Fix for move of session.

* Exposes method for use by SAML package.

* Switches GroupSyncableType from int to string.

* Update Jenkins build files.

* Removes unused variable assignment.

* Removes old unnecessary early return.

* Removes unnecessary variable.

* Moves param parsing before license and permissions checks.

* Removes old code.

* Compares agains underlying error rather than error id.

* Switches tests to assertions.

* Adds more assertions.

* Adds missing return.

* Adds space after comma for added legibility.

* Moves a view model to the api package.

* Unexports method.

* Uses id validator function.

* Fix docker-compose flag.

* Typo fix.

* Moves index creation to supplier.

* Removes bad merge.

* Renames parameter.

* Re-adds space.

* Removes unnecessary transaction.

* Escapes the Groups table name with backticks because it is a reserved keyword.

* Fix roles cache bug

* Removing unnecesiary deserializing function

* Switches table name rather than custom SQL everywhere for Postgres without backticks.

* Removes redundant check for sql.ErrNoRows.

* Removes redundant check for sql.ErrNoRows.

* Removes data integrity check and redundant nil conditional.

* Removes redundant check for sql.ErrNoRows.

* Removes unnecessary query.

* Removes ID length validation from persistence tier.

* Makes some supplier methods idempotent.

* Removes some empty switch defaults.

* Renames Group Type field to Source.

* Fix for mistaken field name change.

* Uses IsValidId function.

* Removes comment.

* Changes json key name.

* Removes test because no longer validating user.

* Moves model state validation to app layer.

* Don't create Groups.CanLeave column until phase 2.

* Removes state validation until properties are used in phase 2.

* Removes duplicated check.

* Removes state validation until properties are used in phase 2.

* Removes some tests until phase 2.

* Comment-out a bunch of test related to CanLeave.

* Extra unmarshal validation check. Removes more code for CanLeave.

* Removes tests for CanLeave.

* Explict error msg.

* Rewrite queries.

* Changes index name. Adds index.

* Removes assertion.

* Adds experimental feature flag.
2019-01-10 15:17:31 -05:00
Lev
fae4f60b33 MM-7633: Optimize memory utilization during file uploads (#9835)
* MM-7633: Optimize memory utilization during file uploads

Refactored the file upload code to reduce redundant buffering and stream
directly to the file store. Added tests.

Benchmark results:
```
levs-mbp:mattermost-server levb$  go test -v -run nothing -bench Upload -benchmem ./app
...
BenchmarkUploadFile/random-5Mb-gif-raw-ish_DoUploadFile-4         	      10	 122598031 ns/op	21211370 B/op	    1008 allocs/op
BenchmarkUploadFile/random-5Mb-gif-raw_UploadFileTask-4           	     100	  20211926 ns/op	 5678750 B/op	     126 allocs/op
BenchmarkUploadFile/random-5Mb-gif-UploadFiles-4                  	       2	1037051184 ns/op	81806360 B/op	 3705013 allocs/op
BenchmarkUploadFile/random-5Mb-gif-UploadFileTask-4               	       2	 933644431 ns/op	67015868 B/op	 3704410 allocs/op
BenchmarkUploadFile/random-2Mb-jpg-raw-ish_DoUploadFile-4         	     100	  13110509 ns/op	 6032614 B/op	    8052 allocs/op
BenchmarkUploadFile/random-2Mb-jpg-raw_UploadFileTask-4           	     100	  10729867 ns/op	 1738303 B/op	     125 allocs/op
BenchmarkUploadFile/random-2Mb-jpg-UploadFiles-4                  	       2	 925274912 ns/op	70326352 B/op	 3718856 allocs/op
BenchmarkUploadFile/random-2Mb-jpg-UploadFileTask-4               	       2	 995033336 ns/op	58113796 B/op	 3710943 allocs/op
BenchmarkUploadFile/zero-10Mb-raw-ish_DoUploadFile-4              	      30	  50777211 ns/op	54791929 B/op	    2714 allocs/op
BenchmarkUploadFile/zero-10Mb-raw_UploadFileTask-4                	      50	  36387339 ns/op	10503920 B/op	     126 allocs/op
BenchmarkUploadFile/zero-10Mb-UploadFiles-4                       	      30	  48657678 ns/op	54791948 B/op	    2719 allocs/op
BenchmarkUploadFile/zero-10Mb-UploadFileTask-4                    	      50	  37506467 ns/op	31492060 B/op	     131 allocs/op
...
```

https://mattermost.atlassian.net/browse/MM-7633 https://github.com/mattermost/mattermost-server/issues/7801

[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]
- [x] Added or updated unit tests (required for all new features)
- [ ] Added API documentation (required for all new APIs)
- [ ] All new/modified APIs include changes to the drivers
  *N/A*???
- [x] Includes text changes and localization file ([.../i18n/en.json](https://github.com/mattermost/mattermost-server/blob/master/i18n/en.json)) updates

Overview of changes:
- api4
    - Replaced `uploadFile` handler with `uploadFileStream` that reduces
      unnecessary buffering.
    - Added/refactored tests for the new API.
    - Refactored apitestlib/Check...Status functions.
- app
    - Added App.UploadFileTask, a more efficient refactor of UploadFile.
    - Consistently set `FileInfo.HasPreviewImage`
    - Added benchmarks for the new and prior implementations
    - Replaced passing around `*image.Image` with `image.Image` in the
      existing code.
- model
    - Added a more capable `client4.UploadFiles` API to match the new server
      API’s capabilities.
- I18n
    - Replaced `api.file.upload_file.bad_parse.app_error` with a more generic
      `api.file.upload_file.read_request.app_error`
- plugin
    - Removed type `plugin.multiPluginHookRunnerFunc` in favor of using
      `func(hooks Hooks) bool` explicitly, to help with testing
- tests
    - Added test files for testing images

Still remaining, but can be separate PRs - please let me know the preferred
course of action
    - Investigate JS client API - how does it do multipart?
    - Performance loss from old code on (small) image processing?
    - Deprecate the old functions, change other API implementations to use
      UploadFileTask

Definitely separate future PRs - should I file tickets foe these?
    - Only invoke t.readAll() if there are indeed applicable plugins to run
    - Find a way to leverage goexif buffer rather than re-reading

Suggested long-term improvements - should I file separate tickets for these?
- Actually allow uploading of large (GB-sized) files. This may require a
  change in how the file is passed to plugins.
- (Many) api4 tests should probably be subtests and share a server setup -
  will be much faster
- Performance improvements in image processing (goexif/thumbnail/preview)
  (maybe use https://mattermost.atlassian.net/browse/MM-10188 for this)

Questions:
1. I am commiting MBs of test images, are there better alternatives? I can
   probably create much less dense images that would take up considerably less
   space, even at pretty large sizes
2. I18n: Do I need to do anything special for the string change? Or just wait
   until it gets picked up and translated/updated?
3. The image dimensions are flipped in resulting FileInfo to match the actual
   orientation. Is this by design? Should add a test for it, perhaps?
4. What to do in the case of partial success? S3 but not DB, some files but not
   others? For now, just doing what the old code did, I think.
5. Make maxUploadDrainBytes configurable? Also, should this be the systemic
   behavior of all APIs with non-empty body? Otherwise dropped altogether?
   Check all other ioutil.ReadAll() from sockets. Find a way to set a total
   byte limit on request Body?

* WIP - Fixed for GetPluginsEnvironment() changes

* WIP - PR feedback

1. Refactored checkHTTPStatus to improve failure messages
2. Use `var name []type` rather than `name := ([]type)(nil)`
3. Replaced single-letter `p` with a more intention-revealing `part`
4. Added tests for full image size, `HasPreviewImage`

* WIP - rebased (c.Session->c.App.Session)

* WIP - PR feedback: eliminated use of Request.MultipartReader

Instead of hacking the request object to use r.MultipartReader now have own
functions `parseMultipartRequestHeader` and `multipartReader` eliminating the
need to hack the request object to use Request.MultipartReader limitations.

* WIP - PR feedback: UploadFileX with functional options

* WIP - PR feedback: style

* WIP - PR feedback: errors cleanup

* WIP - clarified File Upload benchmarks

* WIP - PR feedback: display the value of erroneous formname

* WIP - PR feedback: fixed handling of multiple channel_ids

* WIP - rebased from master - fixed tests

* PR Feedback

* PR feedback - moved client4.UploadFiles to _test for now
2018-12-13 22:32:07 +01:00
Jesse Hallam
d39d9a5caf Dockerized build updated tests (#9943)
* testlib: introduce and leverage

This doesn't yet factor out the individual test helpers: many packages
still rely on `api4` directly to do this, but now wire up the test store
setup through this package. `app` and `store`, in particular, don't use
`testlib` because of circular dependencies at the moment.

* cmd: command_test.go: use api4 testlib

* cmd: plugin_test.go: remove dependence on test-config.json

* cmd: config_test.go use configured database settings

* ensure test-(te|ee) exit with status code

* test-server: run all tests, deprecating test-te/test-ee

* cmd/mattermost/commands: fix unit tests

Instead of relying on (and modifying) a config.json found in the current path, explicitly create a temporary one from defaults for each test. This was likely the source of various bugs over time, but specifically allows us to override the SqlSettings to point at the configured test database for all tests simultaneously.

* wrap run/check into a test helper

It was insufficient to set a config for each invocation of CheckCommand or RunCommand: some tests relied on the config having changed in a subsequent assertion. Instead, create a new test helper embedding api4.TestHelper. This has the nice advantage of cleaning up all the teardown.

* additional TestConfigGet granularity

* customized config path to avoid default location

* be explicit if the storetest initialization fails

* generate safe coverprofile names in the presence of subtests

* additional TestConfigShow granularity

* fix permission_test.go typo

* fix webhook tests

* actually flag.Parse() to skip database setup on os.Execed tests

* fix recent regression in #9962, not caught by unit tests
2018-12-06 13:19:32 -05:00
Harrison Healey
159c2a44dc MM-10987 Turn on ExperimentalLimitClientConfig by default and remove setting (#9696) 2018-10-24 11:52:53 +01:00
Jesse Hallam
d1805733fe MM-12505: remove all things WebRTC (#9700) 2018-10-18 14:49:10 -03:00
Daniel Schalla
557fd9ea18 Set default ciphers, set tls 1.2 via config, set curve prefs (#9315)
Config Checks at StartUp Part1

Config Checks; Tests for TLS Server

HSTS header implementation + tests

make gofmt happy with new go version...

make gofmt happy with new go version #2...

fix logic bug

fix typo

Fix unnecessary code block
2018-10-16 07:51:46 -07:00
Druhin Bala
301e7eb1c4 MM-10699 Add channel renaming to CLI (#9094) 2018-07-18 11:49:48 +01:00
Santos Solorzano
f57d279dc0 [MM-10117] Add support to add/delete and enable/disable plugins via CLI (#8745)
* Testing caching for emojis

* MM-10117 Add support to add/delete and activate/deactivate plugins via CLI

* Removing old work

* MM-10117 Moved files and addedd plugin test

* MM-10117 Renamed commands to enable/disable and updated add test

* MM-10117 Finished plugin test and improved error message for plugin commands

* MM-10117 Fixing plugin directories for test

* MM-10117 Renamed commands and updated commands to support multiple plugins

* MM-10117 Updating removed to deleted textclear

* MM-10117 Fixing nil pointer error for listing plugins

* MM-10117 Removing fileReader close

* MM-10117 Declaring error for GetPlugins

* MM-10117 Removing unnecessary nil check
2018-07-11 12:17:19 -04:00
Christopher Speller
1e5c432e10 MM-10702 Moving plugins to use hashicorp go-plugin. (#8978)
* Moving plugins to use hashicorp go-plugin.

* Tweaks from feedback.
2018-06-25 12:33:13 -07:00
lindalumitchell
be00e242d7 Updated to Google Drive location (#7856)
Missed a link to the sample files when I updated before. Also tidied a sentence and added some clarifying instructions.
2017-11-20 12:36:19 -08:00
lindalumitchell
bdd47539bf Update with new link to mm_file_testing in Drive (#7812) 2017-11-10 11:51:30 -08:00
lindalumitchell
87e816da23 Remove duplicate Nature section (#7571) 2017-10-04 12:01:26 -07:00
Joram Wilander
899ab31fff Implement experimental REST API endpoints for plugins (#7279)
* Implement experimental REST API endpoints for plugins

* Updates per feedback and rebase

* Update tests

* Further updates

* Update extraction of plugins

* Use OS temp dir for plugins instead of search path

* Fail extraction on paths that attempt to traverse upward

* Update pluginenv ActivePlugins()
2017-09-01 09:00:27 -04:00
lindalumitchell
eec79e042e PLT-7356 and 7378 Reduce size of emoji test posts (#7291)
* PLT-7356 Split emoji test 1 into smaller test files

Prevent test files from splitting into multiple posts.

Reducing the number of emoji in each file (test-emoticons1.md through test-emoticons4.md) so each post is <4000 characters. Adding two new files to accommodate the emoji overflow: test-emoticons5.md and test-emoticons6.md

* PLT-7356 Split emoji test 2 into smaller test files

Prevent test files from splitting into multiple posts.

Reducing the number of emoji in each file (test-emoticons1.md through test-emoticons4.md) so each post is <4000 characters. Adding two new files to accommodate the emoji overflow: test-emoticons5.md and test-emoticons6.md

* PLT-7378 Split emoji test 3 into smaller test files

Prevent test files from splitting into multiple posts.

Reducing the number of emoji in each file (test-emoticons1.md through test-emoticons4.md) so each post is <4000 characters. Adding two new files to accommodate the emoji overflow: test-emoticons5.md and test-emoticons6.md

* PLT-7378 Split emoji test 4 into smaller test files

Prevent test files from splitting into multiple posts.

Reducing the number of emoji in each file (test-emoticons1.md through test-emoticons4.md) so each post is <4000 characters. Adding two new files to accommodate the emoji overflow: test-emoticons5.md and test-emoticons6.md

* PLT-7356 and 7378 Create test-emoticons5.md

Prevent test files from splitting into multiple posts.

Reducing the number of emoji in each file (test-emoticons1.md through test-emoticons4.md) so each post is <4000 characters. Adding two new files to accommodate the emoji overflow: test-emoticons5.md and test-emoticons6.md

* PLT-7356 and 7378 Create test-emoticons6.md

Prevent test files from splitting into multiple posts.

Reducing the number of emoji in each file (test-emoticons1.md through test-emoticons4.md) so each post is <4000 characters. Adding two new files to accommodate the emoji overflow: test-emoticons5.md and test-emoticons6.md
2017-08-25 10:43:06 -04:00
lindalumitchell
19804c4e40 Update test .md files with new emoji set (#7104)
* Updated with new emoji set for testing

Changed test-emoticons4.md from Symbols and formatting tests to Flags, Special, and formatting tests.

* Updated with new emoji set for testing

Updated test-emoticons3.md from Places to Objects and Symbols.

* Updated with new emoji set for testing

Updated test-emoticons2.md from Objects to Food, Activity, and Travel.

* Updated with new emoji set for testing

Updated test-emoticons1.md to new emoji set, still containing the intro, People, and Nature sections.
2017-08-03 08:54:34 -04:00
Harrison Healey
3e9b5c2776 PLT-6813 Removed misleading and outdated link tests (#6767)
* PLT-6813 Removed misleading tests using default ports

* PLT-6813 Removed tests using basic authentication
2017-06-28 07:27:32 -07:00
lindalumitchell
a32f6f7733 Update test-tables.md (#6775) 2017-06-28 09:37:43 -04:00
Jason Blais
647d5b1b45 Fix places where we still refer to "/loadtest" (#6732)
* Update README.md

* Update developer_settings.jsx

* Update command_loadtest_test.go
2017-06-23 13:36:13 -04:00
Jeff Schering
1594cf8af1 Renamed /loadtest to /test (#6669) 2017-06-19 10:54:19 -07:00