Commit Graph

12208 Commits

Author SHA1 Message Date
Jesús Espino
7c3e651a7b Not skiping tests on commands, and removing unnecesary main_test.go files (#14032) 2020-03-11 16:25:45 +01:00
Jesús Espino
27d536b212 MM-21552: Adding SaveMultiple to posts (#13766)
* Adding SaveMultiple to posts

* Improving tests

* fixing i18n

* Fixing tests

* Improving testing on top of Save and SaveMultiple

* Fixing shadow variables

* Addressing some PR comments

* More clear update post test

* Addressing some PR comments

* Addressing some PR comments and simplifying the code

* Improting replies in bulk too

* Fixing reply count and processing last imported replies

* Adding OverwriteMultiple to posts aggregating everything in the same transaction

* Adding 2 pending tests to implement

* Adding tests for overwrite multiple posts

* Adding tests for TeamStore.GetByNames method

* Fixing shadow variables

* Addressing PR comments

* Extracting i18n strings

* Fixing tests

* Fixing tests

* Adding more test cases

* Using a variable instead of a fake timestamp
2020-03-11 14:29:32 +01:00
Alejandro García Montoro
2bec92a404 MM-21987 Resolve mentions in slash commands (#13762)
* Create infrastructure to manage mentions

Two new files have been added (along with their tests); namely:

- model/at_mentions.go: utilities to parse and manage mentions; for the moment,
it just contains a regex and a couple of functions to parse possible mentions
and to post-process them, but it can be extended in the future.
- model/mention_map.go: it contains two new types (UserMentionMap and
ChannelMentionMap) that both have FromURLValues and ToURLValues. These types
can be used when adding the mentions to the payload of the plugin slash
commands.

* Extend custom commands payload with mentions

Two couples of new fields are added to the payload; namely:

- user_mentions and user_mentions_ids: two aligned arrays of the same length
containing all the different @-mentions found in the command: the i-th element
of user_mentions_ids is the user identifier of the i-th element of
user_mentions.
- channel_mentions and channel_mentions_ids: two aligned arrays of the same
length containing all the different ~-mentions found in the command: the i-th
element of channel_mentions_ids is the channel identifier of the i-th element
of channel_mentions.

* Fix shadowing of variables and redundant return

* Fix shadowing of variable

* Address review comments (HT @lieut-data)

- Improvements in mentionsToTeamMembers and mentionsToPublicChannels:
	- Scope implementation details inside the functions.
	- Improve goroutines synchronization by using a sync.WaitGroup.
	- Retry lookup of username only if the returned error is http.StatusCode,
	  so we can return early if the error is more severe.
- Invert check in PossibleAtMentions to improve readability.
- Make user and channel mention keys private to the module.
- Allow the specification of an empty map of mentions in
(Channel|User)MentionsFromURLValues when both mentions keys are absent.
- Replace custom functions in tests with require.Equal on maps.

* Test functions to parse mentions from messages

* Extend plugin commands payload with mentions

* Add functions to CommandArgs to add mentions

The functions make sure that the maps are initialized before adding any value.

* Address review comments (HT @lieut-data)

- Adds a mlog.Warn to avoid burying the error when the user is not found.
- Improve readability in loop populating the mention map by moving the
initialization of the map closer to the loop and by iterating over the channel
itself, not over its length.

* File was not gofmt-ed with -s

* Close channel when all goroutines are finished

* Again, all code should be checked with gofmt -s

* Refactor code out of a goroutine

This change helps improve the readability of the code and does not affect its
overall performance. Less complexity is always better.

* Close channel and iterate over its range

Adapt mentionsToPublicChannels to have the same structure in the management
of the mentions channel as in mentionsToTeamMembers.

* Adapt mentionsToTeamMembers to new App

Commit 17523fa changed the App structure, making the *Server field
private, which is now accessed through the Srv() function.

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-11 11:50:12 +01:00
Shota Gvinepadze
5d928b4f94 [MM-18150] plugin panic trace should not be lost (#13559)
* Transit panic from debug to error

* Parse plugin's StdErr and output panic to the mlog.Error

* Add unit tests

* Change log test

* Remove buffer from logger

* Remove 'panic' string filter

* Change *Buffer to io.Writer

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-11 11:41:11 +04:00
Ibrahim Serdar Acikgoz
771574b652 [MM-17797] Add Timeout To SMTP Connection (#13251)
* model/config: add timeout setting to email settings

* services/mailservice: add timeout to connection

* app/server_app_adapter: show SMTP connection errors on log

* services/mailservice: add r/w deadline to smtp conn

* services/mailservice: add context timeout

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-10 18:34:31 +03:00
Ogundele Olumide
8327e9b0ba MM-16819 Helper function to build path to plugin asset (#13626)
Implement helper method to create a URL to the resource from a directory
2020-03-10 12:41:29 +01:00
Mario de Frutos Dieguez
ee86413cdd Include websocket users metric (#14020)
This commit adds new metrics to know the number of registered users
we have in every WebSocket broadcasts hub
2020-03-10 10:15:56 +01:00
Jesús Espino
cd382412fd Migrating file info store cache to the cache layer (#13045)
* Migrating file info store cache to the cache layer

* Remove unrelated changes

* Updating timer layer store

* Fixing usage of invalidate cache

* Removed repeated tests and unnecesiary require

* Using doInvalidateCacheCluster instead of removing the cache directly

* Addressing PR comments

* Fixing imports

* Fixing tests

* Fixing license header
2020-03-09 18:34:25 +01:00
ABDUL SATTAR MAPARA
b70d5df5c2 Added documentation to a few methods of sql store 'SqlTeamStore' (#14016)
* Added documentation to a few methods of sql store 'SqlTeamStore'

* Update store/sqlstore/team_store.go

Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com>

* Update store/sqlstore/team_store.go

Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com>

* Added documentation to a few methods of sql store 'SqlTeamStore'

* Corrected documentation for Save method

* Updated store/sqlstore/team_store.go

Co-Authored-By: Justine Geffen <justinegeffen@users.noreply.github.com>

Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com>
Co-authored-by: Justine Geffen <justinegeffen@users.noreply.github.com>
2020-03-09 15:13:02 +01:00
Carlos Tadeu Panato Junior
f2548d4e3d Upgrade 5.21 master (#13999)
* db: updgrade db to 5.21

* db: prepare db upgrade for 5.22

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-09 11:42:53 +01:00
Agniva De Sarker
0587c813e0 Fixing flaky test TestUnlinkGroupTeam (#14019)
There is a race at the DB level where the goroutine c.App.SyncRolesAndMembership
would set the the SchemeAdmin attribute to false. If this happens after
UpdateUserToTeamAdmin happens and before the Login, then the user does not
have the required permissions to unlink the group and it fails.

Thanks to @streamer45 for spotting the issue.

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-07 22:42:23 +05:30
Mario de Frutos Dieguez
c8a923d9e3 MM-22247: Space in content-type breaks integration with 3rd party apps (#13839) 2020-03-07 09:43:57 +01:00
Hossein Ahmadian-Yazdi
34668511cf fix read only channel when join channel (#13946)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-06 15:58:14 -05:00
Ibrahim Serdar Acikgoz
d5df6a3fc4 [MM-22314] Remove read limit at app/import which cause bulk import to fail (#13863)
* app/import: increase buffer size to avoid failing in large inputs

* remove commented out lock

* app/import: make maxTokenSize constant

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-06 23:14:35 +03:00
Miguel de la Cruz
e922541cc5 Add an app-layers check to the server's CI (#14015)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-06 17:59:47 +01:00
ebiiim
5f77479573 fix comment on PostAction.Options (#13957)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-06 10:21:42 -05:00
Doug Lauder
398a12329a MM-20913 add client API Client.GetServerBusy to fetch server busy state and timeout. (#13986)
* MM-20913 add client API to fetch server busy state and timeout

* deprecate current API that only returns timeout
* required for mmctl MM-20913

* restore deleted unit test for GetServerBusyExpires

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-06 11:37:07 +01:00
Miguel de la Cruz
cdc916ff52 Add missing opentracing layer changes (#14013) 2020-03-06 12:18:02 +05:30
Farhan Munshi
79c786bc0c [MM-22206] Add PATCH channel moderations (PUT /moderations/patch) (#13845)
* MM-22205 Add get channel moderations endpoint

* MM-22206 Add patch channel moderations endpoint

Add api tests for patch channel moderations

* MM-22205 Ensure ordered permissions returned and create struct ChannelModeratedRoles

* MM-22206 Use structs instead of map

* MM-22206 Add test cases for GetChannelModeratedPermissions

* MM-22206 Add tests for ChannelModeratedPermissionsChangedByPatch

* MM-22206 Use NewBool instead of defining booleans

* MM-22206 Tie Channel Mentions to Create Posts when building Channel Moderations

* Revert "MM-22206 Tie Channel Mentions to Create Posts when building Channel Moderations"

This reverts commit a0bfc95f17.

* MM-22206 Review changes

Modify GetSchemeRolesForChannel to return named variables
Change calls to SessionHasPermissionToChannel to SessionHasPermissionTo
Add a CreateChannelScheme method
Add a DeleteChannelScheme method
Move GetChannelModeratedPermissions to Role model

* Fix lint

* Add ChannelModeration methods to App interface

* MM-22206 Rename method to GetTeamSchemeChannelRoles

* MM-22206 Check CHANNEL_MODERATED_PERMISSIONS_MAP for existing permission before iterating through it

* Modify wording to higherScoped to match #13813

* MM-22206 Delete channel scheme between tests

* MM-22206 Fix tests

* Actually patch role

* MM-22206 Shadow declaration of err
2020-03-05 10:04:34 -05:00
Miguel de la Cruz
bfde6d1f3e Removing unnecesary main_test.go files (#13979) 2020-03-05 14:46:52 +01:00
Miguel de la Cruz
182c29b456 MM-21898: Part 2. Add opentracing (#13904)
* initial implementation of opentracing

* app layer

* Revert Makefile

* .

* cleanup

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* [ci]

* autogenerate interface

* .

* missed vendor files

* updated interfaces

* updated store layers

* lint fixes

* .

* finishing layer generators and nested spans

* added errors and b3 support

* code review

* .

* .

* fixed build error due to misplased flag.Parse()

* code review addressed
2020-03-05 14:46:08 +01:00
Ben Schumacher
5a34ec4793 Fix app error messages (#13852)
* Summary
Fixing inconsistencies of the app_error messages. Now all end with a 
period and there is the same user experience all over the system. 

We agreed on this change on the community server a while ago and I just 
came around finally doing it.


Ticket Link
N/A

Checklist
 Includes text changes and localization file (.../i18n/en.json and 
.../webapp/i18n/en.json) updates

* Update i18n/en.json

Approving changes

Co-Authored-By: Jason Blais <13119842+jasonblais@users.noreply.github.com>

* Update i18n/en.json

Approving changes

Co-Authored-By: Jason Blais <13119842+jasonblais@users.noreply.github.com>

* Update i18n/en.json

Approving changes

Co-Authored-By: Jason Blais <13119842+jasonblais@users.noreply.github.com>

* Fix test failures due string differences

Co-authored-by: Jason Blais <13119842+jasonblais@users.noreply.github.com>
Co-authored-by: Miguel de la Cruz <mgdelacroix@gmail.com>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-04 20:14:23 +01:00
Jesús Espino
54bb6d10f3 Making InvalidateCacheForUser public again (#13997) 2020-03-04 16:16:27 +01:00
Carlos Tadeu Panato Junior
3c8a5f863b update golang.org/x/crypto (#13930)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-04 16:06:39 +01:00
Eli Yukelzon
73dae67c8e MM-22980 - Revert "server init order fixed" (#13994)
This reverts commit daebc93d90.
2020-03-04 16:25:45 +02:00
Mario de Frutos Dieguez
b162cf92cd MM-21976: Include cache layer to be tested (#13749) 2020-03-04 14:18:03 +01:00
Jesús Espino
c9a0418a32 Making private some methods that are not needed publicly (#13959)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-03 22:43:48 +01:00
Jesús Espino
ccc57e56c3 MM-22235: Fixing stop/start behavior on Elasticsearch engine (#13887)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-04 00:52:59 +08:00
upwell
058f235050 [GH-13931] Return error in parseDSN func #13931 (#13947)
Automatic Merge
2020-03-03 11:42:17 -05:00
Jesús Espino
25e8eb9eef Documenting sql store license methods (#13896)
* Documenting sql store license methods

* Update store/sqlstore/license_store.go

Co-Authored-By: Justine Geffen <justinegeffen@users.noreply.github.com>

* Update store/sqlstore/license_store.go

Co-Authored-By: Justine Geffen <justinegeffen@users.noreply.github.com>

* Update store/sqlstore/license_store.go

Co-Authored-By: Justine Geffen <justinegeffen@users.noreply.github.com>

* Addressing PR comments

* Update store/sqlstore/license_store.go

Co-Authored-By: Justine Geffen <justinegeffen@users.noreply.github.com>

Co-authored-by: Justine Geffen <justinegeffen@users.noreply.github.com>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-03 15:51:18 +01:00
Patryk Pomykalski
662eef6881 Cache test users to speed up testing (#13741)
* Cache test users to speed up testing

Users were recreated for many tests and hashing passwords takes a lot of
time. Testing /api4 takes around 120s less and /app 60s less.

* fix build

* simplifying a bit the approach

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Jesús Espino <jespinog@gmail.com>
2020-03-03 14:19:54 +01:00
Mario de Frutos Dieguez
89f4e9d7ef Metrics operations shouldn't be inside goroutine (#13950)
Metrics operations are done in memory so there is no need
to spawn a new goroutine
2020-03-03 16:23:32 +05:30
Jesús Espino
2a5d30f8f3 Making private some sqlstore methods (#13895)
* Making private some sqlstore methods

* Calling for create indexes on missing stores

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-03 11:45:49 +01:00
Farhan Munshi
aec606500f [MM-22622] Add Ephemeral response when using mentions without permissions (#13902)
* MM-22282 Add Ephemeral response when using mentions without permission and add new prop to disable mention highlights on client

* MM-22622 Make test name test actually what it does and fix comment style

* MM-22622 Check ephemeral post created when post create with mentions on API

* MM-22622 More tests for App>CreatePost

* MM-22622 Make DisableMentionHighlights more concise and rename ephemeral post

* MM-22622 Dont send ephemeral message for system message created by user

* Trigger build
2020-03-03 15:52:49 +05:30
Jason Frerich
4c7fee7ac8 [MM-22485] Sync bot message to message displayed in webapp (#13867)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-02 15:57:29 -06:00
Mario de Frutos Dieguez
a43439dd60 Include deleted channels on get user channels endpoint (#13877) 2020-03-02 22:15:15 +01:00
Elisabeth Kulzer
8f0bedf9c9 Fix triggering the release logic on a branch with release in its name. (#13915)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-02 22:01:45 +01:00
ABDUL SATTAR MAPARA
e6d216090b MM-22679 - Update example plugin documentation (#13944)
Automatic Merge
2020-03-02 14:22:48 -05:00
Mario de Frutos Dieguez
9742a0595d Include Mutex for Hubs property (#13917)
Hubs property has been detected by the race condition detector
as a possible one because we could set the Hubs variable
and also we're spawning goroutines that could ask for that property

This condition is not happening right now because we're setting the
Hubs variable in the server start and after that, we're spawning the
goroutines but the tests are failing because of this and in the future
this could be a problem
2020-03-02 19:37:57 +01:00
Agniva De Sarker
a981099652 Add error checks to TestUnlinkGroupTeam (#13956)
I could not figure out the actual root cause for this.

But a possible area of concern is that somewhere in the call stack,
the login failed partially. Therefore, the session did not have the teamMembers
populated. And the permission check failed therefore.

Adding the error checks should let us know in future if this is not the case.

Another possible case is the app.ClearTeamMembersCache which gets called in
the goroutine app.SyncRolesAndMembership. If this races with the session,
then it's possible that the teamMember is wiped off from the session. But this
happens after the permission check. So it's not very likely.

Also, while we are here, I found that (*app).ClearTeamMembersCache
does not log the internal error bubbled up. This prevents us from understanding
what actually happened. Added that logging.
2020-03-03 00:04:39 +05:30
Jesús Espino
7035e09fe9 Adding changes to separate unit tests and integration tests (#13670)
* Introducing unit (not integration) tests for the app layer

* Initial support for unit tests at the API

* Adding unit tests support to the store layer

* Add unit tests support in commands

* Adding last tests needed for run unit tests properly

* Fixing govet

* Removing some duplication

* Fixing tests

* Fixing tests

* Not compiling test helpers with the main module for api

* Revert "Not compiling test helpers with the main module for api"

This reverts commit 36a199bbe0.

* Fixing tests

* Fixing unit tests

* More consistency between api4/apiteslib.go and app/helper_test.go

* Renaming things to make more obvious the new Setup functions purpose

* Reverting change in go.sum

* Start with empty mock for app layer

* Start with empty mock for api layer

* Start with empty mock for web layer

* Renaming SetupWithStoreMockConfig to SetupConfigWithStoreMock

* Fixing tests on web package

* Removing unnecesary function
2020-03-02 08:13:39 -08:00
JtheBAB
cd36c9f041 MM-22355 - Fix typos detected by goreportcad.com (#13842)
* Update webhook_test.go

* Update user_test.go

* Update filesstore_test.go

* Update plugin_requests.go

* Update syncables.go

* Update helper.go

* Update html_entities.go

* Update user.go

* Update team.go

* Update notification.go

* Update notification_test.go

* Update plugin_api_test.go

* Update post_metadata.go

* Update channel_test.go

* Update database.go

* Update channel.go

* Update user_store.go

* Update team_test.go

* revert andd

* Revert back to infintie

Co-authored-by: Jesús Espino <jespinog@gmail.com>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-02 17:10:40 +01:00
Miguel de la Cruz
98ccc1ccf5 [MM-22572] Modify get user by email route to accept the / character in the email (#13897)
Automatic Merge
2020-03-01 13:52:16 -05:00
Jesús Espino
69b29476d5 Running tests in mysql and postgres (#13832)
* Running tests in mysql and postgres

* Fixing ci

* Fixing ci

* Fixing ci

* Fixing ci

* Fixing ci

* Fixing ci

* Fixing ci

* Fixing ci

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-02-28 18:50:22 +01:00
Mario de Frutos Dieguez
75197d291a Added Websocket's broadcast queue load metrics (#13935)
Now we send metrics when the websocket's broadcast queue receive or
consume a message from the queue
2020-02-28 11:44:56 +01:00
Jesse Hallam
b07898b8a2 Improved kv testing (#13943)
* actually use doer in doTestPluginSaveOrUpdate

* handle off-by-1000ms in KVSetWithOptions
2020-02-27 19:28:14 -04:00
Agniva De Sarker
5c9ffd00ae Clean localcache/layer.go (#13949)
Insert appropriate spaces and comments where needed.
2020-02-27 21:20:46 +05:30
Arianna Vespri
146afedd51 Add output indicating success for restore channel command (#13802)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-02-27 09:15:21 -05:00
Agniva De Sarker
376ec9e0e3 Remove unnecessary struct2interface dependency (#13929)
* Remove unnecessary struct2interface dependency

Running go mod tidy on the repo removes it.
This was also preventing 1.14beta from running the repo,
because now it verifies modules.txt with go.mod, which had a mismatch.

* fixing CI

* Fix test-server target

* Fix some discrepancies in vendor

* Removing the Makefile target for now

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-02-27 09:46:13 +05:30
Agniva De Sarker
2835163b18 Fix flaky TestPlugin (#13933)
- Fix panic due to not checking for JSON marshal error
- Reorder the FileExists check which would fail if we were to pass
the -short flag to bypass the slow URL test. In which case, the plugin
would have been removed from the filesystem.
So we move the check before we remove the plugin.

https://mattermost.atlassian.net/browse/MM-22666
2020-02-26 15:11:24 +01:00