Commit Graph

12305 Commits

Author SHA1 Message Date
catalintomai
b90f4f46e2 MM-23015: Enable or disable group mentions (#14010)
* MM-23015: Enable or disable group mentions + show them in suggestion list


Co-authored-by: Catalin Tomai <catalin.tomai@mattermost.com>
2020-04-13 11:37:29 -07:00
Agniva De Sarker
0bfb063c78 MM-24095: Add IdleTimeout to telemetry (#14279)
Automatic Merge
2020-04-13 11:08:57 -07:00
Mario de Frutos Dieguez
9b6bb2f98d Add some missing methods in time and opentracing layers (#14261) 2020-04-12 11:54:55 +02:00
Martin Kraft
df5cb41a64 MM-23399: Fix for old team schemes that have a blank guest role. (#14267) 2020-04-10 13:58:40 -04:00
Martin Kraft
a9ea3587bb MM-23244: Validate that AuthData and AuthService fields are mutually inclusive. (#14175)
* MM-23244: Validate that either both or neither AuthData and AuthService fields are set.

* MM-23244: Readability improvement.

* MM-23244: Adds translation. Tests for error id.

* MM-23244: Fix test.

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-04-09 18:51:27 -04:00
Harrison Healey
bfb380630c Fix community signup link (#14262) 2020-04-09 09:49:18 -04:00
Mario de Frutos Dieguez
698b4b6934 [MM-23882] Include database server version in telemetry (#14258)
* Add database server version to telemetry

Also added a new query in the store to retrieve the database version

* Add test for the GetDbVersion function

* More drivers in the tests

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-04-09 11:08:15 +02:00
Jesús Espino
f6f025aeaa Adding guest accounts count telemetry (#14257)
* Adding guest accounts count telemetry

* Adding missed methods to the opentracing an timer layer
2020-04-09 11:05:43 +02:00
Eli Yukelzon
819f57037b fixed app-layer generator, re-enabled struct2interface in make… (#14260)
* fixed app-layer generator, re-enabled struct2interface in makefile
2020-04-09 11:08:39 +03:00
Agniva De Sarker
1f036b7588 Jenkins: Make mysql and postgres tests run in parallel (#14026)
* Jenkins: Make mysql and postgres tests run in parallel

* Trying again

* Start docker containers individually for each stage

This is because the parallel block does not allow
a common step to be specified.

* Giving a random name to each docker-compose

* Fixing order of the param

* Giving unique project names for tests

* Setting the right datasource and drivername

* Trying to use different copies of the src directory for test the databas3

* moving workspace copy to the beginning of postgres tests

* Fixing some missed out patches

* Added missing LDAP population command

* Upload all test results

And set allowEmptyResults to false.

* Pruning remaining docker networks

* Remove unneeded settings

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Jesús Espino <jespinog@gmail.com>
2020-04-08 23:22:58 +05:30
Agniva De Sarker
583daeb132 MM-23547: remove redundant JSON parsing in push notifications (#14181)
Automatic Merge
2020-04-08 08:38:57 -07:00
Jesse Hallam
b8eb69281b MM-23666: rewrite TestHTTPClient (#14236)
* MM-23666: rewrite TestHTTPClient

The existing `TestHTTPClient` actually tries to reach google.com, making it brittle to network hiccups (as unlikely as that might be). Rewrite the test to rely on a local server instead.

I found the existing implementation of `TestHTTPClient` extremely hard to understand, so I've rewritten it more explicitly, expanded coverage somewhat. In doing so, I found myself somewhat surprised by the current behaviour. If `allowHost` returns `true`, we ignore `allowIP` altogether. If `allowHost` returns `false`, we check with `allowIP`. There's a lack of symmetry here that the current filter names don't make clear. Perhaps `allowHost` should be renamed `skipIPCheckForHost`?

Fixes: https://mattermost.atlassian.net/browse/MM-23666

* more compact assertions
2020-04-08 09:40:42 -03:00
Ian Whitlock
8084620911 MM-23824 Clean up orphaned plugin process at test completion (#14235)
* Clean up orphaned plugin process at test completion

* Add supervisor shutdown to second test for consistency
2020-04-08 09:35:42 -03:00
George Goldberg
f14c79f170 Dependency updates. (#14033)
* Update dependencies.

* Fix tests.
2020-04-08 10:51:04 +01:00
Agniva De Sarker
ac4153df03 Bump minimum Go version required (#14250)
After the auto-vendoring change, minimum Go version
required is 1.14.

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-04-08 14:28:55 +05:30
Doug Lauder
6a27ed4a1d MM-22785 audit server CLI (#14144)
Add auditing to server CLI.

Also:
- simplify auditing in API layer
- reduce number of AddMeta calls
- have models serialize themselves
- more consistent field naming
2020-04-08 00:52:30 -04:00
Doug Lauder
e2d1af17de MM-23489 Fix poor Unicode handling in display names (#14214)
Filter blacklisted Unicode characters from:

user: first name, last name, nickname, bot description, username
team: name, display name, description, company name
channel: name, display name
2020-04-07 16:56:07 -04:00
Carlos Tadeu Panato Junior
84859e7159 db: prepare db upgrade to 5.23 (#14249) 2020-04-07 17:01:01 +02:00
Agniva De Sarker
4210ae60ab MM-22056: Fix flaky test TestUploadFiles (#14234)
Automatic Merge
2020-04-07 07:09:04 -07:00
Carlos Tadeu Panato Junior
22a00104bd update dockerfile to use the latest release available (#14232)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-04-07 15:25:15 +02:00
Ben Schumacher
8e858d4dc3 Bump app version to 5.24 (#14220) 2020-04-07 15:22:54 +02:00
Eli Yukelzon
92380d83fc OpenAPI Mattermost Go Vet integration (#14216)
* integrating govet
* added api spec check to CI
2020-04-07 16:18:08 +03:00
Martin Kraft
f92d3fa518 MM-23876: Fix for patching channel moderations with a null team schem… (#14239)
* MM-23876: Fix for patching channel moderations with a null team scheme channel guest role.

* MM-23876: Tests the moderations response.
2020-04-07 17:35:03 +05:30
Elisabeth Kulzer
27a0ff4d52 Revert "Updating to use a different network cidr for the mm-test network (#14223)" (#14247)
This reverts commit d1d1e3d27a.
2020-04-07 12:57:24 +02:00
Carlos Tadeu Panato Junior
ff505457a2 db: upgrade db to 5.22 (#14224)
* db: upgrade db to 5.22
2020-04-07 11:44:08 +02:00
Agniva De Sarker
6f6793e579 MM-22782: Write fuzz testing for websockets (#14203)
* MM-22782: Write fuzz testing for websockets

* Moved to fuzz function

* incorporate review comments

* Remove some more comments
2020-04-07 14:41:55 +05:30
Agniva De Sarker
125ed3a4cf MM-23817: Use auto-vendoring (#14231)
With Go 1.14 (https://tip.golang.org/doc/go1.14#go-command),
it now automatically checks for the existence of a vendor directory.

So there is no need to explicitly set the `-mod=vendor` flag.
2020-04-07 09:01:06 +05:30
Jason Paul Deland
d1d1e3d27a Updating to use a different network cidr for the mm-test network (#14223)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-04-06 12:45:11 -04:00
Agniva De Sarker
ef209c5e0b MM-23620: Handle error from GetUser (#14204)
* MM-23620: Handle error from GetUser

In case of high DB load, the DB will start to throw errors.
Unless we handle the error appropriately, the server will crash.

* Removing unnecessary lines
2020-04-06 21:07:30 +05:30
Elisabeth Kulzer
6fef02ed6c IS-255: Rm trigger for now. (#14183)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-04-03 16:56:38 +02:00
Ibrahim Serdar Acikgoz
f88e6f5fe0 [MM-21209] Add LRU cache for userstore.get (#13838)
* store/cache: add cache to user layer

* store/localcache: fix cache data access

* app/login: add invalidate for user

* store/user: move user sanitization to cache

* store/user: remove sanitize checks form the tests

* store/user: remove sanitization from store and cache layer

* store/cache: remove unnecessary error wraps

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-04-03 12:25:45 +03:00
Jesús Espino
1e89ef7314 Fixing searching users without team (#14182) 2020-04-02 17:22:08 +02:00
Martin Kraft
19cf3e85b7 MM-23770: Fix for blank DefaultChannelGuestRole on team schemes. (#14210)
* MM-23770: Fix for blank DefaultChannelGuestRole on team schemes.

* MM-23770: Adds test for an team scheme with a blank DefaultChannelGuestRole field.

* MM-23770: Fix for unexpected Schemes.Get.
2020-04-02 08:09:23 -04:00
Eli Yukelzon
c0fc6c13d5 Opentracing layer generator improvement (#14206) 2020-04-02 11:33:43 +03:00
Jesse Hallam
e276a2bf57 avoid panic in search channel store (#14209)
Seen in Kibana logs:
```
http: panic serving 10.128.148.31:47536: runtime error: invalid memory address or nil pointer dereference
goroutine 1063732 [running]:
net/http.(*conn).serve.func1(0xc0042995e0)
	net/http/server.go:1767 +0x139
panic(0x1a4e080, 0x2edab40)
	runtime/panic.go:679 +0x1b2
github.com/mattermost/mattermost-server/v5/model.(*AppError).Error(0x0, 0xc000032000, 0x1a618c0)
	github.com/mattermost/mattermost-server/v5@/model/utils.go:73 +0x26
go.uber.org/zap/zapcore.encodeError(0x1c9c1d0, 0x5, 0x1fd6cc0, 0x0, 0x2028c60, 0xc0089fdfb0, 0xc0089fdfb0, 0x34)
	go.uber.org/zap@v1.13.0/zapcore/error.go:46 +0x3b
go.uber.org/zap/zapcore.Field.AddTo(0x1c9c1d0, 0x5, 0x19, 0x0, 0x0, 0x0, 0x1aa8c00, 0x0, 0x2028c60, 0xc0089fdfb0)
	go.uber.org/zap@v1.13.0/zapcore/field.go:165 +0xc8c
go.uber.org/zap/zapcore.addFields(0x2028c60, 0xc0089fdfb0, 0xc001df0800, 0x2, 0x2)
	go.uber.org/zap@v1.13.0/zapcore/field.go:199 +0xcf
go.uber.org/zap/zapcore.(*jsonEncoder).EncodeEntry(0xc00167d200, 0x2, 0xbf9678532a114ede, 0x3d02c134ae83, 0x2f284e0, 0x0, 0x0, 0x1d0d926, 0x2a, 0x1, ...)
	go.uber.org/zap@v1.13.0/zapcore/json_encoder.go:376 +0x1e4
go.uber.org/zap/zapcore.(*ioCore).Write(0xc00167d230, 0x2, 0xbf9678532a114ede, 0x3d02c134ae83, 0x2f284e0, 0x0, 0x0, 0x1d0d926, 0x2a, 0x1, ...)
	go.uber.org/zap@v1.13.0/zapcore/core.go:86 +0xa9
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc004b2e8f0, 0xc001df0800, 0x2, 0x2)
	go.uber.org/zap@v1.13.0/zapcore/entry.go:216 +0x117
go.uber.org/zap.(*Logger).Error(0xc001fc4f00, 0x1d0d926, 0x2a, 0xc001df0800, 0x2, 0x2)
	go.uber.org/zap@v1.13.0/logger.go:203 +0x7f
github.com/mattermost/mattermost-server/v5/mlog.(*Logger).Error(...)
	github.com/mattermost/mattermost-server/v5@/mlog/log.go:184
github.com/mattermost/mattermost-server/v5/store/searchlayer.(*SearchChannelStore).SaveMember(0xc0009749c0, 0xc003e69a40, 0x2f284e0, 0x0)
	github.com/mattermost/mattermost-server/v5@/store/searchlayer/channel_layer.go:84 +0x2ed
github.com/mattermost/mattermost-server/v5/store.(*TimerLayerChannelStore).SaveMember(0xc000974ae0, 0xc003e69a40, 0xc000974ae0, 0xc)
	github.com/mattermost/mattermost-server/v5@/store/timer_layer.go:1559 +0x66
github.com/mattermost/mattermost-server/v5/app.(*App).createGroupChannel(0xc006d54f00, 0xc0085d1740, 0x3, 0x4, 0xc00a7396a0, 0x1a, 0xc008a319a0, 0xc0089fd2c0)
	github.com/mattermost/mattermost-server/v5@/app/channel.go:431 +0x6de
github.com/mattermost/mattermost-server/v5/app.(*App).CreateGroupChannel(0xc006d54f00, 0xc0085d1740, 0x3, 0x4, 0xc00a7396a0, 0x1a, 0x0, 0x1710a4516c7)
	github.com/mattermost/mattermost-server/v5@/app/channel.go:371 +0x81
github.com/mattermost/mattermost-server/v5/api4.createGroupChannel(0xc0085d1640, 0x1ff94a0, 0xc003e698f0, 0xc004dfae00)
	github.com/mattermost/mattermost-server/v5@/api4/channel.go:551 +0x437
github.com/mattermost/mattermost-server/v5/web.Handler.ServeHTTP(0xc0050cb6f0, 0x1da88d8, 0x2b0cb00, 0x12, 0x10001, 0x0, 0x0, 0x1ff94a0, 0xc003e698f0, 0xc004dfae00)
	github.com/mattermost/mattermost-server/v5@/web/handlers.go:212 +0x1e9a
github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1ff8f20, 0xc0060882a0, 0xc004dfae00)
	github.com/NYTimes/gziphandler@v1.1.1/gzip.go:336 +0x23f
net/http.HandlerFunc.ServeHTTP(0xc004024120, 0x1ff8f20, 0xc0060882a0, 0xc004dfae00)
	net/http/server.go:2007 +0x44
github.com/gorilla/mux.(*Router).ServeHTTP(0xc001680180, 0x1ff8f20, 0xc0060882a0, 0xc004dfac00)
	github.com/gorilla/mux@v1.7.3/mux.go:212 +0xe2
net/http.serverHandler.ServeHTTP(0xc00140c540, 0x1ff8f20, 0xc0060882a0, 0xc004dfac00)
	net/http/server.go:2802 +0xa4
net/http.(*conn).serve(0xc0042995e0, 0x1ffeee0, 0xc0063ee780)
	net/http/server.go:1890 +0x875
created by net/http.(*Server).Serve
	net/http/server.go:2927 +0x38e
```
2020-04-01 16:40:12 -03:00
Eli Yukelzon
52c92d6659 MM-23185 - Markdown image hosted by plugins are not shown if l… (#14185)
* avoid image proxy for local images

* added test for local images
2020-04-01 11:45:26 +03:00
Michael Kochell
005cc00ccc [MM-23281] Persist registeredPlugin in plugin.Environment when plugin is deactivated (#14110)
* store failed timestamps on health check job instead of on registeredPlugin

Update test

* change EnsurePlugin calls

* Make env.SetPluginState private

* Write test for plugin deactivate and PluginStateFailedToStayRunning

* Add license comment

* adjust comments, use time.Since

* Additional PR feedback:

time.Since cleanup
test cleanup
remove duplicate .Store() call

* PR Feedback

- Add test case for reactivating the failed plugin
- Change `crashed` to `healthy` and `hasPluginCrashed` to `isPluginHealthy`
- remove stale timestamps from health check job

* Keep registeredPlugins in env when plugin is deactivated, so the crashed state of a plugin can be persisted.

* PR feedback

* PR feedback from Jesse

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-31 21:20:22 -03:00
Doug Lauder
8e5b626854 MM-23747 fix Sprintf linter errors (#14196)
- replace all instances of `fmt.Sprintf("whatever")` with simply "whatever".

- replace all instances of `fmt.Println(fmt.Sprintf("whatever %d", i))` with `fmt.Printf("whatever %d\n", i)`
2020-03-31 16:50:06 -04:00
Fedor Vitkovskiy
896c3f736d MM-23484 - add simple validation to getUserStatusesByIds (#14158)
* MM-23484 - add simple validation to GetUsersStatusesByIds

* MM-23484 - return early in validation, add extra test case

* MM-23484 - add test case for empty array

Co-authored-by: Fedor Vitkovskiy <fedorvitkovskiy@pop-os.localdomain>
2020-03-31 20:28:26 +02:00
Doug Lauder
c82c2db8ce Update Makefile (#14195)
`make searchengine-mocks` was failing due to not finding `mockery`, although it was installed.
2020-03-31 12:11:32 -04:00
Jesús Espino
07c1e76b02 Adding team_name to post props for channel mentions (#14150)
* Adding team_name to post props for channel mentions

* Fixing tests

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-31 15:25:53 +02:00
Farhan Munshi
ab338e8417 [MM-23019] Create use_group_mentions permission and migrate existing roles (#13987)
* MM-22962 Create use_group_mentions permission and give to all non guest roles that can create post

* Add use_group_mentions to team admin role for test

* Trigger CI

* MM-22962 Remove old migration keys
2020-03-30 23:08:48 -04:00
Federico Martín Alconada Verzini
bf1d9b0ae9 [GH-14136] Document minimum server version required for plugin hooks (#14153)
* Document minimum server version required for plugin hooks

* change mins < v5.2 for v5.2

Co-authored-by: Federico Martín Alconada Verzini <fedealconada@gmail.com>
2020-03-30 11:22:09 -07:00
catalintomai
59e92e9ae1 MM-4887: Remove back to MM button for Zapier-MM OAuth integration setup (#13890)
* MM-4887: Remove back to MM button got Zapier OAuth setup

* MM-4887: remove Zapier dependency

Co-authored-by: Catalin Tomai <catalin.tomai@mattermost.com>
2020-03-30 11:19:15 -07:00
Jesse Hallam
f149ada16a MM-23261 plugin stderr debug logs (#14166)
* explicitly assert panic as error log

* Revert "[MM-18150] plugin panic trace should not be lost (#13559)"

This reverts commit 5d928b4f94, while leaving the unit tests intact
and now asserting debug logs instead.

* missing license header
2020-03-30 15:00:45 -03:00
Jesús Espino
383e45b13d Adding correctness in the ReplyCount generation (#14047)
* Adding correctness in the ReplyCount generation

* Applying suggestion from reflog

* More reliable reply count generation

* Some tests fixed

* Adding i18n translation

* Fixing reply count on save behavior

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-30 19:30:30 +02:00
Miguel de la Cruz
4fe25b1cdd [MM-21551] Add search tests structure to test the search engines (#14031)
* WIP

* Adding bleve to go modules

* WIP

* Adding missing files from searchengine implementation

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* User and channel indexing and searches implemented

* Make bleve tests run with in-memory indexes

* Implement post index and deletion tests

* Initial commits for the search layer

* Removing unnecesary indexing

* WIP

* WIP

* More fixes for tests

* Adding the search layer

* Finishing the migration of searchers to the layer

* Removing unnecesary code

* Allowing multiple engines active at the same time

* WIP

* Add simple post search

* Print information when using bleve

* Adding some debugging to understand better how the searches are working

* Making more dynamic config of search engines

* Add post search basics

* Adding the Purge API endpoint

* Fixing bleve config updates

* Adding missed file

* Regenerating search engine mocks

* Adding missed v5 to modules imports

* fixing i18n

* Fixing some test around search engine

* Removing all bleve traces

* Cleaning up the vendors directory and go.mod/go.sum files

* Regenerating timer layer

* Adding properly the license

* Fixing govet shadow error

* Fixing some tests

* Fixing TestSearchPostsFromUser

* Fixing another test

* Fixing more tests

* Fixing more tests

* Removing SearchEngine redundant text from searchengine module code

* Fixing some reindexing problems in members updates

* Fixing tests

* Addressing PR comments

* Reverting go.mod and go.sum

* Addressing PR comments

* Fixing tests compilation

* Fixing govet

* Adding search engine stop method

* Being more explicit on where we use includeDeleted

* Adding GetSqlSupplier test helper method

* Mocking elasticsearch start function

* Fixing tests

* Search tests

* Fix tests

* Fix mod

* Fixing searchEngine for test helpers with store mocks

* Remove loglines

* Fix i18n strings

* Migrate search posts tests

* Fix linter

* Do not run search tests if -short flag is enabled

* Migrate back store tests that didn't belong to the searchlayer

* Fix scopelint issues

Co-authored-by: Jesús Espino <jespinog@gmail.com>
2020-03-30 19:17:40 +02:00
Agniva De Sarker
2b1b001bcc MM-23503: Fix race in websocket_client writer (#14160)
* MM-23503: Fix race in websocket_client writer

We create a separate writer goroutine where all the writes happen.

* Fixing the case of double close

* Incorporate review comments

* Use CAS

* Fix incorrect comment

* Check if client is closed in pingHandler too

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-30 21:09:52 +05:30
Hossein Ahmadian-Yazdi
bd2c1f4522 [MM-21517] Team Member Manage User Modal not display correct roles (#14043)
* join on the users table and order by username

* don't inclue users who have been deleted

* Address PR comments

* update tests

* fix linting

* fix linting

* include ExcludeDeletedUsers flag

* fix gofmt

* fix nil teamMembersGetOptions

* fix gofmt error

* Add Unit Tests

* fix gofmt bugs

* partially address comments

* fix incorrect import

* Address Comments and fix golint errors

* store mocks

* address PR comments about tests and styling

* Update model/team_member.go

Co-Authored-By: Jesse Hallam <jesse.hallam@gmail.com>

* Address PR comments

* update client function name

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
2020-03-30 10:48:03 -04:00
Agniva De Sarker
7ce68e89d7 MM-22044: Fix panic on web_conn send hello (#13799)
* MM-22044: Fix panic on web_conn send hello

(*Hub).Start is the central place for sending all web connection
related traffic. However, there was this one call to (*WebConn).Hello
which tried to send a message to a webconn separately.

This was a rare case, but it did occur under stress conditions generated
from a load test.

When the websocket send SEND_QUEUE_SIZE would get filled up and we would
attempt to make a broadcast, the non-blocking send would close the Send
channel of the web connection. During that time, if a web connection
would try to perform a broadcast, it would try to send to a closed channel
and cause a panic.

The solution is to bring back the sending of hello into the same goroutine
inside (*Hub).Start so that all state is centralised and we avoid
sending to a closed channel by sending the hello message inside the registering
code itself.

* Adding non-blocking send

* Simplify things

* Remove test

* Bring sendHello back

* Improve code further

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-30 18:12:40 +05:30