Commit Graph

3333 Commits

Author SHA1 Message Date
Farhan Munshi
101c6c7c01 [MM-27623] Add new session prop for oauth (#15221)
* Add new session prop for oauth

* Make it isOAuthUser to differentiate better

* Fix up caps

* Fix tests

* Add tests for IsOAuthUser
2020-08-11 10:24:26 -04:00
Doug Lauder
7f64199a37 MM-27184 deprecate model.SetExpireInDays (#15165)
Mobile users were having their sessions unexpectedly expired, despite having ServiceSettings.ExtendSessionLengthWithActivity enabled. 

Every time a mobile app is opened it called `/api/v4/sessions/device` which calls attachDeviceId which calls `(*Session)SetExpireInDays`. This code above assumed the expiry should be relative to CreateAt which is incorrect when ExtendSessionLengthWithActivity is enabled. Therefore, every time the mobile app was opened, the maximum expiry was set in memory to CreateAt + session_length, even if the session was extended.

(*Session)SetExpireInDays is now deprecated and replaced with (*App)SetSessionExpireInDays which takes into account the ExtendSessionLengthWithActivity setting.
2020-08-04 16:10:37 -04:00
Scott Bishel
c65a0e9f9c Remove back to mm button with incorrect callback url (#15001)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-07-28 19:36:19 -06:00
Agniva De Sarker
50fa5f0f3e MM-27275: Log the correct error when GetUser fails in MFA Authentication (#15115)
* MM-27275: Log the correct error when GetUser fails in MFA Authentication

We were not logging the error returned from GetUser which makes debugging things
very hard in case of logout issues like https://mattermost.atlassian.net/browse/MM-27270.

We fix this by appending the error inside model.AppError and setting a proper error string
which says exactly what has happened.

* Update web/context.go

Co-authored-by: Doug Lauder <wiggin77@warpmail.net>

Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-07-28 14:06:41 +05:30
Jesús Espino
2de5f75f47 Moving store layers into independent packages (#15004) 2020-07-27 15:14:16 +02:00
catalintomai
549e5b57cd Add metric warning support (announcement bar and DM) (#14483)
* Admin. Advisory: Add warning for number of active users metric status

Co-authored-by: Catalin Tomai <catalin.tomai@mattermost.com>
2020-07-22 20:32:21 -07:00
Doug Lauder
56fb31f06f MM-22784 Advanced logging config for audit (#15076)
Adds the advanced logging config for audit. Existing support for auditing to a single file remains for E0 and E10 licenses instances, and a new config item ExperimentalAuditSettings.AdvancedLoggingConfig is added that behaves like LogSettings.AdvancedLoggingConfig.

Supported destinations:

- file
- syslog (with out without TLS)
- raw TCP socket (with out without TLS)

ExperimentalAuditSettings.AdvancedLoggingConfig can contain a filespec to a config file, a database DSN, or JSON.

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
2020-07-22 18:48:46 -04:00
Siyuan Liu
c7f7bef9ec move cache2 package to cache (#14921)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-07-18 13:31:06 +05:30
Rodrigo Villablanca
93a537a636 OAuthStore migration (#15013)
Automatic Merge
2020-07-17 12:56:08 +02:00
Eli Yukelzon
b15c0484c6 MM-26834 - Move opentracing into a separate package (#15011) 2020-07-15 12:59:11 +03:00
Jesús Espino
b977017ca7 Extracting email functions into a service (#14802)
* Extracting email functions into a service

* Fixing two shadowing errors

* Address PR review comments

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-07-07 10:03:21 +02:00
Harrison Healey
de6a57cdc3 Sidebar V2 Phase 2 (#14967)
* parent 48256721c4 (#14358)

author Eli Yukelzon <reflog@gmail.com> 1585814774 +0300
committer Eli Yukelzon <reflog@gmail.com> 1589111022 +0300

Sidebar caregories implemented

Apply suggestions from code review

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

Update store/sqlstore/channel_store.go

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

Update store/sqlstore/channel_store.go

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

code review suggestions

status messages

edge case

bugs...

timeout reverse

* MM-25126 Add a separate default sorting method for categories (#14575)

* MM-25158 Add user to initial sidebar categories when joining team (#14570)

* MM-25281  Place new categories in the correct position (#14609)

* MM-25277  Return channels that aren't in a category as part of the Channels/Direct Messages categories (#14601)

* MM-25276  Remove categories when leaving a team (#14600)

* Remove categories when leaving a team

* layers

* corrected cleanup function

* lint

* .

* corrected errors in postgres

* .

* MM-25280  Ensure that the "update category order" API call only contains real category IDs and isn't missing any IDs (#14626)

* Ensure that the "update category order" API call only contains real category IDs and isn't missing any IDs

* tests

* correct status code

* MM-25278  Change "update category" API to return 400 when changing unsupported fields (#14599)

* MM-25279  Change category migration to only populate channels in Favorites (#14627)

* MM-25157 Add API to delete custom categories  (#14574)

* MM-25157 Add API to delete custom categories

* get categories fix

* maxorder fix

* Use correct websocket event when deleting category

* Fix tests and remove debug code

* Actually use the right websocket event this time

* test cleanup

* Update test for new category order

Co-authored-by: Eli Yukelzon <reflog@gmail.com>

* MM-24914 Various fixes for sidebar channel handling (#14756)

* Fix checking for channel membership when reordering channels

* Remove unique constraint on SidebarCategories

* Set column sizes for SidebarCategories and SidebarChannels tables

* Allow changing the sorting method for non-DM categories

* Fix nil pointers in error handling

* Fix orphaned channels from other team being returned in Channels category

* Fix non-orphaned channels being duplicated in the Channels category

* Remove unique constraint on SidebarChannels

* Fix category/name of favorites preferences

* Fix testSidebarChannelsMigration

* Rename err to nErr and appErr to err

* Fix channel order returned by GetSidebarCategories on MySQL

* Fix adding/removing favorites preferences

* Remove leftover TODO

* Change SidebarCategoryType enums to use full names (#14786)

* Change SidebarCategoryType enums to use full names

* Fix Channels constant

* Remove leftover debug code

* MM-24914 Fix updateCategory endpoint returning the wrong type (#14795)

* MM-24914 Make some changes to UpdateSidebarCategories (#14806)

* Fix orphaned DMs not always being returned

* MM-24914 Make some changes to UpdateSidebarCategories

* Run updateSidebarCategoryOrderT in a transaction

* Fix deleting SidebarChannels based on order of arguments to UpdateSidebarCategories

* bump for api testing

* bump for api testing

* Change CreateInitialSidebarCategories to return a plain error

* Change MigrateSidebarCategories to return a plain error

* Remove usage of UpdateColumns when updating sidebar categories (#14843)

* Remove usage of UpdateColumns when changing category order

* Add a random test case

* Remove usage of UpdateColumns when updating sidebar categories (#14843)

* Remove usage of UpdateColumns when changing category order

* Add a random test case

* Remove usage of UpdateColumns when updating sidebar categories (#14843)

* Remove usage of UpdateColumns when changing category order

* Add a random test case

* MM-26343 Make CreateInitialSidebarCategories idempotent (#14870)

* Fix bad merge

* Fix another bad merge

* Fix unintentionally removed i18n string

Co-authored-by: Eli Yukelzon <reflog@gmail.com>
2020-07-06 18:20:35 -04:00
Hossein Ahmadian-Yazdi
4c50c7c59b [MM-25780] Fix incorrect session length when logging in through mobile using SSO (#14874)
* Pass device ID

* dont use device id as way of detecting

* fix spelling mistake

* update layers

* fix test

* fix linting

* save schema

* put columns in correct place

* fix linting

* update

* upgrade go change

* use props

* fix stuff

* update session tests

* address PR comments

* address PR comments
2020-06-30 10:34:05 -04:00
Rodrigo Villablanca
0118db9d23 Migration of AuditStore to return plain errors (#14825)
Automatic Merge
2020-06-26 06:26:35 +02:00
Siyuan Liu
aaea36a24d MM-25115 - make cacheProvider2 accessible to localcachelayer (#14691)
* make cacheProvider2 accessible to localcachelayer

* migrate localcachelayer

* revert test.sh

* workaround gob deserialization issue

* fix web_test.go

* merge conflicts

* resolve merge conflicts

* revert back test workaround

* fix the wrong merge
2020-06-25 17:08:53 +05:30
George Goldberg
e5addef19b MM-23574: Remove Segment. (#14712)
* MM-23574: Eliminate Segment.

* Update dependencies.

* Fix review suggestions.
2020-06-22 08:35:03 +01:00
Mario de Frutos Dieguez
4a974eabea Include response log trace (#14841)
Include the status_code in the log trace
2020-06-19 16:34:49 +02:00
Martin Kraft
2bb6071f73 MM-26065: Reverts Brotli encoding. (#14834)
* MM-26065: Reverts Brotli encoding.

* MM-26065: Removes package reference.
2020-06-16 18:15:31 -04:00
Jesús Espino
f5eab1271b Removing all FakeApp usages (#14174)
* Removing some other fake apps

* More FakeApp removed

* Removing entirely FakeApp

* Fixing some tests

* Fixing get Cluster id from get plugin status

* Fixing failing tests

* Fixing tests

* Fixing test initialization for web

* Fixing InitServer for server tests

* Fixing InitServer for server tests

* Reverting go.sum and go.mod

* Removing unneded HTMLTemplates function in App layer

* Moving back some functions to its old place to easy the review

* Moving back some functions to its old place to easy the review

* Using the last struct2interface version

* Generating store layers

* Fixing merge problems

* Addressing PR comments

* Small fix

* Fixing app tests build

* Fixing tests

* fixing tests

* Fix tests

* Fixing tests

* Fixing tests

* Fixing tests

* Moving license to server struct

* Adding some fixes to the test compilation

* Fixing cluster and some jobs initialization

* Fixing some license tests compilation problems

* Fixing recursive cache invalidation

* Regenerating app layers

* Fix test compilation

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-06-12 13:43:50 +02:00
Mario de Frutos Dieguez
beadeaf8b5 [MM-25677] Content-Type is optional (#14705)
* Content-Type is optional

mime.ParseMimeType returns and "no media type" error if the passed
string is empty.

Given that the Content-Type header is optional we shouldn't return an error
in that case, so we're fixing that allowing the users to call the webhook
without passing that header

* Include webhook id in the error message

Given that the number of webhooks could be big the user could
need the id to check which one of the multiple webhooks are failing
so include the id aids in that part
2020-06-01 21:07:02 +02:00
Martin Kraft
c529d5190a MM-25040: Restrict associated groups to channels when team is group-constrained. (#14619)
* MM-25040: Only return team-associated groups if the team is group-constrained.
MM-25040: Prevents associating a group to a channel if the team doesn't have the group first.

* MM-25040: Fix lints.

* MM-25040: Still add the groupteam if the team is not group-constrained.

* MM-25040: Wraps groupteam upsert in else branch for efficiency.

* MM-25040: Removes unnecessary page iteration.

* MM-25040: Fix typo.

* MM-25040: Moves filtering to SQL.

* MM-25040: Updates tests, check pagination.

* MM-25040: Fix lint error.

* MM-25040: Adds some more group store tests.

* MM-25040: Fix for wrong test parameter.
2020-05-29 10:46:52 -04:00
Martin Kraft
e8daab6b84 MM-25369: Disable Brotli in 'dev' mode. (#14608) 2020-05-20 19:05:21 -04:00
Miguel de la Cruz
0d89ff5d0e Mm 23710 mmctl local mode (#14561)
* [MM-24146] Add unix socket listener for mmctl local mode (#14296)

* add unix socket listener for mmctl local mode

* add a constant for local-mode socket path

* reflect review comments

* [MM-24401] Base approach for Local Mode (#14333)

* add unix socket listener for mmctl local mode

* First working PoC

* Adds the channel list endpoint

* Add team list endpoint

* Add a LocalClient to the api test helper and start local mode

* Add helper to test with both SystemAdmin and Local clients

* Add some docs

* Adds TestForAllClients test helper

* Incorporating @ashishbhate's proposal for adding test names to the helpers

* Fix init errors after merge

* Adds create channel tests

* Always init local mode to allow for enabling-disabling it via config

* Check the RemoteAddr of the request before marking session as local

* Mark the request as errored if it's local and the origin is remote

* Set the socket permissions to read/write when initialising

* Fix linter

* Replace RemoteAddr check to ditch connections with the IP:PORT shape

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

* Fix translations order

* [MM-24832] Migrate plugin endpoints to local mode (#14543)

* [MM-24832] Migrate plugin endpoints to local mode

* Fix client reference in helper

* [MM-24776] Migrate config endpoints to local mode (#14544)

* [MM-24776] Migrate get config endpoint to local mode

* [MM-24777] Migrate update config endpoint to local mode

* Fix update config to bypass RestrictSystemAdmin flag

* Add patchConfig endpoint

* MM-24774/MM-24755: local mode for addLicense and removeLicense (#14491)

Automatic Merge

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Ashish Bhate <bhate.ashish@gmail.com>
2020-05-19 18:20:41 +02:00
Martin Kraft
d7cb890f34 MM-25095: Handles static JS and CSS using Brotli encoding. (#14524)
* MM-25095: Handles static JS and CSS files using Brotli.

* MM-25095: Linting fix.

* MM-25095: Adds missing license.

* MM-25095: Moves initialization of slice.

* MM-25095: Moves initialization of slice.
2020-05-19 08:40:13 -04:00
Agniva De Sarker
624980ff54 MM-25005: Remove HttpRequestsDuration bucket (#14520)
* MM-25005: Remove HttpRequestsDuration bucket

The ApiTimesHistograms already captures what HttpRequestsDuration does.
And it's more granular than that. So there's no need to duplicate stuff.
Let's remove this one.

* Updating resource class to xlarge

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Jason Deland <jaydeland@gmail.com>
2020-05-14 10:53:55 +05:30
Shibasis Patel
882b0324b5 MM_22682_Centralize_ID_Validation (#14237)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-05-07 13:27:35 -04:00
Doug Lauder
5e59b5f70c MM-23935 extend session expiry on user activity (#14275)
* MM-23935  extend session expiry on user activity

- if user types anything before a session expires the session will be extended to now + session length

- ensures new session expiries are not written to DB too frequently

- new session store func for updating session ExpiresAt

- session length defaults for mobile and web/ldap changed from 180 days to 30 days
2020-05-06 15:41:10 -04:00
Eli Yukelzon
7800116429 MM-23093 Implement Server Setup telemetry - server configuration (#14374)
* added advanced first day diagnostics reporting

* typo

* config corrected

* defaults

* moved from config to system db table

* missing file

* added error handling

* tests

* typos

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-04-30 18:18:12 +03:00
George Goldberg
6cabc40e62 MM-23568: Add rudder to server diagnostics. (#14151)
* MM-23568: Add rudder to server diagnostics.

* Add unit test.

* Go mod tidy.

* CSP Header fix.

* Fix review comments.

* Update web/handlers.go

Co-Authored-By: Jesse Hallam <jesse.hallam@gmail.com>

* Partially address review comments.

* fix tests.

* Finish implementing review suggestions and then fixing tests.

* Fix CSP Header tests.

Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
2020-04-21 09:23:00 +01:00
Mario de Frutos Dieguez
aafea55976 MM-23131 Include HTTP status code in the metrics (#14240)
* ResponseWriter wrapper to get status code

For our metrics, we need the status code returned by a request
so this wrapper includes a new method StatusCode() that includes
the desired code

* Shadow the responsewriter variable in the handlers

In order to avoid confusion to people deciding what variable to use.
I've also changed the tests to reflect this change and added a new
one that checks the Flush method works
2020-04-14 14:15:00 +02:00
catalintomai
b90f4f46e2 MM-23015: Enable or disable group mentions (#14010)
* MM-23015: Enable or disable group mentions + show them in suggestion list


Co-authored-by: Catalin Tomai <catalin.tomai@mattermost.com>
2020-04-13 11:37:29 -07:00
Doug Lauder
6a27ed4a1d MM-22785 audit server CLI (#14144)
Add auditing to server CLI.

Also:
- simplify auditing in API layer
- reduce number of AddMeta calls
- have models serialize themselves
- more consistent field naming
2020-04-08 00:52:30 -04:00
catalintomai
59e92e9ae1 MM-4887: Remove back to MM button for Zapier-MM OAuth integration setup (#13890)
* MM-4887: Remove back to MM button got Zapier OAuth setup

* MM-4887: remove Zapier dependency

Co-authored-by: Catalin Tomai <catalin.tomai@mattermost.com>
2020-03-30 11:19:15 -07:00
Doug Lauder
16b535314d MM-23222 mirror audit logs to file (#14062)
* MM-23222 add file target (with rotation) to audit

* MM-23222 mirror syslog audits to local filesystem

    * provides config options for file name, max size, max age

    * rotates files based on max size and max age; delete as needed based on max backups

* include cluster id in log records

* sort meta data fields
2020-03-17 16:12:56 -04:00
Agniva De Sarker
42081a1441 Remove unncessary recover() calls (#14066)
* Remove unncessary recover() calls

A recover which is not called inside a defer will always return nil.
So there's no use of calling it.

And even if we did call it inside a defer, recovering and immediately
panicking does not make sense either. So we just remove it.

* Fixed another instance
2020-03-17 11:04:39 +00:00
Doug Lauder
4ac0619c90 MM-22273 New auditing system (phase 1) (#13967)
* New auditing API outputting to syslog via TLS

* New config section for specifying remote syslog server IP, port, and cert.

* Legacy audit API retained for access history feature
2020-03-12 15:50:21 -04:00
Mario de Frutos Dieguez
c8a923d9e3 MM-22247: Space in content-type breaks integration with 3rd party apps (#13839) 2020-03-07 09:43:57 +01:00
Miguel de la Cruz
182c29b456 MM-21898: Part 2. Add opentracing (#13904)
* initial implementation of opentracing

* app layer

* Revert Makefile

* .

* cleanup

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* [ci]

* autogenerate interface

* .

* missed vendor files

* updated interfaces

* updated store layers

* lint fixes

* .

* finishing layer generators and nested spans

* added errors and b3 support

* code review

* .

* .

* fixed build error due to misplased flag.Parse()

* code review addressed
2020-03-05 14:46:08 +01:00
Mario de Frutos Dieguez
b162cf92cd MM-21976: Include cache layer to be tested (#13749) 2020-03-04 14:18:03 +01:00
Mario de Frutos Dieguez
a43439dd60 Include deleted channels on get user channels endpoint (#13877) 2020-03-02 22:15:15 +01:00
Jesús Espino
7035e09fe9 Adding changes to separate unit tests and integration tests (#13670)
* Introducing unit (not integration) tests for the app layer

* Initial support for unit tests at the API

* Adding unit tests support to the store layer

* Add unit tests support in commands

* Adding last tests needed for run unit tests properly

* Fixing govet

* Removing some duplication

* Fixing tests

* Fixing tests

* Not compiling test helpers with the main module for api

* Revert "Not compiling test helpers with the main module for api"

This reverts commit 36a199bbe0.

* Fixing tests

* Fixing unit tests

* More consistency between api4/apiteslib.go and app/helper_test.go

* Renaming things to make more obvious the new Setup functions purpose

* Reverting change in go.sum

* Start with empty mock for app layer

* Start with empty mock for api layer

* Start with empty mock for web layer

* Renaming SetupWithStoreMockConfig to SetupConfigWithStoreMock

* Fixing tests on web package

* Removing unnecesary function
2020-03-02 08:13:39 -08:00
Ali Farooq
e4fb5791b0 MM-21626, MM-21627 - Plugin API/Hooks Prometheus instrumentati… (#13825)
* MM-21626,MM-21627 - Plugin API/Hooks Prometheus instrumentation

* Updated einterface mocks

* Fixed supervisor tests

* ignoring golint errors for plugin metrics wrappers

* Making golangci happy

* Using variadic form when generating wrapper code

* Removed artificial delay

* Removed comments from tests

* Renaming plugin wrappers to api/hooks_timer_layer

* updating vendor dir and mod files

* Recording plugin api/hook responses in prometheus

* Updated einterfaces-mocks

* Updating go sum

* Updating go sum

* Fixing conflicts

* More conflicts fixing

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-02-14 15:47:43 -05:00
Eli Yukelzon
17523fa5d9 MM-21898 - Part 1: Generate and use an interface instead of *A… (#13840)
* Generate and use an interface instead of *App
2020-02-13 13:26:58 +01:00
Shota Gvinepadze
a7854f1b97 [MM-22037] Enable uppercase letters in the email (#13794)
* Enable uppercase letters in the email

* Lowercase email on every input

* Remove invalid test
2020-02-12 08:51:45 -05:00
Jesús Espino
1d1ab03c38 Passing t to all tests setup functions (#13841)
* Passing t to all tests setup functions

* Fixing build
2020-02-10 19:31:41 +01:00
Agniva De Sarker
73ce92400b MM-22057: Limit incoming request bodies (#13827)
* MM-22057: Limit incoming request bodies

Set the max request body size to be equal to MaxFileSize.
Ideally, non-file request bodies should be smaller than file request bodies,
but we don't have a clean way to identify all file upload handlers.

There shouldn't be any valid request which exceeds the max file upload size.
So this is a safe global limit to apply.

* Fix tests

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-02-07 14:59:59 -05:00
Agniva De Sarker
a986aeb401 Remove explicit application/wasm mimetype (#13804)
Go mime sniffer supports the application/wasm mimetype natively
now. So there is no need to set this explicitly.
2020-02-05 09:06:17 +05:30
Agniva De Sarker
11d15107b0 MM-21922: Add metrics for http verb (#13679)
* MM-21922: Add metrics for http verb

* Use multiple params

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-01-28 08:52:00 +05:30
Doug Lauder
28ec291910 MM-19250 - new endpoint to retrieve command by id (#13598)
* MM-19250 add endpoint to retrieve command by id

* endpoint
* client
* unit tests

* MM-19250 update comment; remove redundant unit test

* MM-19250 rename GetCommand to GetCommandById

* MM-19250 don't filter on autocomplete flag

* MM-19250: require team_id when using GetCommandById

* team_id added to endpoint query string for GET
* unit test to check for mismatch teamid param and command teamid

* Revert "MM-19250: require team_id when using GetCommandById"

This reverts commit ed78e27964.

* MM-19250 don't leak existence of id when user doesn't have perms

* return 404 not_found when id not found
* return 404 not_found when id exists but user missing perms to view team
* return 404 not_found when id exists but user missing perms to manage commands

* MM-19250 fix typos in comments

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-01-24 09:32:56 -05:00
Vladimir Lebedev
50e965510a MM-19462 - Migrate tests from "web/oauth_test.go" to use testify (#13525)
* Migrate tests from "web/oauth_test.go" to use testify

* make test cleaner
2020-01-20 15:30:13 +01:00