MM-27744 disable Zap for unit tests.
Zap has no concept of shutdown or close. Zap is only shutdown when the app exits. Not a problem for console logging, but when creating a new Zap logger that outputs to files on every unit test, that leaves no easy way to clean up until process exit. Depending on what else is running this can exhaust all file handles and cause unit tests to fail.
Zap is now disabled unit tests and uses Logr instead, regardless of config settings. `make test-server` peak file handle usage dropped from ~5K to less than 100.
* MM-23832: Initial set of changes
* MM-23832: further iteration
* MM-23832: further iteration
* MM-23832: further iteration
* MM-23832: Fixes merge.
* create migration for new Roles
* MM-23832: Renames some roles.
* MM-23832: Adds ability to see logs.
* MM-23832: Removes manage roles from restricted admin.
* MM-23832: Make authentication section read-only for restricted admin.
* MM-23832: Allow restricted admin to purge caches.
* MM-23832: Adds ability to recycle DB connections.
* MM-23832: Adds ability to purge indexes.
* MM-23832: Adds ability to test email and S3 config.
* MM-23832: Adds abilituy to read job status.
* MM-23832: Adds ability to read plugin statuses.
* MM-23832: Renames Restricted Admin to System Manager.
* MM-23832: Adds manage team roles to system_user_manager.
* MM-23832: Updates some permissions.
* MM-23832: Allow get all channels and get moderations.
* MM-23832: Adds some permissions to User Manager.
* MM-23832: Remove write users from user manager.
* MM-23832: Changes permissions for the usermanagement > users sysconsole section.
* MM-23832: Removes read_settings and write_settings permissions. Ensures the usermanagement parent permissions encompass the sub-permissions.
* MM-23832: Updates permissions.
* MM-23832: Changes some permissions checks, adds new permissions to roles.
* MM-23832: Adds ability to update a role.
* MM-23832: Permissions updates.
* MM-23832: Removes write access to plugins for system manager.
* MM-23832: Removes read compliance from new roles.
* MM-23832: Adds mock for new roles creation migration.
* MM-23832: Changes to variadic param.
* MM-23832: Removes some duplication in the permissions model. Renames some permissions constants.
* MM-23832: Updates some migrations.
* MM-23832: Removes some unnecessary constants.
* MM-23832: Changes back to old app method name.
* MM-23832: Fixes incorrect permission check.
* MM-23832: Changes write to read permission check.
* MM-23832: Removes the authentication permission from link/unlink group.
* MM-23832: Enable testing LDAP with read permissions.
* MM-23832: Make testing elasticsearch a read permission.
* MM-23832: Warn metrics are associated to any system console read permissions.
* MM-23832: Updates some permissions checks.
* MM-23832: Removes non-systemconsole permissions from roles.
* MM-23832: Update default permission assignment of sysadmin.
* MM-23832: Fixes incorrect permission check. Removes some unused stuff.
* MM-23832: Update permission to check.
* MM-23832: Switches to struct tags.
* MM-23832: Adds some docs for the permissions tag.
* MM-23832: Removes whitespace.
* MM-23832: Combines system admin restricted access with other acess-control tag.
* MM-23832: Fixes some tests.
* MM-23832: Clarifies docs, does not assume prior permission check in '-' access value case.
* MM-23832: Updates to correct access tag value.
* MM-23832: Adds test of the config settings tag access.
* MM-23832: Undoes whitespace change.
* MM-23832: Removes comment.
* MM-23832: Adds the permissions to the new roles rather than using OR conditions on the permissions checks.
* MM-23832: Removes or condition on permission check.
* MM-23832: Updates mapping.
* MM-23832: Typo fix.
* MM-23832: Adds new 'read_jobs' permission.
* MM-23832: Add read_jobs to all roles with manage_jobs.
* MM-23832: Adds new permission read_other_users_teams.
* MM-23832: Adds read filtering of config.
* MM-23932: Change tag value.
* MM-23832: Fixes some tests. Adds test for read config access tag.
* MM-23832: Adds permissions to list teams.
* MM-23832: Removes the '-' tag value. Adds a new permission read_channel_groups. Updates a permission check.
* MM-23832: Removes unnecessary parent permission for user_management. Fixes permission check change error.
* MM-23832: Removes unused parameter to filter/merge function.
* MM-23832: Renames migration name.
* MM-23832: Fix for godoc.
* MM-23832: Fixes tests.
* MM-23832: Only makes a map once rather than every function call. Doesn't require access tag on config field structs. Reverts one test update and fixes another.
* MM-23832: Removes all of the unnecessary uses of (*App).SessionHasPermissionToAny since removing the user_management parent permission.
* MM-23832: Updates constant type.
* MM-23832: Removes unnecessary comment.
* MM-23832: Renames permissions.
* MM-23832: Fix for permission name changes.
* MM-23832: Adds missing config access tags. Adds some requirec ancillary permissions for write_usermanagement_teams.
* MM-23832: Adds local API endpoint for getting config.
* MM-23832: If tag value is blank or restrict_sys_admin_write then don't do the permission check.
* MM-23832: nil check for strings prior to dereferencing.
* MM-23832: Fix for config display logic.
* MM-23832: Updates godoc.
* MM-23832: Delays the unrestricted check for parity with other permissions checks if the channel id does not exist.
* MM-23832: Removes tautology.
* MM-23832: Re-adds status code check.
* MM-23832: Adds new permission to edit brand image.
* MM-23832: Exports variable for use by mmctl.
* MM-23832: Initialize exported map for use by mmctl.
* MM-23832: Accept deprecated permissions as valid.
* MM-23832: Adds missing permissions to archive a channel.
* MM-23832: Adds missing permissions for managing team.
* MM-23832: Properly filters config values in patch and update API responses.
* MM-23832: Fixes license viewing and writing permissions.
* MM-23832: Require license to assign 'new system roles'.
* MM-23832: Adds translation keys.
* MM-23832: Updates translation order.
* MM-27529: Splits read_channel_groups into read_public_channel_groups and read_private_channel_groups.
* MM-23832: Prevent read-only permissions from editing site url test parameter.
* MM-23832: Prevent read permissions from sniffing ports and elastic password.
* MM-23832: Adds missing permission required for write user management channels.
* MM-23832: Allows new roles to search for channels.
* MM-23832: Adds ability for system_manager to manage jobs.
* MM-23832: Cluster status access by sysconsole permission, not manage_system.
* MM-23832: Adds 'add_user_to_team' permission to sysconsole write usermanagement teams.
* MM-23832: Fixes lint.
* MM-23832: Test fix.
* MM-23832: Test fix.
Co-authored-by: Catalin Tomai <catalin.tomai@mattermost.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Adding Upgrade to Enterprise version feature
* Addressing PR review comments, and adding some minor improvements
* Add tests file
* Addressing PR comments
* fix linter checks
* Storing and exposing the upgraded from TE info
* Fix showing errors on mac
* A more appropiate status code for not-supported upgrade
* Fixing tests
* Handling permissions errors
* More server logging around upgrade failures
* Apply text changes suggested from code review
Co-authored-by: Eric Sadur <57730300+esadur@users.noreply.github.com>
* Address PR review comments
* Only allow to restart the system after an upgrade
* Verify file signature before upgrade
* Adding limit to the downloaded file
* Simplifying the upgrade binary process with backup in memory
* Fixing backup/restore mechanism for the binary file
* Improve file permissions handling
* Askin the permissions for the right place (the parent directory)
* Fixing tests
* Addressing PR review comments
* Fix license headers
* Fixing retry layer
* Making it work on windows builds
* Adding license header
* Fixing 2 tests
* Fixing tests that need UpgradeFromTE System key mock
* Extracting i18n translation
* Apply suggestions from code review
Co-authored-by: Eric Sadur <57730300+esadur@users.noreply.github.com>
* Improving how the errors are written
* Fixing another error text
* Removing unneeded translation
* Fixing upgrade status strings
* Update i18n/en.json
Co-authored-by: Eric Sadur <57730300+esadur@users.noreply.github.com>
* Fixing tests
Co-authored-by: Eric Sadur <57730300+esadur@users.noreply.github.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Mobile users were having their sessions unexpectedly expired, despite having ServiceSettings.ExtendSessionLengthWithActivity enabled.
Every time a mobile app is opened it called `/api/v4/sessions/device` which calls attachDeviceId which calls `(*Session)SetExpireInDays`. This code above assumed the expiry should be relative to CreateAt which is incorrect when ExtendSessionLengthWithActivity is enabled. Therefore, every time the mobile app was opened, the maximum expiry was set in memory to CreateAt + session_length, even if the session was extended.
(*Session)SetExpireInDays is now deprecated and replaced with (*App)SetSessionExpireInDays which takes into account the ExtendSessionLengthWithActivity setting.
* MM-27275: Log the correct error when GetUser fails in MFA Authentication
We were not logging the error returned from GetUser which makes debugging things
very hard in case of logout issues like https://mattermost.atlassian.net/browse/MM-27270.
We fix this by appending the error inside model.AppError and setting a proper error string
which says exactly what has happened.
* Update web/context.go
Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Adds the advanced logging config for audit. Existing support for auditing to a single file remains for E0 and E10 licenses instances, and a new config item ExperimentalAuditSettings.AdvancedLoggingConfig is added that behaves like LogSettings.AdvancedLoggingConfig.
Supported destinations:
- file
- syslog (with out without TLS)
- raw TCP socket (with out without TLS)
ExperimentalAuditSettings.AdvancedLoggingConfig can contain a filespec to a config file, a database DSN, or JSON.
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
* Extracting email functions into a service
* Fixing two shadowing errors
* Address PR review comments
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* parent 48256721c4 (#14358)
author Eli Yukelzon <reflog@gmail.com> 1585814774 +0300
committer Eli Yukelzon <reflog@gmail.com> 1589111022 +0300
Sidebar caregories implemented
Apply suggestions from code review
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Update store/sqlstore/channel_store.go
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Update store/sqlstore/channel_store.go
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
code review suggestions
status messages
edge case
bugs...
timeout reverse
* MM-25126 Add a separate default sorting method for categories (#14575)
* MM-25158 Add user to initial sidebar categories when joining team (#14570)
* MM-25281 Place new categories in the correct position (#14609)
* MM-25277 Return channels that aren't in a category as part of the Channels/Direct Messages categories (#14601)
* MM-25276 Remove categories when leaving a team (#14600)
* Remove categories when leaving a team
* layers
* corrected cleanup function
* lint
* .
* corrected errors in postgres
* .
* MM-25280 Ensure that the "update category order" API call only contains real category IDs and isn't missing any IDs (#14626)
* Ensure that the "update category order" API call only contains real category IDs and isn't missing any IDs
* tests
* correct status code
* MM-25278 Change "update category" API to return 400 when changing unsupported fields (#14599)
* MM-25279 Change category migration to only populate channels in Favorites (#14627)
* MM-25157 Add API to delete custom categories (#14574)
* MM-25157 Add API to delete custom categories
* get categories fix
* maxorder fix
* Use correct websocket event when deleting category
* Fix tests and remove debug code
* Actually use the right websocket event this time
* test cleanup
* Update test for new category order
Co-authored-by: Eli Yukelzon <reflog@gmail.com>
* MM-24914 Various fixes for sidebar channel handling (#14756)
* Fix checking for channel membership when reordering channels
* Remove unique constraint on SidebarCategories
* Set column sizes for SidebarCategories and SidebarChannels tables
* Allow changing the sorting method for non-DM categories
* Fix nil pointers in error handling
* Fix orphaned channels from other team being returned in Channels category
* Fix non-orphaned channels being duplicated in the Channels category
* Remove unique constraint on SidebarChannels
* Fix category/name of favorites preferences
* Fix testSidebarChannelsMigration
* Rename err to nErr and appErr to err
* Fix channel order returned by GetSidebarCategories on MySQL
* Fix adding/removing favorites preferences
* Remove leftover TODO
* Change SidebarCategoryType enums to use full names (#14786)
* Change SidebarCategoryType enums to use full names
* Fix Channels constant
* Remove leftover debug code
* MM-24914 Fix updateCategory endpoint returning the wrong type (#14795)
* MM-24914 Make some changes to UpdateSidebarCategories (#14806)
* Fix orphaned DMs not always being returned
* MM-24914 Make some changes to UpdateSidebarCategories
* Run updateSidebarCategoryOrderT in a transaction
* Fix deleting SidebarChannels based on order of arguments to UpdateSidebarCategories
* bump for api testing
* bump for api testing
* Change CreateInitialSidebarCategories to return a plain error
* Change MigrateSidebarCategories to return a plain error
* Remove usage of UpdateColumns when updating sidebar categories (#14843)
* Remove usage of UpdateColumns when changing category order
* Add a random test case
* Remove usage of UpdateColumns when updating sidebar categories (#14843)
* Remove usage of UpdateColumns when changing category order
* Add a random test case
* Remove usage of UpdateColumns when updating sidebar categories (#14843)
* Remove usage of UpdateColumns when changing category order
* Add a random test case
* MM-26343 Make CreateInitialSidebarCategories idempotent (#14870)
* Fix bad merge
* Fix another bad merge
* Fix unintentionally removed i18n string
Co-authored-by: Eli Yukelzon <reflog@gmail.com>
* Pass device ID
* dont use device id as way of detecting
* fix spelling mistake
* update layers
* fix test
* fix linting
* save schema
* put columns in correct place
* fix linting
* update
* upgrade go change
* use props
* fix stuff
* update session tests
* address PR comments
* address PR comments
* Removing some other fake apps
* More FakeApp removed
* Removing entirely FakeApp
* Fixing some tests
* Fixing get Cluster id from get plugin status
* Fixing failing tests
* Fixing tests
* Fixing test initialization for web
* Fixing InitServer for server tests
* Fixing InitServer for server tests
* Reverting go.sum and go.mod
* Removing unneded HTMLTemplates function in App layer
* Moving back some functions to its old place to easy the review
* Moving back some functions to its old place to easy the review
* Using the last struct2interface version
* Generating store layers
* Fixing merge problems
* Addressing PR comments
* Small fix
* Fixing app tests build
* Fixing tests
* fixing tests
* Fix tests
* Fixing tests
* Fixing tests
* Fixing tests
* Moving license to server struct
* Adding some fixes to the test compilation
* Fixing cluster and some jobs initialization
* Fixing some license tests compilation problems
* Fixing recursive cache invalidation
* Regenerating app layers
* Fix test compilation
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* Content-Type is optional
mime.ParseMimeType returns and "no media type" error if the passed
string is empty.
Given that the Content-Type header is optional we shouldn't return an error
in that case, so we're fixing that allowing the users to call the webhook
without passing that header
* Include webhook id in the error message
Given that the number of webhooks could be big the user could
need the id to check which one of the multiple webhooks are failing
so include the id aids in that part
* MM-25040: Only return team-associated groups if the team is group-constrained.
MM-25040: Prevents associating a group to a channel if the team doesn't have the group first.
* MM-25040: Fix lints.
* MM-25040: Still add the groupteam if the team is not group-constrained.
* MM-25040: Wraps groupteam upsert in else branch for efficiency.
* MM-25040: Removes unnecessary page iteration.
* MM-25040: Fix typo.
* MM-25040: Moves filtering to SQL.
* MM-25040: Updates tests, check pagination.
* MM-25040: Fix lint error.
* MM-25040: Adds some more group store tests.
* MM-25040: Fix for wrong test parameter.
* [MM-24146] Add unix socket listener for mmctl local mode (#14296)
* add unix socket listener for mmctl local mode
* add a constant for local-mode socket path
* reflect review comments
* [MM-24401] Base approach for Local Mode (#14333)
* add unix socket listener for mmctl local mode
* First working PoC
* Adds the channel list endpoint
* Add team list endpoint
* Add a LocalClient to the api test helper and start local mode
* Add helper to test with both SystemAdmin and Local clients
* Add some docs
* Adds TestForAllClients test helper
* Incorporating @ashishbhate's proposal for adding test names to the helpers
* Fix init errors after merge
* Adds create channel tests
* Always init local mode to allow for enabling-disabling it via config
* Check the RemoteAddr of the request before marking session as local
* Mark the request as errored if it's local and the origin is remote
* Set the socket permissions to read/write when initialising
* Fix linter
* Replace RemoteAddr check to ditch connections with the IP:PORT shape
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Fix translations order
* [MM-24832] Migrate plugin endpoints to local mode (#14543)
* [MM-24832] Migrate plugin endpoints to local mode
* Fix client reference in helper
* [MM-24776] Migrate config endpoints to local mode (#14544)
* [MM-24776] Migrate get config endpoint to local mode
* [MM-24777] Migrate update config endpoint to local mode
* Fix update config to bypass RestrictSystemAdmin flag
* Add patchConfig endpoint
* MM-24774/MM-24755: local mode for addLicense and removeLicense (#14491)
Automatic Merge
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Ashish Bhate <bhate.ashish@gmail.com>
* MM-25005: Remove HttpRequestsDuration bucket
The ApiTimesHistograms already captures what HttpRequestsDuration does.
And it's more granular than that. So there's no need to duplicate stuff.
Let's remove this one.
* Updating resource class to xlarge
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Jason Deland <jaydeland@gmail.com>
* MM-23935 extend session expiry on user activity
- if user types anything before a session expires the session will be extended to now + session length
- ensures new session expiries are not written to DB too frequently
- new session store func for updating session ExpiresAt
- session length defaults for mobile and web/ldap changed from 180 days to 30 days
* ResponseWriter wrapper to get status code
For our metrics, we need the status code returned by a request
so this wrapper includes a new method StatusCode() that includes
the desired code
* Shadow the responsewriter variable in the handlers
In order to avoid confusion to people deciding what variable to use.
I've also changed the tests to reflect this change and added a new
one that checks the Flush method works
Add auditing to server CLI.
Also:
- simplify auditing in API layer
- reduce number of AddMeta calls
- have models serialize themselves
- more consistent field naming
* MM-23222 add file target (with rotation) to audit
* MM-23222 mirror syslog audits to local filesystem
* provides config options for file name, max size, max age
* rotates files based on max size and max age; delete as needed based on max backups
* include cluster id in log records
* sort meta data fields
* Remove unncessary recover() calls
A recover which is not called inside a defer will always return nil.
So there's no use of calling it.
And even if we did call it inside a defer, recovering and immediately
panicking does not make sense either. So we just remove it.
* Fixed another instance
* New auditing API outputting to syslog via TLS
* New config section for specifying remote syslog server IP, port, and cert.
* Legacy audit API retained for access history feature
* Introducing unit (not integration) tests for the app layer
* Initial support for unit tests at the API
* Adding unit tests support to the store layer
* Add unit tests support in commands
* Adding last tests needed for run unit tests properly
* Fixing govet
* Removing some duplication
* Fixing tests
* Fixing tests
* Not compiling test helpers with the main module for api
* Revert "Not compiling test helpers with the main module for api"
This reverts commit 36a199bbe0.
* Fixing tests
* Fixing unit tests
* More consistency between api4/apiteslib.go and app/helper_test.go
* Renaming things to make more obvious the new Setup functions purpose
* Reverting change in go.sum
* Start with empty mock for app layer
* Start with empty mock for api layer
* Start with empty mock for web layer
* Renaming SetupWithStoreMockConfig to SetupConfigWithStoreMock
* Fixing tests on web package
* Removing unnecesary function
* MM-22057: Limit incoming request bodies
Set the max request body size to be equal to MaxFileSize.
Ideally, non-file request bodies should be smaller than file request bodies,
but we don't have a clean way to identify all file upload handlers.
There shouldn't be any valid request which exceeds the max file upload size.
So this is a safe global limit to apply.
* Fix tests
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
