* restrict creation of direct channels to team members
* run make i18n-extract
* add suggestions from hahmadia
* place common-team-check logic in app layer
* use flat SQL query
* show more specific error message to user
* MM-7968: Fmt file.
* MM-7968: Fix for moved session field.
Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Martin Kraft <martinkraft@gmail.com>
Co-authored-by: Martin Kraft <martin@upspin.org>
* Add support for timed DND status
- accept a date time value in api query when dnd mode for user needs to be unset
- Create a new function to handle SetDNDStatus calls
- Create a scheduled task to unset dnd mode to wahtever mode was before setting it to DND
* update schema version
* Model changes to make fields more intuitive
- move dndendtime to status model
- add new field prev status in status to keep track of previous status of user
- update db migration function
- make use of prevstatus and dndendtime from status model
* set prev status and dndendtime appropriately after unsetting dnd mode
* add json tag for dndendtime
* unset dnd status only if not changed manually by user
* update dnd statuses after server restart
* make app-layers
* fix failing tests
* don't create sched task when setting status to DND
* get only expired statuses from db
- convert end time from any timezone to utc
- store dnd end time in unix format for usability reasons
* run update dnd status only on leader
* make mocks
* fix tests
* run UpdateDNDStatusOfUsers as recurring task
* save all statuses at once in db and update UpdateDNDStatusOfUsers logic
* add app method to get timezone of user
* store dnd end time in context.Params
* set max size of prevstatus
* update status model to take endtime input as string and store in db as unix time(int64)
* Add tests for SetStatusDoNotDisturbTimed
* if dnd_end_time is not passed the call old api to set dnd mode
* fix tests
* new plugin api to use new timed dnd mode
* get and update rows in a single db query
* dnd end time will be stored in request body and not route param
* exclude statuses which has dndendtimeunix < 0
* update and get the updated dnd statuses in single db query
* add updated status to cache
* DNDEndTimeUnix and PrevStatus need not to be visible to users
* update db schema version for migration
* Keep Status and PrevStatus varchar size same
* add test to verify status is restored after dnd end time expires
* expect endtime in utc from client
- remove store method GetTimezone as no longer needed
- add documentation for SetStatusDoNotDisturbTimed
* reduce sleep time for dnd timed restore test
* more appropriate name for new api to update user status
* update db migration function
* parse and validate time before potentially triggering db query to get status of user
* add migration changes in to existing upgrade function
* not supporting un-timed dnd status via api
* don't call Srv.Store directly, call via app layer
* rename dndendtime to statuscleartime to make it suitable for custom status usage as well
* Revert "rename dndendtime to statuscleartime to make it suitable for custom status usage as well"
This reverts commit fa69152d9a.
* mysql doesn't support RETURNING clause so add tx to get and update statuses
* add UpdateDNDStatusOfUsers mock in tests
* update store mock import path
* add mock in storelib
* Add status mocks to empty store
* Close the task during server shutdown
* Do not cancel a nil task
* update squirrel queries
* remove untimed dnd test
* start recurring task to unset statuses on leadership change
* set dndTask to nil after cancelling it upon server shutdown
* new recurring task which starts at nearest rounded time of the interval
* mock Get() call for status
* return updated statuses in case of mysql
* remove unneccessary code
* add Get() mock to empty store
* fix mocking for once and all
* address review comments
fix mysql updateStatus fn
protect dndTask with mutex
minor refactors
* move runDNDStatusExpireJob to server.go and pass App as arg instead of method receiver
* frontend will send endtime in unix epoch format so get rid of double representation
* scan for all fields and not just two
* add some tests and fix review comments
* remove extra sql query and create needed result in go
* add storetest for UpdateExpiredDNDStatuses
* add migrations to latest version
* update min supported version
* add comment to fix a bug in future
* update test to expect 1 status in return
* rename UpdateUserStatusWithDNDTimeout to SetUserStatusTimedDND
* rename DNDEndTimeUnix to DNDEndTime
* cast int to int64 for equality
* fix tests and error handling
* move updating values to retrieved statuses fields outside sql transaction
* move migrations to 5.36
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* add includeRemovedMembers flag
* fix API call in client4.go
* remove check for 'since'
* add comments
* run make app-layers
* re-run CI tests
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* add MessageExportCursor
* sort by PostUpdateAt and PostId
* re-run CI tests
* remove panic debugging line
Co-authored-by: Martin Kraft <martin@upspin.org>
* don't send auto response if already responded today
* update query to get posts from channel for given user and Updatetime requires value in milli seconds
* regenerate mocks and layers
* update function to return true/false on existence of auto responded post in channel and add tests
* add store tests
* bubble up error and propagate upstream
* fix error handling logic
* use require instead of assert
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* rename variable for better redability and logging fixes
* update comment explaining function
* use new function to generate test ids
* add comments to clarify NewTestId copies
* add translations for error id
* fix translation
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Saturnino Abril <saturnino.abril@gmail.com>
Fixes a bug and adds a feature for shared channels:
- The Bug: when creating new shared channels, users that had already been sync'd via another channel were not added to the new channel's member list, since the users were not sync'd again. This PR sync's users per channel.
- The Feature: support custom statuses
Support for handling username collisions between remote clusters. Users belonging to remote clusters have their username changed to include the remote name e.g. wiggin becomes wiggin:mattermost.
@mentions are also modified so the munged username is replaced with the original username when the post is sync'd with the remote the user belongs to.
When adding remote users:
- append the remote name to the username with colon separator
- append the remote name to the email address with colon separator
- store the original username and email address in user props
- when resolving @mentions replace with the stored original username
* Add extract documents content command
* Adding the extraction command and making the pure go pdf library as secondary option
* Improving the memory usage and docextractor interface
* Enable content extraction by default in all the instances
* Tiny improvement on archive indexing
* Adding App interface generation and the opentracing layer
* Fixing linter errors
* Addressing PR review comments
* Addressing PR review comments
Remote Cluster Service
- provides ability for multiple Mattermost cluster instances to create a trusted connection with each other and exchange messages
- trusted connections are managed via slash commands (for now)
- facilitates features requiring inter-cluster communication, such as Shared Channels
Shared Channels Service
- provides ability to shared channels between one or more Mattermost cluster instances (using trusted connection)
- sharing/unsharing of channels is managed via slash commands (for now)
* MM-33818: Add replica lag metric
We add two new metrics for monitoring replica lag:
- Monitor absolute lag based on binlog distance/transaction queue length.
- Monitor time taken for the replica to catch up.
To achieve this, we add a config setting to run a user defined SQL query
on the database.
We need to specify a separate datasource field as part of the config because
in some databases, querying the replica lag value requires elevated credentials
which are not needed for usual running of the application, and can even be a security risk.
Arguably, a peculiar part of the design is the requirement of the query output to be in a (node, value)
format. But since from the application, the SQL query is a black box and the user can set any query
they want, we cannot, in any way templatize this.
And as an extra note, freno also does it in a similar way.
The last bit is because we need to have a separate datasources, now we consume one extra connection
rather than sharing it with the pool. This is an unfortunate result of the design, and while one extra
connection doesn't make much of a difference in a single-tenant scenario. It does make so, in a multi-tenant scenario.
But in a multi-tenant scenario, the expectation would already be to use a connection pool. So this
is not a big concern.
https://mattermost.atlassian.net/browse/MM-33818
```release-note
Two new gauge metrics were added:
mattermost_db_replica_lag_abs and mattermost_db_replica_lag_time, both
containing a label of "node", signifying which db host is the metric from.
These metrics signify the replica lag in absolute terms and in the time dimension
capturing the whole picture of replica lag.
To use these metrics, a separate config section ReplicaLagSettings was added
under SqlSettings. This is an array of maps which contain three keys: DataSource,
QueryAbsoluteLag, and QueryTimeLag. Each map entry is for a single replica instance.
DataSource contains the DB credentials to connect to the replica instance.
QueryAbsoluteLag is a plain SQL query that must return a single row of which the first column
must be the node value of the Prometheus metric, and the second column must be the value of the lag.
QueryTimeLag is the same as above, but used to measure the time lag.
As an example, for AWS Aurora instances, the QueryAbsoluteLag can be:
select server_id, highest_lsn_rcvd-durable_lsn as bindiff from aurora_global_db_instance_status() where server_id=<>
and QueryTimeLag can be:
select server_id, visibility_lag_in_msec from aurora_global_db_instance_status() where server_id=<>
For MySQL Group Replication, the absolute lag can be measured from the number of pending transactions
in the applier queue:
select member_id, count_transaction_remote_in_applier_queue FROM performance_schema.replication_group_member_stats where member_id=<>
Overall, what query to choose is left to the administrator, and depending on the database and need, an appropriate
query can be chosen.
```
* Trigger CI
* Fix tests
* address review comments
* Remove t.Parallel
It was spawning too many connections,
and overloading the docker container.
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
