Commit Graph

1787 Commits

Author SHA1 Message Date
Carlos Tadeu Panato Junior
fff09bf210 db: upgrade db to 5.36 (#17687) 2021-06-02 23:12:02 +02:00
Agniva De Sarker
e35c78cac2 Pass-through DB Driver implementation (#17685)
* Pass-through DB Driver implementation

This is the first step in implementing a DB layer via RPC.

The plan is to migrate the mattermost-plugin-api to use
this DB connector so that all queries start to get
routed through this library.

And then we will add the DB query capability to the plugin RPC
API and route all queries via RPC. At that point, this will be
completely transparent to all plugins because they will already
be using the DB connector and everything will be behind the scenes
for them.

https://focalboard-community.octo.mattermost.com/workspace/zyoahc9uapdn3xdptac6jb69ic?id=285b80a3-257d-41f6-8cf4-ed80ca9d92e5&v=495cdb4d-c13a-4992-8eb9-80cfee2819a4&c=c7386db7-65fd-469b-8bcf-8dc8f8e61e4f

```release-note
NONE
```

* remove deprecated interfaces
2021-05-27 14:59:06 +05:30
Eli Yukelzon
46649292f8 MM-34758 Collapsed Reply Threads without mobile support (#17424)
Summary
added support for legacy clients accessing server
added collapsed_threads_supported param to viewChannel API and setPostUnread API

Ticket Link
https://mattermost.atlassian.net/browse/MM-34758

Related Webapp PR
mattermost/mattermost-webapp#7933
2021-05-26 20:40:25 +05:30
Ben Cooke
3960d29bb4 [MM-34303] Add team ID to response when linking a channel to a group (#17576)
* add team ID to channel group response

Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-05-26 10:12:34 -04:00
Ashish Bhate
a5a22cdd12 MM-34895: fix test from PR that went in earlier (#17678) 2021-05-25 21:51:40 -03:00
Ashish Bhate
a95f90cec7 MM-34895: LastViewed should be updated correctly for new thread membership (#17673)
Summary
Update LastViewed correctly for new thread membership

Ticket Link
https://mattermost.atlassian.net/browse/MM-34895
2021-05-25 18:51:42 +05:30
Ashish Bhate
6f5e14dd5e MM-3543/MM-35436: fix thread being marked unread when updating reply post (#17645)
Summary
Threads should not be returned as unread if a reply post has been edited or reacted to since the time the thread was last viewed

Ticket Link
https://mattermost.atlassian.net/browse/MM-35436
https://mattermost.atlassian.net/browse/MM-35437
2021-05-25 17:08:14 +05:30
Claudio Costa
0bc065d147 Apply DB migrations regardless of version patch value (#17667)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-05-23 11:35:03 +02:00
Jesse Hallam
ff383990f8 Revert "[MM-8497] Ability to set Do Not Disturb for a specified period of time (#16067)" (#17657)
This reverts commit 77d42568f9.
2021-05-21 09:38:27 -03:00
Doug Lauder
2b02b03497 MM-35133 trigger sync user immediately after change (#17579)
- ensure changes to user profile sync immediately
- refactor sync send
2021-05-20 12:07:40 -04:00
Max Erenberg
9ef41a55e2 [MM-7968] restrict creation of direct channels to team members (#17222)
* restrict creation of direct channels to team members

* run make i18n-extract

* add suggestions from hahmadia

* place common-team-check logic in app layer

* use flat SQL query

* show more specific error message to user

* MM-7968: Fmt file.

* MM-7968: Fix for moved session field.

Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Martin Kraft <martinkraft@gmail.com>
Co-authored-by: Martin Kraft <martin@upspin.org>
2021-05-19 08:45:03 -04:00
Claudio Costa
4c4d889739 Remove redundant additions to cache hit/miss counters (#17636) 2021-05-18 17:45:03 +02:00
Claudio Costa
09f99e2426 [MM-35767] Ignore readTimeout param from MySQL datasource (#17624)
* Ignore readTimeout from MySQL datasource

* Add TODO

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-05-17 13:26:28 +02:00
darkLord19
77d42568f9 [MM-8497] Ability to set Do Not Disturb for a specified period of time (#16067)
* Add support for timed DND status

- accept a date time value in api query when dnd mode for user needs to be unset
- Create a new function to handle SetDNDStatus calls
- Create a scheduled task to unset dnd mode to wahtever mode was before setting it to DND

* update schema version

* Model changes to make fields more intuitive

- move dndendtime to status model
- add new field prev status in status to keep track of previous status of user
- update db migration function
- make use of prevstatus and dndendtime from status model

* set prev status and dndendtime appropriately after unsetting dnd mode

* add json tag for dndendtime

* unset dnd status only if not changed manually by user

* update dnd statuses after server restart

* make app-layers

* fix failing tests

* don't create sched task when setting status to DND

* get only expired statuses from db

- convert end time from any timezone to utc
- store dnd end time in unix format for usability reasons

* run update dnd status only on leader

* make mocks

* fix tests

* run UpdateDNDStatusOfUsers as recurring task

* save all statuses at once in db and update UpdateDNDStatusOfUsers logic

* add app method to get timezone of user

* store dnd end time in context.Params

* set max size of prevstatus

* update status model to take endtime input as string and store in db as unix time(int64)

* Add tests for SetStatusDoNotDisturbTimed

* if dnd_end_time is not passed the call old api to set dnd mode

* fix tests

* new plugin api to use new timed dnd mode

* get and update rows in a single db query

* dnd end time will be stored in request body and not route param

* exclude statuses which has dndendtimeunix < 0

* update and get the updated dnd statuses in single db query

* add updated status to cache

* DNDEndTimeUnix and PrevStatus need not to be visible to users

* update db schema version for migration

* Keep Status and PrevStatus varchar size same

* add test to verify status is restored after dnd end time expires

* expect endtime in utc from client

- remove store method GetTimezone as no longer needed
- add documentation for SetStatusDoNotDisturbTimed

* reduce sleep time for dnd timed restore test

* more appropriate name for new api to update user status

* update db migration function

* parse and validate time before potentially triggering db query to get status of user

* add migration changes in to existing upgrade function

* not supporting un-timed dnd status via api

* don't call Srv.Store directly, call via app layer

* rename dndendtime to statuscleartime to make it suitable for custom status usage as well

* Revert "rename dndendtime to statuscleartime to make it suitable for custom status usage as well"

This reverts commit fa69152d9a.

* mysql doesn't support RETURNING clause so add tx to get and update statuses

* add UpdateDNDStatusOfUsers mock in tests

* update store mock import path

* add mock in storelib

* Add status mocks to empty store

* Close the task during server shutdown

* Do not cancel a nil task

* update squirrel queries

* remove untimed dnd test

* start recurring task to unset statuses on leadership change

* set dndTask to nil after cancelling it upon server shutdown

* new recurring task which starts at nearest rounded time of the interval

* mock Get() call for status

* return updated statuses in case of mysql

* remove unneccessary code

* add Get() mock to empty store

* fix mocking for once and all

* address review comments

fix mysql updateStatus fn

protect dndTask with mutex

minor refactors

* move runDNDStatusExpireJob to server.go and pass App as arg instead of method receiver

* frontend will send endtime in unix epoch format so get rid of double representation

* scan for all fields and not just two

* add some tests and fix review comments

* remove extra sql query and create needed result in go

* add storetest for UpdateExpiredDNDStatuses

* add migrations to latest version

* update min supported version

* add comment to fix a bug in future

* update test to expect 1 status in return

* rename UpdateUserStatusWithDNDTimeout to SetUserStatusTimedDND

* rename DNDEndTimeUnix to DNDEndTime

* cast int to int64 for equality

* fix tests and error handling

* move updating values to retrieved statuses fields outside sql transaction

* move migrations to 5.36

Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-05-07 18:20:55 +02:00
Max Erenberg
9f4902e188 [MM-33908] Resync LDAP Groups to Teams and Channels (#17372)
* add includeRemovedMembers flag

* fix API call in client4.go

* remove check for 'since'

* add comments

* run make app-layers

* re-run CI tests

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-05-04 12:37:36 -04:00
Max Erenberg
3e5b0a7d7f [MM-29666] compliance export skipped posts (#17445)
* add MessageExportCursor

* sort by PostUpdateAt and PostId

* re-run CI tests

* remove panic debugging line

Co-authored-by: Martin Kraft <martin@upspin.org>
2021-05-04 09:08:09 -04:00
Doug Lauder
e2b9cb98aa MM-33903 rename slash commands & permission (#17494)
- rename slash commands
    - "remote" -> "secure-connection"
    - "share" -> "share-channel"
- change status icons to text (translated)
- remove channel invite "Description" field
- rename permission "manage_remote_clusters" -> "manage_secure_connections"
2021-04-30 14:59:29 -04:00
Claudio Costa
a7e6eef836 [MM-35316] Fix panic in app.CreateSidebarCategory (#17546)
* Fix returned error

* Fix expected error
2021-04-30 18:28:31 +02:00
Eli Yukelzon
51dfa9a54b CRT: Don't ignore own replies during reply count calculation (#17531)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-29 17:39:51 +03:00
Eli Yukelzon
fc0dbe0ace MM-35206 CRT: is_following on posts not reflecting state properly (#17533)
* forgot to copy isFollowing param

* /thread API didn't return thread posts, just the root

* cleanup

* [ci]

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-29 13:58:19 +02:00
Doug Lauder
12dbf0f527 MM-35181 Move SharedChannelsUsers.ChannelId column add to 5.36 migration (#17518)
* add 5.36 migration
2021-04-28 16:46:10 +02:00
Haardik Dharma
ccc7365ac3 Bump golangci version to v1.38.0 (#17154)
* Update golangci to v1.38.0

Co-authored-by: Claudio Costa <cstcld91@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-28 16:43:41 +02:00
Jesús Espino
757ea36211 Fix file info index batch (#17525) 2021-04-27 17:18:57 +02:00
Eli Yukelzon
3d6e69b002 MM-35128 CRT: Global Threads item should not be affected by 'mark as unread' in channel (#17505)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-27 16:56:06 +03:00
Eli Yukelzon
ba302cf512 MM-35125 CRT: Mention badge on threads doesn't appear until refresh (#17504) 2021-04-27 10:18:20 +03:00
Carlos Tadeu Panato Junior
246e4f9508 upgrade DB for 5.35 (#17488) 2021-04-23 15:28:01 +02:00
Max Erenberg
9cd50a4e22 [MM-33853] remove CSV row limit in compliance monitoring (#17185) 2021-04-23 09:19:13 -04:00
Eli Yukelzon
8c453bbcca MM-34895 CRT: Replying to a root post with an at-mention triggers an unread mention (#17474)
* don't count your own replies as unread

* corrected totalUnreadThreads query
2021-04-22 16:30:43 +03:00
darkLord19
f6505e1ee6 MM-31339: Send only one direct message reply within one calendar day. (#17181)
* don't send auto response if already responded today

* update query to get posts from channel for given user and Updatetime requires value in milli seconds

* regenerate mocks and layers

* update function to return true/false on existence of auto responded post in channel and add tests

* add store tests

* bubble up error and propagate upstream

* fix error handling logic

* use require instead of assert

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

* rename variable for better redability and logging fixes

* update comment explaining function

* use new function to generate test ids

* add comments to clarify NewTestId copies

* add translations for error id

* fix translation

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Saturnino Abril <saturnino.abril@gmail.com>
2021-04-22 12:33:45 +08:00
Ibrahim Serdar Acikgoz
28ff4dc8d0 [MM-32281] SqlRoleStore/GetByName: add context to allow read from master (#17176)
* role_store/GetByName: add context

* propagate context in the app layer

* propagate context in the app layer

* add missing import

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-21 18:01:02 +03:00
Eli Yukelzon
56198a99c6 MM-34871 CRT: Participants of thread include non-replying followers (and past followers) (#17447)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-21 09:48:30 +03:00
Max Erenberg
f36f5c74b1 Mm 30807 granular data retention scaffold (#17464) 2021-04-20 13:16:40 -04:00
Jesús Espino
368b642105 Fixing problem with read-replica on indexing files (#17460) 2021-04-20 12:30:43 +02:00
Claudio Costa
3984381011 [MM-13799] Avoid queries timing out during upgrade (#17444)
* Avoid timing out during upgrade

* Update gorp dep

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-19 20:34:21 +02:00
John Tzikas
9f203bbc96 Hotfix add multistatements param on mysql url (#17428)
Automatic Merge
2021-04-19 14:36:04 +02:00
Claudio Costa
4bd6f4942f Fix 5.32 migration (#17415) 2021-04-19 11:40:03 +02:00
Eli Yukelzon
b0d8ea5853 MM-34792 test shouldn't depend on upgrade sequence (#17425) 2021-04-19 09:10:31 +03:00
Agniva De Sarker
e0efdd708b Revert "Mm 30807 granular data retention scaffold (#16891)" (#17437)
This reverts commit 3ea75332e7.
2021-04-18 22:41:50 +05:30
Ibrahim Serdar Acikgoz
3a13987ee1 [MM-34725] app/user: check if username or email is in use before patching a user (#17392)
* app/user: check if username or email is in use before patching a user

* reflect review comments

* fix tests
2021-04-16 23:41:32 +08:00
Max Erenberg
3ea75332e7 Mm 30807 granular data retention scaffold (#16891)
create the necessary tables, models and APIs for the granular data retention policy feature
2021-04-16 11:32:09 -04:00
Claudio Costa
03473f98ac Fix 5.35 migration (#17416)
* Only run performance heavy migration queries once

* Fix ordering and test
2021-04-16 10:37:21 +02:00
Jesús Espino
09a52aab3e Fix problem with nulls in file infos content field (#17404) 2021-04-16 09:38:48 +02:00
Eli Yukelzon
f90209c8a3 MM-34609 Mark-as-unread on a post in a thread should cause auto-follow (#17343)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-16 10:26:08 +03:00
Doug Lauder
9799fe9be6 MM-34549 shared channels; add users to channel that were already sync'd (#17361)
Fixes a bug and adds a feature for shared channels:
- The Bug: when creating new shared channels, users that had already been sync'd via another channel were not added to the new channel's member list, since the users were not sync'd again. This PR sync's users per channel.
- The Feature: support custom statuses
2021-04-14 14:59:26 -04:00
Doug Lauder
f69cb38249 MM-32133 shared channel username collisions (#17347)
Support for handling username collisions between remote clusters. Users belonging to remote clusters have their username changed to include the remote name e.g. wiggin becomes wiggin:mattermost.

@mentions are also modified so the munged username is replaced with the original username when the post is sync'd with the remote the user belongs to.

When adding remote users:
- append the remote name to the username with colon separator
- append the remote name to the email address with colon separator
- store the original username and email address in user props
- when resolving @mentions replace with the stored original username
2021-04-13 10:40:12 -04:00
Max Erenberg
869da7a78b [MM-32044] Reset SAML auth data (#17161)
Automatic Merge
2021-04-13 00:46:30 +02:00
Jesús Espino
35d00b4644 Enable the errorAssertions govet check for mattermost-server code (#17346)
* Enable the errorAssertions govet check for mattermost-server code

* Removing unnecesary change

* Fixing some tests

* Fixing tests

* Fixing more after merge

* Fixing new offending entries

* Fixing small vet checks

* Fixing new cases detected by govet

* Fixing remote_cluster_test errors

* Fixing assertion

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-12 12:51:31 +02:00
jingkai
0eafaa502d Refactor SQL queries in store/sqlstore/preference_store.go to use the squirrel builder (#17086)
* refactor: refactored deleteUnusedFeatures

* refactor: refactored Get to remove hardcoded sql queries

* refactor: remove debug log

* refactor: refactored getcategory

* refactor: refactored GetAll to replace with sq

* refactor: refactor delete with sq

* refactor: refactored delete category with sq

* refactor: refactored cleanup flagsbatch

* refactor: refactor save to usq sq

* fix: fixed missing wildcard in LIKE operator

* refactor: fixed previous double call to database with a cleaner approach

* refactor: removed debug logs

* refactor: removed debug logs

* fix: added a new error checking as Limit accepts uint and the function parameter accepts int

* fix: golangcilint error

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-09 11:13:31 +02:00
Jesús Espino
819e4c0c64 Improve document extraction and including a document extraction command (#17183)
* Add extract documents content command

* Adding the extraction command and making the pure go pdf library as secondary option

* Improving the memory usage and docextractor interface

* Enable content extraction by default in all the instances

* Tiny improvement on archive indexing

* Adding App interface generation and the opentracing layer

* Fixing linter errors

* Addressing PR review comments

* Addressing PR review comments
2021-04-07 13:27:20 +02:00
Hossein
daec0b3201 All Sections to SubSections (#16917)
* initial

* Revert "initial"

This reverts commit 3d631aeecd.

* [MM-32352] Add Experimental Subsections BACKEND (#16887)

Automatic Merge

* update appiface

* Fix app layers

* Ancillary Permissions on backend (#17061)

Automatic Merge

* [MM-32799] Add About Section (#17015)

* Add About Section

* add mock key

* Update role.go

* Update role.go

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>

* [MM-33437] Fix config access tags for experimental settings (#17111)

Automatic Merge

* [MM-32794] Reporting Sub Section (#17035)

* test

* revert

* add permissions

* add new permission stuff

* add store mock

* fix bad merge

* gofmt fix

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>

* [MM-32343] Environment SubSection (#17054)

* pre-checkout commit

* fix permission for testSiteURL

* pre-merge commit

* increase size of Permissions column in Roles table

* add entry for ENVIRONMENT to testlib/store.go

* use TEXT for Permissions column in Roles table

* use environment subsection permissions for API endpoints

* use subsections permissions for /config/environment

* add suggestions from hahmadia

* update tests to use subsection permissions

* add permissions column back in

* comment out code in upgradeDatabaseToVersion534

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* MM-32351: Add Compliance Subsections (#17023)

* add subsections for compliance sectin

* add to mock functions

* updates for read job

* fixes

* fix test

* update tests

* update tests

* another test fix

* some cleanup

* update mlog

* fix linting

* Fix bad merges

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Hossein <hahmadia@users.noreply.github.com>
Co-authored-by: Hossein Ahmadian-Yazdi <hyazdi1997@gmail.com>

* MM-32347 Site Subsections (#17095)

* Init

* Added migration key in testlib store

* Fix syntax error

* fix bad merge

* fix lint

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* MM-32350 Integrations (#17097)

* implement server subsections

* fix tests

* update test

* go fmt

Co-authored-by: Hossein Ahmadian-Yazdi <hyazdi1997@gmail.com>

* patch forgotten endpoints

* Adding subsection permissions for Authentication (#17087)

* adding new permissions, migrations to do

* permission migrations and ancilary permissions

* running make app-layers

* fixing tests and lint

* adding permissions to saml

* ldap write permissions

* running make app-layers

* fixing conflict

* making app layers

* clean up and fix tests

* change job type

* fix js error, if site url not returned

Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Hossein Ahmadian-Yazdi <hyazdi1997@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* Update permissions_migrations.go

* gofmt

* upgrade to 535

* gofmt

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
Co-authored-by: Anurag Shivarathri <anurag6713@gmail.com>
Co-authored-by: Ben Cooke <benkcooke@gmail.com>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
2021-04-06 10:39:48 -04:00