Commit Graph

44 Commits

Author SHA1 Message Date
Farhan Munshi
b0d74c4e40 [MM-25385] Add group_count_with_allow_reference telemetry (#14614)
* MM-25385 Add group_count_with_allow_reference telemetry

* Add store layers

* Fix tests

* Fix linting
2020-05-22 10:56:15 -04:00
Agniva De Sarker
53cc7a26ea MM-24133: Migrate AppError from bot_store.go (#14339)
* MM-24135: Migrate AppError from SaveChannel/channel_store.go

This is the first POC of migration of store app errors to plain error.

We create a few basic error types in the store package and use
them to return the errors from store methods. In the app layer,
we inspect the error and re-create the exact app errors. This lets
us preserve the same error content, but yet move to plain errors.

Since this is a gradual migration, this means that the error inspection
code will be duplicated across the app layer whenever a store method
is invoked. But all of that should go away once we start propagating
the errors higher up the hierarchy.

There have been a significant amount of changes in the storetest and searchtest
layer, primarily because we have to rename the err variable now that it is of
a different type.

* Addressed review comments

* MM-24132: Migrate AppError from SaveDirectChannel/channel_store.go

This PR migrates 2 new methods SaveDirectChannel and CreateDirectChannel
to return error instead of AppError.

We also need to handle the error internally in SaveMultipleMember for now
until that is migrated too.

* MM-24133: Migrate AppError from bot_store.go

* Fix errors

* Fix err

* Fix bad return

* Fix vet errors

* Fix incorrect error check

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-05-19 08:36:53 +05:30
Agniva De Sarker
090743de86 MM-24132: Migrate AppError from SaveDirectChannel/channel_store.go (#14318)
* MM-24135: Migrate AppError from SaveChannel/channel_store.go

This is the first POC of migration of store app errors to plain error.

We create a few basic error types in the store package and use
them to return the errors from store methods. In the app layer,
we inspect the error and re-create the exact app errors. This lets
us preserve the same error content, but yet move to plain errors.

Since this is a gradual migration, this means that the error inspection
code will be duplicated across the app layer whenever a store method
is invoked. But all of that should go away once we start propagating
the errors higher up the hierarchy.

There have been a significant amount of changes in the storetest and searchtest
layer, primarily because we have to rename the err variable now that it is of
a different type.

* Addressed review comments

* MM-24132: Migrate AppError from SaveDirectChannel/channel_store.go

This PR migrates 2 new methods SaveDirectChannel and CreateDirectChannel
to return error instead of AppError.

We also need to handle the error internally in SaveMultipleMember for now
until that is migrated too.

* Fix layers

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-05-14 23:41:05 +05:30
Agniva De Sarker
729a84a3e6 MM-24135: Migrate AppError from SaveChannel/channel_store.go (#14299)
* MM-24135: Migrate AppError from SaveChannel/channel_store.go

This is the first POC of migration of store app errors to plain error.

We create a few basic error types in the store package and use
them to return the errors from store methods. In the app layer,
we inspect the error and re-create the exact app errors. This lets
us preserve the same error content, but yet move to plain errors.

Since this is a gradual migration, this means that the error inspection
code will be duplicated across the app layer whenever a store method
is invoked. But all of that should go away once we start propagating
the errors higher up the hierarchy.

There have been a significant amount of changes in the storetest and searchtest
layer, primarily because we have to rename the err variable now that it is of
a different type.

* Addressed review comments

* Made all appError origins to be CreateChannel

* Remove typed internal error

* Fix translations

* fix layer generation

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-05-12 21:23:41 +05:30
catalintomai
41e58d9769 MM-23816: Group Mentions: Add ability to rename group names (#14338)
* MM-23816: Group Mentions: Add ability to rename group names
2020-05-12 08:35:03 -07:00
Eli Yukelzon
744ec281d2 Layer generators error return type fix (#14526) 2020-05-11 12:37:02 +03:00
Doug Lauder
5e59b5f70c MM-23935 extend session expiry on user activity (#14275)
* MM-23935  extend session expiry on user activity

- if user types anything before a session expires the session will be extended to now + session length

- ensures new session expiries are not written to DB too frequently

- new session store func for updating session ExpiresAt

- session length defaults for mobile and web/ldap changed from 180 days to 30 days
2020-05-06 15:41:10 -04:00
Jesús Espino
224b72c61e MM-24037 Adding getKnowUsers API endpoint (#14332)
* Adding getKnowUsers API endpoint

* Adding i18n strings

* Fixing golint errors

* Adding doc strings

* Remove debug line

* Updating app_iface

* Fixing gofmt
2020-04-28 12:52:43 +02:00
Farhan Munshi
036f9384b4 [MM-23264] Get channel member counts by group (#14068)
* MM-23264 Add api endpoint for get groups with members in channel

Add store tests

Add tests for api func

Gofmt

Apply changes from code review

* MM-23264 Make store layers

* MM-23264 Check read permission on channel member counts

* Trigger CI
2020-04-24 17:12:54 -04:00
Martin Kraft
70e9647e85 MM-23646: Improve group sync performance. (#14171)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-04-21 15:55:30 -04:00
Farhan Munshi
e88ba85d60 [MM-23017] Add check for groups to SendNotifications (#14039)
* MM-23017 Check group mentions as part of notification logic

* Add nil groups to existing test cases

* MM-23017 Add tests for insertGroupMention and addGroupMention

* MM-23017 Add tests for getExplicitMentions that have groups

* Add tests for group store GetMemberUsersNotInChannel

* MM-23017 Add tests for AllowGroupMentions

* MM-23017 Fix error message name

* MM-23017 Swap Checks to Name

* MM-23017 Code review fixes

* Rename var and fix allowGroupMentions test

* MM-23017 Use GetMemberUsersInTeam inside of insertGroupMentions

* MM-23017 use group mentions permission

* Actually call GetMemberUsersInTeam

* Remove unnecessary new line

* Uncomment filter allow reference

* MM-23017 Fix group channel notifications

* Update store layer

* MM-23017 Improve test coverage for group channels

* Trigger CI

* Trigger CI
2020-04-17 19:22:54 -04:00
Farhan Munshi
2a9f465bf2 Fix broken master build by updating store (#14291)
* Fix broken master build by updating store

* Run store-mocks
2020-04-14 16:24:30 -04:00
catalintomai
b90f4f46e2 MM-23015: Enable or disable group mentions (#14010)
* MM-23015: Enable or disable group mentions + show them in suggestion list


Co-authored-by: Catalin Tomai <catalin.tomai@mattermost.com>
2020-04-13 11:37:29 -07:00
Jesús Espino
f6f025aeaa Adding guest accounts count telemetry (#14257)
* Adding guest accounts count telemetry

* Adding missed methods to the opentracing an timer layer
2020-04-09 11:05:43 +02:00
Hossein Ahmadian-Yazdi
bd2c1f4522 [MM-21517] Team Member Manage User Modal not display correct roles (#14043)
* join on the users table and order by username

* don't inclue users who have been deleted

* Address PR comments

* update tests

* fix linting

* fix linting

* include ExcludeDeletedUsers flag

* fix gofmt

* fix nil teamMembersGetOptions

* fix gofmt error

* Add Unit Tests

* fix gofmt bugs

* partially address comments

* fix incorrect import

* Address Comments and fix golint errors

* store mocks

* address PR comments about tests and styling

* Update model/team_member.go

Co-Authored-By: Jesse Hallam <jesse.hallam@gmail.com>

* Address PR comments

* update client function name

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
2020-03-30 10:48:03 -04:00
Jesús Espino
cfe65a33a5 Team/Channel members bulk save/update/delete (#14053)
* Members Save Multiple

* Multiple Channel Members implemented

* Working version with member multi save and multi update

* Adding tests to the save multiple team members store methods

* Tests passing for SaveMember

* Adding remove members tests

* Working on channel store tests

* More work on testing properly SaveMember and SaveMultipleMembers on channels

* More work on testing properly SaveMember and SaveMultipleMembers on teams

* Adding tests for update multiple members for team

* Adding tests for update multiple members for channel

* Fixing some tets

* Fixing tests

* Fixing tests

* Fixing i18n

* Addressing govet problems

* Addressing govet problems

* Addressing govet problems

* Addressing govet problems

* Addressing PR comments (and removing TODO)

* Addressing PR comments

* Fixed broken test

* Removing unnecesary i18n translation string
2020-03-27 13:29:17 +01:00
Martin Kraft
4d99aa22ba MM-22212 & MM-22208: Read from the higher-scoped scheme if the permission is non-moderated. (#13813)
* MM-22212: Read non-moderated permissions from higher-scoped scheme.

* MM-2212: Corrects test count in comment.

* MM-22212: Adds godoc comment.

* MM-2212: Switches to the channel roles check in a few more places.

* MM-22212: Refactors and fixes.

* MM-22212: Reverts change, no longer required.

* MM-22212: Removes translation.

* MM-22212: Un-comments merged new permission.

* MM-22212: Un-comments merged new permission.

* MM-22212: Performance tweak.

* MM-22212: Fixes some fmting.

* MM-22212: Add unit test for newly-added store methods.

* MM-22212: Renames app method.

* MM-22212: Re-uses existing function to find string in slice.

* MM-22212: Keeps 'higher-scoped' terminology for consistency.

* MM-22212: Refactors based on PR feedback.

* MM-22212: Fix for some bad merging.

* MM-22212: Renamed some things.

* MM-22212: Use an 'else' instead of a 'continue' for readability.

* MM-22212: Caches (*SqlRoleStore).ChannelRolesUnderTeamRole.

* MM-22212: Adds mock to new cache store.

* MM-22212: Adds missing open tracing app layer methods.

* MM-22212: Adds migration to add moderated permissions to channel_admin if present on channel_user.

* MM-22212: Migrates team schemes. Removes unused AppError.

* MM-22212: Fix for for if.

* MM-22212: Fixes iterator.

* MM-22212: Updates open tracing generated methods.

* MM-22212: Fix mocks.

* MM-22212: Change migration key name.

* MM-22212: Switched to data structure from other branch.

* MM-22212: Fixes tests after adding 'use_channel_mentions' to the channel_admin role.

* MM-22212: Adds tracking of channel moderation.

* Revert "MM-22212: Adds tracking of channel moderation."

This reverts commit 23689efa22.

* MM-22212: Switch some functions to methods and vice versa.

* MM-22212: Fix for refactor bug not notifiying websocket about changed role.

* MM-22212: Adds test for public/private 'manage_members' handling.

* MM-22122 Fix manage channel members edge case for public and private channels (#14049)

* MM-22212: Adds moderated permission to team_admin.

* MM-22212: Updates migration.

* MM-22212: Revert unnecessary update to default roles.

* Add channel scheme updated event when channel scheme is deleted or created (#14057)

* MM-22212: Adds newline.

* MM-22212: Migration fix.

* MM-22212: Fix for migration.

* MM-22212: Test fix.

Co-authored-by: Farhan Munshi <3207297+fm2munsh@users.noreply.github.com>
2020-03-23 13:44:20 -04:00
Jesús Espino
c66e182b08 Adding the new search engine abstraction (#13304)
* WIP

* Adding bleve to go modules

* WIP

* Adding missing files from searchengine implementation

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* User and channel indexing and searches implemented

* Make bleve tests run with in-memory indexes

* Implement post index and deletion tests

* Initial commits for the search layer

* Removing unnecesary indexing

* WIP

* WIP

* More fixes for tests

* Adding the search layer

* Finishing the migration of searchers to the layer

* Removing unnecesary code

* Allowing multiple engines active at the same time

* WIP

* Add simple post search

* Print information when using bleve

* Adding some debugging to understand better how the searches are working

* Making more dynamic config of search engines

* Add post search basics

* Adding the Purge API endpoint

* Fixing bleve config updates

* Adding missed file

* Regenerating search engine mocks

* Adding missed v5 to modules imports

* fixing i18n

* Fixing some test around search engine

* Removing all bleve traces

* Cleaning up the vendors directory and go.mod/go.sum files

* Regenerating timer layer

* Adding properly the license

* Fixing govet shadow error

* Fixing some tests

* Fixing TestSearchPostsFromUser

* Fixing another test

* Fixing more tests

* Fixing more tests

* Removing SearchEngine redundant text from searchengine module code

* Fixing some reindexing problems in members updates

* Fixing tests

* Addressing PR comments

* Reverting go.mod and go.sum

* Addressing PR comments

* Fixing tests compilation

* Fixing govet

* Adding search engine stop method

* Being more explicit on where we use includeDeleted

* Adding GetSqlSupplier test helper method

* Mocking elasticsearch start function

* Fixing tests

Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-13 15:33:18 +01:00
Jesús Espino
27d536b212 MM-21552: Adding SaveMultiple to posts (#13766)
* Adding SaveMultiple to posts

* Improving tests

* fixing i18n

* Fixing tests

* Improving testing on top of Save and SaveMultiple

* Fixing shadow variables

* Addressing some PR comments

* More clear update post test

* Addressing some PR comments

* Addressing some PR comments and simplifying the code

* Improting replies in bulk too

* Fixing reply count and processing last imported replies

* Adding OverwriteMultiple to posts aggregating everything in the same transaction

* Adding 2 pending tests to implement

* Adding tests for overwrite multiple posts

* Adding tests for TeamStore.GetByNames method

* Fixing shadow variables

* Addressing PR comments

* Extracting i18n strings

* Fixing tests

* Fixing tests

* Adding more test cases

* Using a variable instead of a fake timestamp
2020-03-11 14:29:32 +01:00
Jesús Espino
cd382412fd Migrating file info store cache to the cache layer (#13045)
* Migrating file info store cache to the cache layer

* Remove unrelated changes

* Updating timer layer store

* Fixing usage of invalidate cache

* Removed repeated tests and unnecesiary require

* Using doInvalidateCacheCluster instead of removing the cache directly

* Addressing PR comments

* Fixing imports

* Fixing tests

* Fixing license header
2020-03-09 18:34:25 +01:00
Miguel de la Cruz
182c29b456 MM-21898: Part 2. Add opentracing (#13904)
* initial implementation of opentracing

* app layer

* Revert Makefile

* .

* cleanup

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* [ci]

* autogenerate interface

* .

* missed vendor files

* updated interfaces

* updated store layers

* lint fixes

* .

* finishing layer generators and nested spans

* added errors and b3 support

* code review

* .

* .

* fixed build error due to misplased flag.Parse()

* code review addressed
2020-03-05 14:46:08 +01:00
Eli Yukelzon
597a2b77cd MM-17468 - Improve thread fetching (#13653)
* Revert "Thread fetching revert (#13616)"

This reverts commit 8e0fe90897.

* renamed query param for clarity

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-02-05 13:27:35 +01:00
Martin Kraft
50e9aa01c3 MM-21157: Triggers role sync when a syncable is linked, unlinked, or updated. (#13432)
* MM-20644: Add users to teams as a SchemeAdmin based on a new configuration field on GroupTeams and GroupChannels records.

* MM-20644: Adds SchemeAdmin to mapping of the GroupSyncable struct fields.

* MM-2064: Adds test to ensure SchemeAdmin field value is mapped.

* MM-20644: Adds missing index creation for fresh DBs.

* MM-20644: Duplicates UpdateMembersRole across Team and Channel stores. Adds tests.

* MM-20644: Fixes some old method name references.

* MM-20644: Moves variable declaration; removes Println statement.

* MM-21157: Triggers role sync when a syncable is linked, unlinked, or updated.

* MM-20644: Use a SQL query instead of two to update Team and Channel members.

* MM-20644: Fixes tests; updates query.

* MM-21157: Removes second invocation of function because of refactor that performs it all in a single query.

* MM-21157: Switches a few queries to Squirrel.

* MM-21157: SQL-formats some strings.

* MM-21157: Select with list.

* MM-21157: Converts some more sql to squirrel.

* MM-21157: Fix incorrect conflict resolutions.

* MM-21157: Fix incorrect conflict resolutions.

* MM-21157: Adds missing mocks.

* MM-21157: Clears cache upon syncing roles.

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-01-20 09:10:25 -05:00
Eli Yukelzon
8e0fe90897 Thread fetching revert (#13616)
* Revert "MM-19371 - Reply count disappears from pinned and flagged conv… (#12753)"
2020-01-15 17:14:04 +02:00
Ben Schumacher
6e9e849871 Fix issues found by golangci v1.22.2 (#13568) 2020-01-14 10:08:51 +01:00
Martin Kraft
605040c597 MM-20644: Add users to teams as a SchemeAdmin based on a new configuration field on GroupTeams and GroupChannels records. (#13361)
* MM-20644: Add users to teams as a SchemeAdmin based on a new configuration field on GroupTeams and GroupChannels records.

* MM-20644: Adds SchemeAdmin to mapping of the GroupSyncable struct fields.

* MM-2064: Adds test to ensure SchemeAdmin field value is mapped.

* MM-20644: Adds missing index creation for fresh DBs.

* MM-20644: Duplicates UpdateMembersRole across Team and Channel stores. Adds tests.

* MM-20644: Fixes some old method name references.

* MM-20644: Moves variable declaration; removes Println statement.

* MM-20644: Use a SQL query instead of two to update Team and Channel members.

* MM-20644: Fixes tests; updates query.

* MM-20644: Fix permission check for patching a group syncable.

* MM-20644: Fixes test for change of permissions verification in group patch API request.

* MM-20644: Fix for ORM select vs insert.

* MM-20644: Linting fixes.

* MM-20644: Fixes some tests.

* MM-20644: Skips changing the role of guests.

* MM-20644: Added syncableID filtering
2020-01-10 09:19:39 -08:00
Jesús Espino
464c290e50 Removing certain unneded cluster invalidation messages (#13549)
* Removing certain unneded cluster invalidation messages

* Fixing typo

* Removing channges included in other PR

* Adding doc string to RegisterAllClusterMessageHandlers method

* fixing typos

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-01-08 11:31:10 +01:00
catalintomai
e48d093d01 MM-19877: Remove references to PermanentDeleteBatch from audit store (#13393)
* MM-19877: Remove references to PermanentDeleteBatch from audit store
2019-12-20 15:37:01 +00:00
larkox
d2e78e28a9 [GH-13105] Migrate channelPinnedPostCountsCache cache from store/sqlstore/channel_store.go to the new store/localcachelayer (#13164)
* Migrate channelPinnedPostCountsCache cache from store/sqlstore/channel_store.go to the new store/localcachelayer

* Remove GetPinnedPostCountFromCache

* Rearrange test to avoid undeclared variable
2019-11-28 15:21:42 -05:00
Miguel de la Cruz
2259b7f2a8 [MM-19948] Set version on module file and internal paths (#13186)
* [MM-19948] Set version on module file and internal paths

* Fixes after merge

* Fix i18n checker error
2019-11-28 14:39:38 +01:00
Allen Lai
fdec1ef1f5 MM-20100: Migrate channelGuestCountsCache cache from store/sqlstore/channel_store.go to the new store/localcachelayer (#13135)
* init

* migrate

* add test

* fix channel store

* ...

* missing ')'

* change function interface

* fix build

* fix format

* Update store/localcachelayer/layer.go

Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com>

* Update store/localcachelayer/channel_guest_layer_test.go

Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com>

* im so confused why my builds are failing

* fix build

* format

* fix format

* add license header

* resolve some conflicts

* add test

* fix constant names

* fix more constants

* fix bugs

* delete unused constants

* Update store/localcachelayer/channel_layer.go

Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com>

* typo

* another typo

* get guest

* change test

* go fmt format

* change test

* remove getcountcache

* delete method from interface

* delete method

* update test

* rerun GoLang CI
2019-11-27 09:14:40 -05:00
Martin Kraft
31ac88ef69 MM-20305: Adds ability to choose whether deleted teams are included in the count query. (#13181) 2019-11-27 07:50:15 -05:00
Martin Kraft
0212845385 MM-18356: Adds ability to paginate channel search. (#12830)
* MM-18356: Adds ability to paginate channel search.

* MM-18356: Minor refactor.

* MM-18356: Adds doc.

* MM-18356: Fixes doc.

* MM-18356: Some commentary, adds the total count to non-paginated responses, and removes a stray fmt.

* MM-18356: Fixes shadowed variable.

* MM-18356: Removes paginate field and API parameter.

* MM-18356: Adds method to check if channel search is a paginated request.

* MM-18356: Vet fix.
2019-11-19 11:38:49 -05:00
Harrison Healey
df7cbcb440 Merge branch 'master' into mark-as-unread 2019-11-12 14:20:50 -05:00
Valentijn Nieman
36f3b14420 MM-19337 Enable users to view archived channels (#12995)
* First pass at searching archived channels

* Create endpoint for archived channels

* Update test for GetPublicChannelsForTeam

* Fix unit test for archived channels

* Rename includeDeleted to onlyDeleted

* Remove new /archived endpoint in favour of existing /deleted endpoint

* Fix broken test

* Remove manage team permission from /deleted endpoint

* Fix deletedChannels test

* Test for searching archived channels

* Only return private deleted channels user was a member of

* SearchArchivedChannels also searches private channels (user is a member of)

* Remove for loop to simplify append

* Remove userId from Client4 searcArchivedChannels
2019-11-12 06:15:19 +08:00
Harrison Healey
eb36329e8d Merge branch 'master' into mark-as-unread 2019-10-29 10:11:41 -04:00
Joram Wilander
9307f75fe9 Add some group plugin APIs (#12180)
* Add group store GetByUser

* Add group store GetByName

* Add group plugin APIs

* Minor naming fixes

* Add minimum version comments
2019-10-22 14:38:08 -04:00
Harrison Healey
66da2bab2b MM-18463 Added update_badge notification when marking a channel as unread (#12499)
* MM-18463 Added update_badge notification when marking a channel as unread

* Address feedback
2019-10-07 09:58:39 -04:00
Jesús Espino
99f13ed0e6 Use seconds instead of nanoseconds for the store time layer metrics (#12321)
* Use nanoseconds instead of seconds for the api metrics

* Using seconds instead of nanoseconds in the store layer metrics
2019-09-24 20:00:19 +02:00
Joram Wilander
3d4c941ba8 MM-18512 Use options struct for GetProfilesWithoutTeam and add filtering to API (#12200)
* Use options struct for GetProfilesWithoutTeam and add filtering

* Fix test
2019-09-17 20:13:17 +01:00
Eli Yukelzon
b3517eaf2f MM-17468 - Improving performance of fetching threads (#11980)
fetchThreads parameter support in the API
2019-09-17 14:37:10 +01:00
Woolim Cho
b9a3c2e90e MM-10272: Add a count for pinned posts header icon (#11840) 2019-09-04 13:41:29 +02:00
Jesús Espino
2df3951f6f Create direct message respect the user role (Guest/Normal user) (#11697)
* Create direct message respect the user role (Guest/Normal user)

* Fixing build

* Updating Timer store layer
2019-08-06 11:33:32 +02:00
Jesús Espino
e4582905ea Time Layer for measure the Store methods calls time (#11609)
* Time Layer for measure the Store methods calls time

* Fixing build

* Fixing a formating problem

* Fixing tests

* generating store mocks

* Fixing build

* Updating generated timer layer

* Updating timer layer to the last store interface

* Updating timer layer

* Generating time layer
2019-08-01 22:10:58 +02:00