Commit Graph

830 Commits

Author SHA1 Message Date
Scott Bishel
82b8d4dc07 MM-55966 - Update ArrayFromJSON to use LimitedReader (#25510)
* update ArrayFromJSON to use LimitedReader

* update for bad merge

* fix lint errors

* update test code

* update unit tests

* update unit tests

* fix unit tests

* use consts, other cleanup

* add non sorting duplicate check

* set config to default value, then config setting if available

* fix lint errors

* fixes and debugs

* fix log test

* remove setting from Client, add unlimited Parser to client

* a couple more fixes

* another fix

* rename some variables

* remove superflous call

* check for valid MaximumPayloadSize

* update language file

* fix for e2e-tests

* update util function to return error

* lint fix

* update config property name to include unit

* fix for unit test

* add new config to telemetry

* call function to create LimitedReader

* Deprecate old function, use new function name

* return new AppError on failed parse

* return new AppError on failed parse

* return new AppError on failed parse

* add constant for i18n valid constants

* Update server/public/model/utils_test.go

Co-authored-by: Miguel de la Cruz <mgdelacroix@gmail.com>

* Apply suggestions from code review

Co-authored-by: Miguel de la Cruz <mgdelacroix@gmail.com>

* update error variable, remove unnecessary check

* Update function names

* fix errors from merge

* update unit test to create unique ids

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Miguel de la Cruz <mgdelacroix@gmail.com>
2024-01-09 10:04:16 -07:00
Doug Lauder
93351ff8a4 Fix error message when sharing already shared channel; also make idempotent (#25854)
* fix appError/error assignment bug

* make ShareChannel plugin API idempotent

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-01-09 12:42:11 +01:00
Doug Lauder
241e8edc2e Ping new shared channels remotes immediately (#25850)
* option for auto inviting plugin to all shared channels.

* auto-invite remotes to shared channels when flag set

* fix unit test

* immediately ping new remotes; fix unique siteurl bug

* make i18n-extract

* fix translations

* fix merge conflicts

* make modules-tidy

* revert accidental go.mod change

* revert accidental go.sum changes

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-01-09 12:35:05 +01:00
Doug Lauder
000aa0d891 Plugin API for Shared Channels: support auto invite (#25834)
* option for auto inviting plugin to all shared channels.

* auto-invite remotes to shared channels when flag set

* fix unit test

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-01-08 18:12:18 +01:00
Jesse Hallam
b05093d508 Source available metrics (#24879)
* Expose metrics under a source available license

* do not assume Cluster()

* allow metrics if licensed or dev

* temporary vet override

* simplify BULID_TAGS handling

* auto clean old imports.go

* fix license listener

* e2e test metrics & license semantics

* update from enterprise

* switch back to mattermost-govet/v2@new now

* update metrics from upstream

* Update license_spec.js

Co-authored-by: Saturnino Abril <saturnino.abril@gmail.com>

* Update license_spec.js

Co-authored-by: Saturnino Abril <saturnino.abril@gmail.com>

* Update e2e-tests/cypress/tests/integration/channels/enterprise/metrics/license_spec.js

Co-authored-by: Saturnino Abril <saturnino.abril@gmail.com>

* Update e2e-tests/cypress/tests/integration/channels/enterprise/metrics/license_spec.js

* split up specs

* require/delete license earlier in e2e test

* expanded expect to debug failures

* more logging

* Revert "more logging"

This reverts commit 0bc513fd92.

* e2e: try deleting license first

* update from enterprise

* toggleMetricsOn to work around license delete

* eslint

* ensure admin before deleting license

* update from enterprise

* updates from enterprise

* fix cypress logging

* temp: log at DEBUG for Cypress tests

---------

Co-authored-by: Saturnino Abril <saturnino.abril@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2024-01-08 10:47:24 -04:00
Miguel de la Cruz
6184c36e0b Fix make permission error (#25860) 2024-01-08 14:45:48 +01:00
Miguel de la Cruz
e1a27cf57f Adds a team check to get groups by team (#25462)
* Adds a team check to get groups by team

* Update to take into account groups_by_channel and new permissions

* Update API docs
2024-01-08 13:28:51 +01:00
Nick Misasi
e52940cd08 Lock wrangler behind enterprise (#25703)
* Lock wrangler behind enterprise

* Run tests with SetupEnterprise so Thread Moving is unlocked. Tweak NewTestLicense to include a SkuShortName for 'enterprise'

* Don't set SkuShortName in NewTestLicense just in case. Fix up some old debug code in tests

* Remove checks on enterprise/e20 and just enforce that the workspace has any license

* Update admin_definition.tsx accordingly

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-01-05 13:20:37 -05:00
Devin Binnie
ac854498a4 [MM-56266] Check for public channels when getting channel member counts to avoid 403 errors (#25831)
* [MM-56266] Check for public channels when getting channel member counts to avoid 403 errors

* Fix test
2024-01-05 12:28:22 -05:00
Claudio Costa
077221a940 Revert "MM-55295: Improve DB performance of some APIs (#25318)" (#25852)
This reverts commit ec88ab4ee9.
2024-01-05 10:05:37 -06:00
unified-ci-app[bot]
34e1692286 Update latest minor version to 9.5.0 (#25847)
Automatic Merge
2024-01-04 21:54:56 +02:00
Agniva De Sarker
ca94577cd5 MM-56260: connIndex: safely remove conns while iterating (#25785)
PR https://github.com/mattermost/mattermost/pull/22560
introduced a regression in the case where we had
multiple connections for a single user.

Because if connIndex.Remove was called during the iteration
from connIndex.ForUser, then the slice would be modified
during iteration and if a connection got removed,
then a good connection would move from the last index
to the current index. But since we would be actively
iterating, the last index would be read as nil and we would
never be able to reach the good connection.

We fix this by returning a copy of the original slice
if there are more than one elements in the slice.

https://mattermost.atlassian.net/browse/MM-56260
```release-note
Fix a bug where if there were multiple websocket
connections from a single user, then in case one connection
got removed during a broadcast, there was a possibility
that the other good connection might not get the event.
```
2024-01-04 13:36:22 -06:00
Harrison Healey
59549653a7 MM-56147 Add GetPreferenceForUser plugin API (#25704)
* MM-56147 Add GetPreferenceForUser plugin API

* Change return type to non-pointer for preference consistency

* Fix merge
2024-01-04 13:50:19 -05:00
Ben Schumacher
edc305716f [MM-56284] Remove non-generic StoreResult (#25750) 2024-01-04 12:30:21 +01:00
Harrison Healey
502cd6ef7d MM-56082 Add PreferencesHaveChanged plugin hook (#25659)
* Add interface for PreferencesHaveChanged hook

* Add context to preference-related methods of App

* Implement PreferencesHaveChanged

* Re-add missing "fmt" import

* Update minimum server version for the new hook

* Remove pointers to be consistent with other preference APIs
2024-01-03 12:25:53 -05:00
Harshil Sharma
9016e30044 Fixed user reporting pagination (#25780)
* Fixed pagination and sorting for Postgres

* Updated tests
2024-01-02 10:41:00 -05:00
Felipe Martin
81a1d725a0 [WIP] [MM-55031] OAuth Outgoing Connection App integration (#25379)
* OAuthOutgoingConnection model

* added store

* make generated

* add missing license headers

* fix receiver name

* i18n

* i18n sorting

* update migrations from master

* make migrations-extract

* update retrylayer tests

* replaced sql query with id pagination

* fixed flaky tests

* missing columns

* missing columns on save/update

* typo

* improved tests

* remove enum from mysql colum

* add password credentials to store

* license changes

* OAuthOutgoingConnectionInterface

* Oauth -> OAuth

* make generated

* merge migrations

* renamed migrations

* model change suggestions

* refactor test functionsn

* migration typo

* refactor store table names

* updated sanitize test

* cleanup merge

* refactor symbol

* list endpoint

* oauthoutgoingconnection -> outgoingoauthconnection

* signature change

* i18n update

* granttype typo

* naming

* api list

* uppercase typo

* i18n

* missing license header

* fixed path in comments

* updated openapi definitions

* sanitize connections

* make generated

* test license and no feature flag

* removed t.fatal

* updated testhelper calls

* yaml schema fixes

* switched interface name

* suggested translation

* missing i18n translation

* address comments

* updated i18n
2023-12-26 10:46:20 +01:00
Harshil Sharma
d23c16b9aa Removed user bands (#25806) 2023-12-25 07:11:28 +00:00
Doug Lauder
2d1135ca46 Shared channels plugin APIs for MS Teams plugin (#25805)
New plugin APIs and hooks for accessing Shared Channels service via plugin. 

- RegisterPluginForSharedChannels(opts model.RegisterPluginOpts) (remoteID string, err error)
- UnregisterPluginForSharedChannels(pluginID string) error
- ShareChannel(sc *model.SharedChannel) (*model.SharedChannel, error)
- UpdateSharedChannel(sc *model.SharedChannel) (*model.SharedChannel, error)
- UnshareChannel(channelID string) (unshared bool, err error)
- UpdateSharedChannelCursor(channelID, remoteID string, cusror model.GetPostsSinceForSyncCursor) error
- SyncSharedChannel(channelID string) error
- InviteRemoteToChannel(channelID string, remoteID string, userID string) error
- UninviteRemoteFromChannel(channelID string, remoteID string) error

Hooks
- OnSharedChannelsSyncMsg(msg *model.SyncMsg, rc *model.RemoteCluster) (model.SyncResponse, error)
- OnSharedChannelsPing(rc *model.RemoteCluster) bool
2023-12-22 17:00:27 -05:00
Ibrahim Serdar Acikgoz
0f3553c8ab shared/filestore/localstore.go: remove unnecessary warning (#25801) 2023-12-22 19:26:57 +01:00
Agniva De Sarker
0496483a9e MM-56359: Skip TestUploadLicenseFile (#25800) 2023-12-22 13:21:33 +01:00
Miguel de la Cruz
0e60f3d542 Improves validation in getRolesByName endpoint (#25215)
* Improves validation in getRolesByName endpoint

* Updates the max constant and fixes linter

* Adds a mechanism to split roles in chunks in the webapp client

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-12-22 10:29:51 +01:00
dkkb
be24f108e1 Refactor server runJobs function. (#23854) 2023-12-22 09:51:40 +01:00
Harshil Sharma
a59f5ccded Added API to return user count bands and user count (#25796) 2023-12-21 13:00:19 +00:00
Agniva De Sarker
7dcf5f85a5 MM-56071: Track multiple threads (#25775)
We can have 2 types of threads open at the same time. One from
the thread view, and another from RHS.

We add another variable to distinguish between the 2 states.

In future, if we have the ability for more than 2 threads, then
we would need to track by threadID.

https://mattermost.atlassian.net/browse/MM-56071

```release-note
NONE
```
2023-12-21 08:42:58 +05:30
Miguel de la Cruz
3f2351c71f Remove user search skipped test (#25790) 2023-12-20 14:35:04 -06:00
Agniva De Sarker
b88a848c90 MM-53676: Bump dependencies (#25744)
https://mattermost.atlassian.net/browse/MM-53676
```release-note
NONE
```
2023-12-20 19:44:50 +05:30
Ben Schumacher
645d3a3b35 Add pagination example to Client4 (#25544)
* Add pagination example to Client4

* Break as soon less the expected items are returned

* Print users as directly after fetching them
2023-12-20 06:50:58 +01:00
Matthew Straughn
ebdc964161 Mm 16623 fix all initialism errors in channels/app and channels/api4 dir (#25743)
* updated go mod and sum files

* fixed var-naming errors in channels/app and channels/api4 dir

* Revert "updated go mod and sum files"

This reverts commit 088dd00a84.

* renamed a cost .
2023-12-20 06:46:54 +01:00
Doug Lauder
d56dc9d0ce Add Options field to RemoteClusters (#25771)
* add Option flag to RemoteClusters

* add Options column to RemoteClusters table
2023-12-19 13:01:35 -05:00
Ben Schumacher
799ec680bd [MM-29240] Improve buffer handling of UploadEmojiImage (#25536) 2023-12-19 10:00:01 +01:00
Christopher Poile
8a0cef910a MM-56273 - Update prepackaged calls to v0.22.2 (#25741) 2023-12-18 16:58:10 -05:00
Ben Schumacher
1fdddfe678 [MM-53269] Add configuration setting for integration requests timeout (#23805)
Co-authored-by: Michael Kochell <6913320+mickmister@users.noreply.github.com>
2023-12-18 16:07:00 +01:00
Hosted Weblate
7874daa6f7 Update translation files
Updated by "Cleanup translation files" hook in Weblate.

Translation: Mattermost/server
Translate-URL: https://translate.mattermost.com/projects/mattermost/server/
2023-12-18 15:22:05 +02:00
master7
58aa759445 Translated using Weblate (Polish)
Currently translated at 100.0% (2527 of 2527 strings)

Translation: Mattermost/server
Translate-URL: https://translate.mattermost.com/projects/mattermost/server/pl/
2023-12-18 15:22:05 +02:00
Tom De Moor
6443a1d42a Translated using Weblate (Dutch)
Currently translated at 100.0% (2527 of 2527 strings)

Translation: Mattermost/server
Translate-URL: https://translate.mattermost.com/projects/mattermost/server/nl/
2023-12-18 15:22:05 +02:00
MArtin Johnson
e5de17674e Translated using Weblate (Swedish)
Currently translated at 100.0% (2527 of 2527 strings)

Translation: Mattermost/server
Translate-URL: https://translate.mattermost.com/projects/mattermost/server/sv/
2023-12-18 15:22:05 +02:00
Tom De Moor
cf3a738220 Translated using Weblate (Dutch)
Currently translated at 99.8% (2522 of 2527 strings)

Translation: Mattermost/server
Translate-URL: https://translate.mattermost.com/projects/mattermost/server/nl/
2023-12-18 15:22:05 +02:00
Gabriel Casaro
be8b552c3a Translated using Weblate (Portuguese (Brazil))
Currently translated at 85.1% (2153 of 2527 strings)

Translation: Mattermost/server
Translate-URL: https://translate.mattermost.com/projects/mattermost/server/pt_BR/
2023-12-18 15:22:05 +02:00
Tom De Moor
30ad3c714b Translated using Weblate (Dutch)
Currently translated at 99.7% (2520 of 2527 strings)

Translation: Mattermost/server
Translate-URL: https://translate.mattermost.com/projects/mattermost/server/nl/
2023-12-18 15:22:05 +02:00
master7
be084a9a59 Translated using Weblate (Polish)
Currently translated at 100.0% (2527 of 2527 strings)

Translation: Mattermost/server
Translate-URL: https://translate.mattermost.com/projects/mattermost/server/pl/
2023-12-18 15:22:05 +02:00
Paul Stern
f69620f22c [MM-55299] Migrate server/channels/app/notification.go to make use of GenericStoreResult (#25746)
* [MM-55299] Make use of GenericStoreResult

Make use of GenericStoreResult in server/channels/app/notification.go

Introduce new result variables to workaround the IncompatibleAssign
error.

* [MM-55299] Make use of GenericStoreResult

Use GenericStoreResult for profiles, cmn properties, groups map.
Introduce new result variables for them to workaround IncompatibleAssign
error.

* Resolve merge conflict. Accept incoming master's change

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-12-18 13:13:56 +01:00
Paul Stern
000ee45b5d [MM-55303] channels/app/webhook: Make use of GenericStoreResult (#25722)
Make use of Generic store result in channels/app/webhook.go.
Introduce new variable resultU (result user) to workaround incompatible
assign error (since generics have been introduced).
2023-12-18 07:08:38 +01:00
Devin Binnie
5c061a6f75 [MM-56206] Allow for proper paging and sorting (#25726)
Co-authored-by: Mattermost Build <build@mattermost.com>
2023-12-18 05:56:06 +05:30
Tom De Moor
485ae6ae4e fixing typo (#25714)
Co-authored-by: Tom De Moor <tom@controlaltdieliet.be>
2023-12-15 12:37:02 -05:00
Antonis Stamatiou
c5f5800814 feat: Migrate docker preview repo stuff in monorepo (#25698)
* feat: Migrate docker preview repo stuff in monorepo

* docs: Update docs [skip ci]

* config: Update configuration to minimum [skip ci]
2023-12-15 12:28:02 +02:00
mattermod
bb9c695c1b Update latest version to 9.3.0 2023-12-14 17:01:02 +00:00
Harshil Sharma
97a23d791e New report router and user reporting refactoring (#25713)
* Added materialized view migration

* Renamed mat view

* Added channel membership mat view and indexes

* Added channel membership mat view and indexes

* Added new index

* WIP

* Simplifying user reporting code

* Created app and API layer for cahnnel reporting, reporting refactoring in general

* New router

* Remobved channel reporting meanwhile

* Upodated autogenerated stuff

* Lint fix

* Fixed typo

* api vet

* i18n fix

* Fixed API vetting and removed channel reporting constants

* yaml

* removed app pagination tests
2023-12-14 10:49:19 -05:00
Ben Schumacher
563f51f3db Allow developers to set a custom go version via env variables (#25650) 2023-12-14 09:58:11 +01:00
Agniva De Sarker
9ae451a121 MM-55476: Optimize PostStore.Get (#25448)
While loading a thread, we were unnecessarily re-counting the replies
for all posts in a thread even if they would be the same number.

While this is needed in other queries where the posts can be from different
threads or they can be random ids, but to load a single post thread, there
is no need to recompute it again and again.

Therefore, we use a CTE to precompute the replycount and then just plug in
the value in the subsequent query. This gives an improvement in the query
plan as well:

OLD:
```
explain (analyze, buffers) SELECT p.id, p.rootid, p.createat, (SELECT count(*) FROM Posts WHERE Posts.RootId = (CASE WHEN p.RootId = '' THEN p.Id ELSE p.RootId END) AND Posts.DeleteAt = 0) as ReplyC
ount FROM Posts p WHERE (p.Id = 'h3cer597jb8abbcbitpghpomua' OR p.RootId = 'h3cer597jb8abbcbitpghpomua') AND p.DeleteAt = 0;
                                                                            QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Bitmap Heap Scan on posts p  (cost=45.39..1042149.61 rows=2748 width=49) (actual time=3.156..7906.215 rows=5353 loops=1)
   Recheck Cond: (((id)::text = 'h3cer597jb8abbcbitpghpomua'::text) OR (((rootid)::text = 'h3cer597jb8abbcbitpghpomua'::text) AND (deleteat = 0)))
   Filter: (deleteat = 0)
   Heap Blocks: exact=5308
   Buffers: shared hit=610244
   ->  BitmapOr  (cost=45.39..45.39 rows=2748 width=0) (actual time=0.918..0.920 rows=0 loops=1)
         Buffers: shared hit=47
         ->  Bitmap Index Scan on posts_pkey  (cost=0.00..1.68 rows=1 width=0) (actual time=0.028..0.028 rows=1 loops=1)
               Index Cond: ((id)::text = 'h3cer597jb8abbcbitpghpomua'::text)
               Buffers: shared hit=4
         ->  Bitmap Index Scan on idx_posts_root_id_delete_at  (cost=0.00..42.34 rows=2747 width=0) (actual time=0.889..0.890 rows=5352 loops=1)
               Index Cond: (((rootid)::text = 'h3cer597jb8abbcbitpghpomua'::text) AND (deleteat = 0))
               Buffers: shared hit=43
   SubPlan 1
     ->  Aggregate  (cost=378.10..378.11 rows=1 width=8) (actual time=1.474..1.474 rows=1 loops=5353)
           Buffers: shared hit=604889
           ->  Index Only Scan using idx_posts_root_id_delete_at on posts  (cost=0.57..343.85 rows=13699 width=0) (actual time=0.016..1.039 rows=5352 loops=5353)
                 Index Cond: ((rootid = (CASE WHEN ((p.rootid)::text = ''::text) THEN p.id ELSE p.rootid END)::text) AND (deleteat = 0))
                 Heap Fetches: 0
                 Buffers: shared hit=604889
 Planning Time: 0.194 ms
 Execution Time: 7906.846 ms
```

NEW:
```
explain analyze with replycount as (select count(*) as num from posts where rootid='h3cer597jb8abbcbitpghpomua' and deleteat=0)
select id, rootid, createat, replycount.num from posts, replycount where id='h3cer597jb8abbcbitpghpomua' or rootid='h3cer597jb8abbcbitpghpomua' and deleteat=0;
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Nested Loop  (cost=123.16..3215.48 rows=2748 width=49) (actual time=2.960..9.775 rows=5353 loops=1)
   ->  Aggregate  (cost=77.78..77.79 rows=1 width=8) (actual time=1.455..1.456 rows=1 loops=1)
         ->  Index Only Scan using idx_posts_root_id_delete_at on posts posts_1  (cost=0.57..70.91 rows=2747 width=0) (actual time=0.056..1.145 rows=5352 loops=1)
               Index Cond: ((rootid = 'h3cer597jb8abbcbitpghpomua'::text) AND (deleteat = 0))
               Heap Fetches: 0
   ->  Bitmap Heap Scan on posts  (cost=45.39..3110.20 rows=2748 width=41) (actual time=1.501..7.747 rows=5353 loops=1)
         Recheck Cond: (((id)::text = 'h3cer597jb8abbcbitpghpomua'::text) OR (((rootid)::text = 'h3cer597jb8abbcbitpghpomua'::text) AND (deleteat = 0)))
         Heap Blocks: exact=5308
         ->  BitmapOr  (cost=45.39..45.39 rows=2748 width=0) (actual time=0.797..0.798 rows=0 loops=1)
               ->  Bitmap Index Scan on posts_pkey  (cost=0.00..1.68 rows=1 width=0) (actual time=0.014..0.014 rows=1 loops=1)
                     Index Cond: ((id)::text = 'h3cer597jb8abbcbitpghpomua'::text)
               ->  Bitmap Index Scan on idx_posts_root_id_delete_at  (cost=0.00..42.34 rows=2747 width=0) (actual time=0.782..0.782 rows=5352 loops=1)
                     Index Cond: (((rootid)::text = 'h3cer597jb8abbcbitpghpomua'::text) AND (deleteat = 0))
 Planning Time: 0.220 ms
 Execution Time: 10.052 ms
(15 rows)
```

Observe the `loops=5353` in the first query, and `loops=1` in the next.

https://mattermost.atlassian.net/browse/MM-55476

```release-note
Optimize createPost performance
```

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-12-14 14:09:29 +05:30