Commit Graph

830 Commits

Author SHA1 Message Date
Nick Misasi
fee6606606 Fix flaky test api4/Test_applyIPFilters (#25437)
Automatic Merge
2023-11-14 23:52:23 +02:00
Devin Binnie
1bd72bdb99 [MM-55014][MM-55015] Add last login timestamp for users, add materialized view and refresh job to keep track of post stats for Postgres (#25152)
* [MM-55014][MM-55015] Add last login timestamp for users, add materialized view and refresh job for Postgres

* Check fixes

* Fix type issue

* Add verification that lastlogin was updated

* PR feedback

* Morge'd

* Morge'd again

* Merge'd

* Update admin setting strings

* WIP

* PR feedback

* Oops

* Fix i18n

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-11-14 11:26:27 -05:00
Aydın Davutoğlu
bc58437e63 [MM-55302]Migrate server/channels/app/user.go to make use of GenericStoreResult (#25409) 2023-11-14 15:13:40 +01:00
Nick Misasi
e1c851a3ca [CLD-6324] Cloud IP Filtering (#24726)
* Initial comit for ip filtering service implementation

* Add audit logs for IP Filters

* start of webapp work

* Stashing

* Updates based on Agniva's feedback around service vs einterface

* Updates completed

* Commit before refactoring, everything's working

* First pass of cleanup complete, front-end tests added

* actually add files

* Updates to some translation strings, running i18n-extract

* Lock everything behind a feature flag

* Fix tests, try to fix some linter stuff

* Fixed linter for JS, on to scss

* Fixed linter for scss

* Fix linter

* More fixes for pipeline

* Support for IPV6

* Remove tsx file that was removed in masteR

* Revert package.json and package-lock.json to master, add cidr-regex dep into channels/package.json

* Another commit to force fix Github

* Fixes around IPV6. Some suggestions from Matt re: UX review. Fixing pipelines for tests and types on new cidr-regex package

* Changes to address Matt's feedback

* A few more changes for clean up

* Add support for permissions

* Fix vet for OpenAPI spec

* Actually add the yaml file for openapi

* Add permission migration to allow support for IP Filtering

* Fix tests

* Final fixes from Matt

* Remove cancel button from page, update link outs to documentation

* Update test to account for removed cancel button

* Adjustments based on feedback from Harrison

* More fixes from PR feedback

* Add a t to fix translations that doesn't seem to be breaking anyone else?

* More fix

* updates for PR feedback

* Fix linter

* Fix types

* Now fix the linter again

* Add back tests because Harrison was able to get them running

* Adjustments for PR feedback

* Remove admin_definition.jsx

* Fix linter

* [CLD-6453] IP Filtering notification email for sysadmins (#25224)

* Initial commit for IP filtering alert email

* Updates to style for email, addition of ip_filtering email:

* Fix pipelines

* Adjustments from Matt's feedback

* Padding changes

* template diff (#25249)

Co-authored-by: Gabe Jackson <3694686+gabrieljackson@users.noreply.github.com>

* Fix hardcoded true, remove bool return value

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Gabe Jackson <3694686+gabrieljackson@users.noreply.github.com>

* Lock feature behind enterprise license. Drop cidr-regex in favour of ipaddr.js dependency. Refactor isIpAddressWithinRanges to use ipaddr.js

* Add a couple server tests

* fix linter

* Fix types from merge conflicts

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Gabe Jackson <3694686+gabrieljackson@users.noreply.github.com>
2023-11-14 09:12:04 -05:00
KIMBOH LOVETTE
29cd6177c0 [GH-21565]-Add the request context and logger to all public methods in server/channels/app/audit.go (#25368) 2023-11-14 13:34:47 +01:00
Paul-Stern
5f8133254d [MM-55304] Migrate "server/channels/store/searchlayer/user_layer.go" to use GenericStoreResult (#25351)
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
2023-11-14 10:18:10 +01:00
Christopher Poile
5dea20dd3e MM-54963 - Update prepackaged calls to v0.21.0
Co-authored-by: Mattermost Build <build@mattermost.com>
2023-11-13 21:27:09 -05:00
Harrison Healey
9397970644 MM-55267 Add ability for server-side Markdown code to understand emojis (#25332)
* MM-55267 Add ability for server-side Markdown code to understand emojis

* Remove unused regex
2023-11-13 14:38:05 -05:00
Agniva De Sarker
ec88ab4ee9 MM-55295: Improve DB performance of some APIs (#25318)
Load tests show that channelstore.GetMember and
channelstore.GetMembersForUser are among the chief
queries that take up CPU in the DB.

In this PR, we attempt some strategic optimizations to
reduce/optimize calls to channelstore.GetMember

1. Optimize  `(a *App) HasPermissionToChannel`

We replace GetChannelMember with GetAllChannelMembersForUser
because it's cache backed. So although it gets more data,
it does not hit the DB and saves some latency.

2. Optimize getPostsForChannelAroundLastUnread

We repace getChannelMember with getChannelMemberOnly
which is a lite version of the store call which queries
just the ChannelMembers table. This is because
in the app layer, we just use the LastViewedAt attribute.
Therefore, there is no reason to join with 5 tables when
a single table can do the job.

3. Optimize publishWebsocketEventForPermalinkPost

We use GetAllChannelMembersById instead of GetChannelMembersPage
which again joins with a lot of other tables.

4. Optimize countMentionsFromPost

Again, we replace GetChannelMember which is a costly call joining
multiple tables, with GetAllChannelMembersNotifyPropsForChannel
which is cache-backed and gives us just what we need in the app
layer - notify props.

```release-note
Make small optimizations in several DB calls:
- App.HasPermissionToChannel
- getPostsForChannelAroundLastUnread
- publishWebsocketEventForPermalinkPost
- countMentionsFromPost
```

https://mattermost.atlassian.net/browse/MM-55295

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-11-13 21:15:57 +05:30
Nicolas Le Cam
1ed5d87342 Switch to docker compose v2 (#25048) 2023-11-13 14:30:24 +01:00
Soumyadip Moni
3e3087955b make use of GenericStoreResult in telemetry.go (#25388) 2023-11-13 11:11:17 +05:30
M-ZubairAhmed
9ac389f506 [MM-53102] Add support for multi-word highlights without notifications in web (#24050)
- Adds a new section under settings/notifications for adding custom multi-word keywords that get highlighted without notification
- Adds a new classname for highlighting words although the styling is the same as mentions highlights
- Added a few components to the ReduxFromProps pattern
- Adds supported type for the hook of PluginComponent type
- Add upsell for highlight without notification
- Moved 'setting_item.tsx' to the components folder
- Improved prop names and function structure for setting_item, setting_item_max and setting_item_min
- Moved 'toggle_modal_button.tsx' to the components folder
- Removed t and utility messages from a few components
- Fixed bug where the tooltip was not getting rendered on restrictedButtons
- Improved the mobile view of the settings modal
- Adds E2E for the feature
2023-11-11 19:33:28 +05:30
Jesse Hallam
48bf4e9bd8 Fix issue 25390 (#25403) 2023-11-10 21:55:33 -04:00
Ben Schumacher
612e4458ef [MM-55409] Enable revive linter on enterprise code (#25391) 2023-11-10 15:24:12 +01:00
Paul-Stern
7d92e71daf [MM-55299] Migrate server/channels/app/notification.go to make use of GenericStoreResult (#25338) 2023-11-10 14:45:02 +01:00
Paul-Stern
a602dcdfb2 [MM-55305] Migrate "server/channels/store/sqlstore/post_store.go" to use GenericStoreResult (#25344)
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
2023-11-10 14:43:39 +01:00
Paul-Stern
3a373fffc0 [MM-55300] Migrate "server/channels/app/post.go" to make use of GenericStoreResult (#25339)
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
2023-11-10 14:41:08 +01:00
Ben Schumacher
a31b899818 Wipe LDAP filter when running make config-ldap (#25393) 2023-11-10 09:02:16 +01:00
Pantelis Vratsalis
d9f5cd2740 [MM-52445] Fix double url encoding of oauth redirect URI params (#23176)
* [MM-52445] Fix double url encoding of oauth redirect URI params

* Additional test based on code review

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-11-09 19:18:24 +01:00
Harrison Healey
ef66f7beab MM-54238 Add WebSocket broadcast hook and don't broadcast when other users were mentioned (#24641)
* MM-54238 Initial implementation

* MM-54238 Move websocket hook into app package

* MM-54238 Add tests for mentions in posted websocket messages

* Fix styling

* Fix other styling

* Idiomatic ID naming for new code

* Fix more styles

* Separate hooks to add mentions and followers

* Improved error handling for invalid types in hooks

* Rename HasChanges to ShouldProcess

* Pass broadcast hooks through hubStart

* Add test helper for asserting json unmarshaling

* Fix missing arguments in tests

* Ensure broadcast hooks are sent across the cluster and not to users

* Ensure tests actually cover following a post

* Fix code broken by merge

* Go vet again...

* Deep copy event before processing it with hooks

* Replace RemoveBroadcastHooks with WithoutBroadcastHooks

* Address feedback

* Add helper to fix type information for hook args

* Wrap WebSocketEvent and simplify BroadcastHook

* Address feedback

* Address feedback
2023-11-08 16:17:07 -05:00
Ben Schumacher
5e62ba8ccc Fix bad merge (#25371) 2023-11-08 12:18:33 -05:00
Scott Bishel
818a48190e MM-54774 - update session roles when promote/demote guests (#25156)
* update session roles from user roles.

* update so user is not retrieved again

* return error, rather than log warning

* Update session.go

Fix bad merge

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-11-08 07:27:42 -07:00
KIMBOH LOVETTE
a6ba7163c8 [GH-21566] Add request context and logger to all public methods in server/channels/app/authentication.go (#25270) 2023-11-08 14:39:09 +01:00
Paul Vrn
dd46afacd6 [MM-55004] Add warning log message when app run as root (#25213)
* feat: add log warn when app is run as root user

* Modify warning message

* Move warning message condition to L415

* move check to cmd/mattermost/main.go

* Update server/channels/app/server.go

Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>

* Moved checkForRootUser to root.go

* format root.go

* remove unnecessary space

---------

Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2023-11-08 11:39:37 +01:00
Agniva De Sarker
3563e56a77 MM-54998: Optimize JSON marshalling in websocket broadcast (#25286)
Marshalling a json.RawMessage is not zero overhead. Instead,
it compacts the raw message which starts to have an overhead
at scale.

https://github.com/golang/go/issues/33422

Since we have full control over the message constructed, we
can simply write the byte slice into the network stream.
This gives considerable performance boost.

```
goos: linux
goarch: amd64
pkg: github.com/mattermost/mattermost/server/public/model
cpu: Intel(R) Core(TM) i5-8265U CPU @ 1.60GHz
             │   old.txt    │              new_2.txt              │
             │    sec/op    │   sec/op     vs base                │
EncodeJSON-8   1640.5n ± 2%   289.6n ± 1%  -82.35% (p=0.000 n=10)

             │  old.txt   │             new_2.txt             │
             │    B/op    │    B/op     vs base               │
EncodeJSON-8   528.0 ± 0%   503.0 ± 0%  -4.73% (p=0.000 n=10)

             │  old.txt   │             new_2.txt              │
             │ allocs/op  │ allocs/op   vs base                │
EncodeJSON-8   5.000 ± 0%   4.000 ± 0%  -20.00% (p=0.000 n=10)
```

P.S. No concerns over changing the model API because we are
still using 0.x

https://mattermost.atlassian.net/browse/MM-54998

```release-note
Improve websocket event marshalling performance
```
2023-11-08 12:15:24 +05:30
Aydın Davutoğlu
b19b62b83f Add the request context and logger to all public methods in server/channels/app/auto_responder.go (#25295)
Co-authored-by: Mattermost Build <build@mattermost.com>
2023-11-07 19:25:52 +01:00
Paul-Stern
9596c46115 Mm 55298 generic integration action (#25320) 2023-11-07 12:07:08 +01:00
Ben Schumacher
b0bf5d0765 [MM-55053] Use request-specific logger in channels/app/file.go (#25037) 2023-11-07 10:04:16 +01:00
mattermod
45d6fb122f Update latest version to 9.1.2 2023-11-07 07:36:55 +00:00
Julien Tant
4dbf1f8c26 [MM-54029] IDP descriptor can any string (#24250)
Co-authored-by: Mattermost Build <build@mattermost.com>
2023-11-06 15:24:59 -07:00
Claudio Costa
c516d5cb15 [MM-55164] Fix deadlock on shutdown due to stuck goroutine (#25227)
* Update code.sajari.com/docconv

* go mod tidy
2023-11-06 11:00:58 -06:00
Christopher Speller
3b17fc4fdb Update playbooks to v1.39.1 (#25281) 2023-11-06 07:50:24 -08:00
Ben Schumacher
42822ccdb8 Remove dead method Handle404 (#25315) 2023-11-06 14:54:07 +01:00
Paul-Stern
2a5e04d9b2 [MM-55213] GenericStoreResult in channels (#25234) (#25265) 2023-11-06 13:52:12 +01:00
Ben Schumacher
486e836b83 Fix racy test issues (#24971) 2023-11-06 12:26:17 +01:00
Scott Bishel
db238a205c MM-55202-remove 'boards' from help command (#25225)
* remove 'boards' from help command

* remove boards from /help command

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-11-03 15:27:34 +01:00
Antonis Stamatiou
fc87054f14 feat: Introduce go-version file to support multiple go versions on CI (#25271) 2023-11-03 16:25:54 +02:00
Syed Fazil
b2abd533a1 [MM-55212] Migrate server/channels/app/channel.go to make use of GenericStoreResult (#25262)
* Make use of GenericStoreResult in channels/app/channel.go

* Make use of GenericStoreResult in channels/app/channel.go

* Make use of GenericStoreResult in channels/app/channel.go

* Make use of GenericStoreResult in channels/app/channel.go

* Make use of GenericStoreResult in channels/app/channel.go

* Make use of GenericStoreResult in channels/app/channel.go

* Make use of GenericStoreResult in channels/app/channel.go

---------

Co-authored-by: fazil-syed <fazil@Gojo>
Co-authored-by: Mattermost Build <build@mattermost.com>
2023-11-03 12:50:25 +01:00
Catena cyber
04e7b6bc9e Applies perfsprint linter (#24999)
Replacing usages of fmt.Sprint with faster alternatives

Signed-off-by: Philippe Antoine <contact@catenacyber.fr>
2023-11-03 11:18:18 +01:00
Ben Schumacher
43f4734eae [MM-55051] Assign correct timestamp to support packet files (#25035)
* Assign correct timestamp to support packet files

* Fix linter error

* Add back compression

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-11-03 09:48:18 +01:00
Ben Schumacher
e8569c91af [MM-32406] Introduce trace logging level for LDAP messages (#25118) 2023-11-03 08:06:16 +01:00
Amy Blais
aecfba2cfb Update version.go (#25276)
Automatic Merge
2023-11-02 15:52:23 +01:00
Ben Schumacher
4948aa106b [MM-54403] Introduce separate AdvanctedLogging levels for LDAP messages (#24873) 2023-11-02 12:31:57 +01:00
Vishal
dfb561a641 [MM-54357] Recent Mentions is showing posts for other similar named users. (#25010)
* Handle double quotes in Postgres
* quote the username when performing the search
2023-11-02 11:05:44 +05:30
Ben Schumacher
274adbf077 Reduce usage of mlog.Any (#25180) 2023-11-01 12:57:13 +01:00
Ben Schumacher
73fba481ed [MM-55170] Improve error message when trying to activate a plugin in an unsupported environment (#25160) 2023-11-01 12:18:50 +01:00
Daniel Espino García
b0dc74957a Fix flaky test introduced in #24602 (#25236) 2023-11-01 11:23:56 +01:00
mattermod
ba335b4025 Update latest version to 9.1.1 2023-10-31 16:14:24 +00:00
Ibrahim Serdar Acikgoz
1cd439f8e0 db/migrations: drop nextsyncat and description columns from shared remotes (#25064) 2023-10-31 15:25:16 +03:00
Ben Schumacher
a4ece51391 [MM-45065] Switch all api4 methods to use the new logger (#24896) 2023-10-31 11:00:22 +01:00