Commit Graph

50 Commits

Author SHA1 Message Date
Ben Schumacher
0d5a8b8841 Add whitespace linter (#24855) 2023-10-11 10:13:36 +02:00
Jesse Hallam
47bfa2b66b Pass a logger instead of embedding on job (#24650)
* pass a logger instead of embedding on job

* leverage mlog.Millis

* use worker logger with HandleJobPanic

* rely on existing LogClone instead

* guard Job.LogClone against nil Job

* s/workername/worker_name

* Revert "rely on existing LogClone instead"

This reverts commit 17303cbac9.

* Revert "guard Job.LogClone against nil Job"

This reverts commit f1ae22dee5.
2023-10-09 11:04:55 -03:00
Ben Schumacher
aad25be4e1 [MM-54434] Use job.Logger to capture ldap logs (#24493) 2023-10-06 22:43:21 +02:00
Ibrahim Serdar Acikgoz
53a0f8da27 Add limiting ability to log fields (#24251) 2023-09-28 19:28:40 +03:00
Ben Schumacher
30b12f199b [MM-54132] Use annotated logger for log messages from jobs (#24275) 2023-09-07 08:50:22 +02:00
Ben Cooke
791ee40568 [MM-53291] Data retention improvements (#24253)
* adding new migration for RetentionIdsForDeletion, changing logic for deleting orphaned reactions. Updating delete user and channel endpoints to remove respective reactions
2023-09-06 08:25:27 -04:00
Ben Schumacher
b2f36c7cdf [MM-54318] Add file storage information to support package (#24474)
* Reorder stats in support package struct

* Add file storage information to support package

* Add clusterID

* Change method name to DriverName

* Ordering

* Fix test
2023-09-06 09:21:25 +02:00
M-ZubairAhmed
bac317dfe5 MM-54181 : Clean up code around deprecated Gfycat (#24343)
* first pass of removal

* first pass of removal

* Update admin_console.js

* rev beta

* remove more

* Update initial_state.ts

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-09-05 16:40:37 -04:00
Felipe Martin
9f7521d003 Move request package into public/shared folder (#24420)
* move request package to public/shared

* updated app-layers

* update app layer

* remove original package location
2023-09-05 09:47:30 +02:00
Harrison Healey
f2fcf3d839 MM-53107 Add setting to limit the max number of markdown nodes (#24414)
* MM-53107 Add setting to limit the max number of markdown nodes

* Add extra validation for a couple config fields

* Add new setting to telemetry
2023-08-31 14:36:24 -04:00
Agniva De Sarker
b84236d555 MM-49627: Suppress transformation of embedded images via image proxy (#24401)
There was no check for embedded images which made "copy text" paste
the siteURL instead of the actual image content.

This does not change the behavior of actually rendering the image
which is how other sites behave as well.

https://mattermost.atlassian.net/browse/MM-49627
```release-note
NONE
```
2023-08-30 22:33:42 +05:30
Matthew Straughn
2e449413e8 fixed initialism errors in accordance with golang conventions in 7 files. (#24287) 2023-08-22 10:15:51 +03:00
Ben Schumacher
3ee5432664 [MM-53968] Includes mattermost-plugin-api into the mono repo (#24235)
Include https://github.com/mattermost/mattermost-plugin-api into the mono repo

Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
Co-authored-by: Michael Kochell <mjkochell@gmail.com>
Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: Alex Dovenmuehle <alex.dovenmuehle@mattermost.com>
Co-authored-by: Michael Kochell <6913320+mickmister@users.noreply.github.com>
Co-authored-by: Christopher Poile <cpoile@gmail.com>
Co-authored-by: İlker Göktuğ Öztürk <ilkergoktugozturk@gmail.com>
Co-authored-by: Shota Gvinepadze <wineson@gmail.com>
Co-authored-by: Ali Farooq <ali.farooq0@pm.me>
Co-authored-by: Maria A Nunez <maria.nunez@mattermost.com>
Co-authored-by: Daniel Espino García <larkox@gmail.com>
Co-authored-by: Christopher Speller <crspeller@gmail.com>
Co-authored-by: Alex Dovenmuehle <adovenmuehle@gmail.com>
Co-authored-by: Szymon Gibała <szymongib@gmail.com>
Co-authored-by: Lev <1187448+levb@users.noreply.github.com>
Co-authored-by: Jason Frerich <jason.frerich@mattermost.com>
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Artur M. Wolff <artur.m.wolff@gmail.com>
Co-authored-by: Madhav Hugar <16546715+madhavhugar@users.noreply.github.com>
Co-authored-by: Joe <security.joe@pm.me>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: José Peso <trilopin@users.noreply.github.com>
2023-08-21 09:50:30 +02:00
Scott Bishel
faea229921 MM-53688 Remove boards dataretention (#24231)
* remove boards data-retention

* remove another variable

* update snapshots, to remove boards

* need to keep config settings until focalboard is updated

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-08-15 08:27:45 -06:00
Jesse Hallam
e39b485c4b Fail vs. fatal on store startup (#24170)
If the store fails to initialize (e.g. run a migration), it would `log.Fatal` and then `os.Exit`. Unfortunately, this trips up `TestMain`, which happily keeps running tests, now guaranteed to fail.

Avoid this by instead returning an error from the store initialization, handling appropriately at the layer above.
2023-08-04 23:05:01 -03:00
Julien Tant
d1691833ad [MM-53881] Make sure export filestore won't use bifrost (#24169) 2023-08-02 09:44:20 -07:00
Jesse Hallam
933b0eb89c MM-53877: fix export config settings (#24167)
* wip test

* test and use Export* config correctly
2023-08-02 10:24:52 -03:00
Agniva De Sarker
6d6e589c11 MM-53747: Create job to encode older image paths (#24073)
Bifrost now encodes all image paths. Due to this
one-way translation, we need to encode all the older
image paths as well.

After this is done, we can remove the double-lookup.

https://mattermost.atlassian.net/browse/MM-53747

```release-note
NONE
```

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-07-25 08:38:35 +05:30
Agniva De Sarker
feeb0cd2ef MM-53747: Bifrost: write original paths (#24079)
Now that we have identified the true fix
from Bifrost side, we will slowly revert
the changes that have happened.

Starting with this first fix, we will revert
to writing out non-encoded paths while still
doing the double read. This will break uploading
files with + for a short while, and then
the new Bifrost fix should fix that as well.

https://mattermost.atlassian.net/browse/MM-53747

```release-note
NONE
```
2023-07-21 08:59:59 +05:30
Julien Tant
077c16ef61 [MM-53454] Add export file settings + slash command for public link (#23915)
Co-authored-by: Mattermost Build <build@mattermost.com>
2023-07-19 13:01:39 -07:00
Agniva De Sarker
c1dfdb867a MM-53709: Handle older non-encoded paths in bifrost (#24058)
We do a check to see if a non-encoded path is present
or not, and in that case, choose not to encode it.

This accrues an additional StatFile call. But after
we have encoded all paths to the new style, we will
get rid of this.

https://mattermost.atlassian.net/browse/MM-53709

```release-note
NONE
```
2023-07-19 22:32:04 +05:30
Ben Cooke
bf0e0ed45a [MM-53466] Remove concurrency from data retention delete batching (#23927)
* Add TimeBetweenBatches config value for data retention

* adding feature flag for data retention concurrency
---------

Co-authored-by: Julien Tant <julien@craftyx.fr>
Co-authored-by: Mattermost Build <build@mattermost.com>
2023-07-05 11:09:03 -04:00
Daniel Espino García
fda5dd8468 Remove guest badge (#23825)
* Remove guest badge

* Address feedback

* Fix lint, test and i18n

* Fix test
2023-06-30 11:06:18 +02:00
Agniva De Sarker
71c5a8512c MM-51753: URL encode the file path in S3 uploads (#23849)
If the request is routed via bifrost, then we need to encode the path
to avoid signature validation errors.
This happens because in bifrost, we are signing the URL outside the SDK
and therefore the signature sent from the bifrost client
will contain the encoded path, whereas the original path is sent un-encoded.

More info at: a57c4d9278/aws/signer/v4/v4.go (L8)

https://mattermost.atlassian.net/browse/MM-51753
```release-note
NONE
```
2023-06-27 09:54:53 +05:30
Felipe Martin
cad2df1259 [MM-52943] fix: enable Elasticsearch properly on restart (#23806)
* fix: enable elasticsearch properly on restart

* use correct method under IsEnabled

* Empty-Commit to retrigger workflow

---------

Co-authored-by: Antonis Stamatiou <stamatiou.antonis@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2023-06-23 14:59:56 +02:00
Ibrahim Serdar Acikgoz
0904be43f8 [MM-53195] services/telemetry: regen client config right after assigning diagnostic ID to platform (#23773) 2023-06-19 18:48:29 +03:00
Michael Kochell
d403c034d4 [MM-48014] Auto-enable Apps Bar by moving EnableAppBar setting to DisableAppBar (#23010)
* auto-enable Apps Bar by moving EnableAppBar setting to DisableAppBar

* update playwright config

* remove EnableAppBar config setting to avoid confusion

* small fixes

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>
2023-06-15 12:33:06 -04:00
Agniva De Sarker
c249ba4a66 MM-52532: Fix golangci warnings (#23709)
https://mattermost.atlassian.net/browse/MM-52532

- Replace golint with revive
- Add makezero linter
- Fix all the required linter failures

Some issues in enterprise and public modules
are yet to be fixed. We send this to expediate things.
2023-06-13 14:08:36 +05:30
Scott Bishel
d0ad46b496 MM-52915 - Remove boards product config setting and system console (#23526)
* Remove boards product config setting and system console

* create config setting to enable/disable playbooks product

* fix to config name

* fix typo

* remove lingering ProductSettings references

* remove lingering ProductSettings references

* revert changes to package-lock.json

* Revert "revert changes to package-lock.json"

This reverts commit 1a7d98ace3.

* revert changes to package-lock.json

* fix build issues

* another fix

* update webap test

* i18n-extract

* Revert "i18n-extract"

This reverts commit c0a98c74ea.

* fix bad i18n-extract

* fix bad i18n-extract

* fix tests from bad merge

* more translation fixes

* more translation fixes

* updates from self-review

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-06-12 18:52:19 +02:00
Agniva De Sarker
efaa6264cc MM-53032: Fix module path after repo rename (#23689)
It was a good decision in hindsight to keep the public module as 0.x
because this would have been a breaking change again.

https://mattermost.atlassian.net/browse/MM-53032
```release-note
Changed the Go module path from github.com/mattermost/mattermost-server/server/v8 to github.com/mattermost/mattermost/server/v8.

For the public facing module, it's path is also changed from github.com/mattermost/mattermost-server/server/public to github.com/mattermost/mattermost/server/public
```
2023-06-11 10:54:35 +05:30
Jesse Hallam
305fac6507 Service environment (#23443)
* fix fileutils.TestFindFile on MacOS

* introduce model.ExternalServiceEnvironment

* pick license public key from external service env

* pick Stripe public key from external service env

* pick Rudder key from external service env

* configure Sentry DSN from external service env

* always log external_service_environment, Unsetenv

* clear faked BuildEnv, improve logging

* strip out unset GOTAGS

* fix Sentry tests

* simplify to just ServiceEnvironment

* relocate ServiceEnvironment in client config

* initialize CWS URLs based on service environment

* unset rudder key for boards dev

* harden service environment to avoid accidental production

* fix TestSentry again

* fix DEFAULT -> ENTERPRISE

* s/dev/test when naming playbooks rudder key

* simplify boards rudder key switch

* use uniform rudderKey variable names

* retain compatibility with existing pipeline

* reduce to just production/test

* unit test with valid test license

* simplify Playbooks telemetry initialization

* restore dev service environment

* emit ServiceEnvironment when running e2e tests
2023-06-07 10:15:33 -03:00
Vishal
9399ce8637 [MM-47751][MM-48102] MPA: Send Persistent Notifications (#21619)
* MM-46410: adds urgency on mention counts

We have introduced priority for posts in
https://github.com/mattermost/mattermost-webapp/pull/10951.
We do need to color the mention badges in the webapp with a prominent
color when a mention is posted in an urgent message.
A thread has urgent mentions if the root post is marked as urgent, and
the replies contain mentions to the user viewing the thread.

This PR adds two columns, urgentmentioncount, and isurgent, in
channelmembers, and threads tables respectively.
Furthermore when asking for team/thread mention counts, we also return
urgent mention counts for the user.

* Adds PostAcknowledgements table and apis

* job init and fetch mentions

* add-migrations

* delete-expired

* send-notifications

* Fetches post priority in batches

* stop-notifications

* stop-notification-on-reply

* MM-47750: Adds PostAcknowledgements table and apis

- Adds post acknowledgement api/app/store methods to be able to save and
delete post acknowledgements by users.
- Adds wesbsocket events for acknowledgement created/deleted
- Returns post acknowledgements in the post's metadata

* add-license-check

* add-pagination

* delete on channel and team

* validate guests

* add configs

* move create priority post check from app to api

* Add desktop notifications

* check status

* use config in job

* add IsUrgent check

* Add last-sent-at

* validate max recipients

* Update lastSentAt

* Validate min. recipient

* send email notification only once

* remove email notifications

* use latest time from config to run job

* Add notifications counter

* publish events to mentioned users only

* pickup license updates in scheduler

* don't allow post owner to stop notifications

* follow normal notifications behaviour

* Validates persistent notifications interval

* move logic of handling valid and expired posts into sql

* Adds persistent notifications in the webapp

---------

Co-authored-by: koox00 <3829551+koox00@users.noreply.github.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2023-05-18 23:44:12 +05:30
Doug Lauder
e4075dae18 MM-43077 Allow inline JSON in config.json for advanced logging config (#23324)
* add AdvancedLoggingJSON to LogSettings and deprecate AdvancedLoggingConfig
* allow embedded JSON in config for advanced logging.
2023-05-15 10:37:48 -04:00
Jesse Hallam
bb02b35048 Expose public/ API as submodule (#23345)
* model -> public/model

* plugin -> public/plugin

* public/model/utils -> public/utils

* platform/shared/mlog -> public/shared/mlog

* platform/shared/i18n -> public/shared/i18n

* platform/shared/markdown -> public/shared/markdown

* platform/services/timezones -> public/shared/timezones

* channels/einterfaces -> einterfaces

* expose public/ submodule

* go mod tidy

* .github: cache-dependency-path, setup-go-work

* modules-tidy for public/ too

* remove old gomodtidy
2023-05-10 13:07:02 -03:00
Jesse Hallam
f28a2bcca7 server/public/ -- pre-requisite changes (#23278)
* invert depdendency: filestore -> model

* markdown: nolint:misspell

* inline jsonutils within model

* push model.GetInfoForBytes -> channels/app

* push channel/utils.CompileGo* -> plugin/utils

* push plugin/scheduler -> channels/jobs/plugins

* push utils.Copy(File|Dir) -> model

* oauthproiders/gitlab -> channels/app/oauthproviders/gitlab

* decouple plugin from einterfaces.MetricsInterface

* fix TestGetInfoForFile

* Revert "Run golangci in server CI (#23240)"

This reverts commit 349e5d4573.

* add model/utils

---------

Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
2023-05-09 13:30:02 -03:00
Harrison Healey
d001994ef9 MM-52282 Remove ExperimentalSettings.PatchPluginsReactDOM (#23172)
* MM-52282 Remove ExperimentalSettings.PatchPluginsReactDOM

* Fix missing file from last commit

* Fix test usage of plugin.NewEnvironment
2023-05-03 15:04:10 -04:00
Harrison Healey
3ba75afa08 MM-52173 Add setting to delay channel autocomplete (#22952)
* Add unit tests for ChannelMentionProvider.handleCompleteWord

* Add a minimum length before the ChannelMentionProvider triggers

* MM-52173 Move delayed autcomplete behind a setting

* Fix e2e test type check

* Update Textbox tests

* Add new setting to server telemetry
2023-05-02 10:11:15 -04:00
Vishal
609ab9c765 [MM-52527] (#23197)
* change recursive to iterative

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-05-01 19:53:18 +05:30
Vishal
344e882f04 Replace string concatenation with StringBuilder (#23021) 2023-04-21 11:55:04 +05:30
Conor Macpherson
c4efd2647c Merge branch 'master' into MM-50966-in-product-expansion 2023-04-19 11:40:50 -04:00
Agniva De Sarker
c34a50a6c7 MM-50427: Make MM survive DB replica outage (#22888)
We monitor the health of DB replicas, and on a fatal error,
take them out of the pool.

On a separate goroutine, we keep pinging the unhealthy replicas,
and on getting a good response back, we add them back to the pool.

https://mattermost.atlassian.net/browse/MM-50427

```release-note
Mattermost is now resilient against DB replica outages and will
dynamically choose a replica if it's alive.

Also added a config parameter ReplicaMonitorIntervalSeconds
whose default value is 5. This controls how frequently unhealthy
replicas will be monitored for liveness check.
```

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-04-19 17:03:18 +05:30
M-ZubairAhmed
7b9d43e308 MM-50122 : Remove and test refetching of channels on browser focus change (#22973) 2023-04-19 11:48:27 +05:30
Conor Macpherson
3a76a324f6 Merge branch 'master' into MM-50966-in-product-expansion 2023-04-18 12:05:00 -04:00
Agniva De Sarker
b200a07881 v8.0 module release (#22975)
https://mattermost.atlassian.net/browse/MM-52079

```release-note
We upgrade the module version to 8.0. The new module path is github.com/mattermost-server/server/v8.
```


Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
2023-04-18 11:05:28 +05:30
Conor Macpherson
f613c655e7 remove self hosted expansion flag in favor of grouping hosted expansion with hosted purhcase and it's flag. 2023-04-17 16:11:43 -04:00
Agniva De Sarker
56bf1b695a MM-49984: Displaying active search backend in system console (#22721)
This will be shown in the system console to let users know
which search backend is active.

Implemented via adding an extra param in the ping response.

https://mattermost.atlassian.net/browse/MM-49984

```release-note
The database section in the system console now has an additional
read-only section which shows the active search backend in use.

This can be helpful to confirm which is the currently active
search engine when there are multiple of them configured.
```
---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-04-14 14:28:50 +05:30
Agniva De Sarker
bcf115e315 Upgrade mockery to fix errors (#22892)
Mockery started to throw errors like "Unexpected package creation during export data loading".

This was consistently reproducible locally and on upgrade, it has gone away.

```release-note
NONE
```
2023-04-10 14:20:57 +05:30
Allan Guwatudde
0140e94d77 [MM-49751] - Turn off Inactive Server Email (#22648)
* [MM-49751] - Turn off Inactive Server Email

* remove unused var

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-03-27 22:38:19 +03:00
Agniva De Sarker
56b18ca7bf MM-45956: Optimize FileInfo stats query (#22603)
* MM-45956: Optimize FileInfo stats query

We Denormalize Post.ChannelId on FileInfo.ChannelId

```release-note
The file info stats query is now optimized by denormalizing the channelID column into the table itself. This will speed up the query to get the file count for a channel on clicking the RHS.

Migration times:
On a MySQL 8.0.31 DB with
1405 rows in FileInfo and 11M posts, it took around 0.3s

On a Postgres 12.14 DB with
1731 rows in FileInfo and 11M posts, it took around 0.27s
```

https://mattermost.atlassian.net/browse/MM-45956
2023-03-23 22:14:04 +05:30
Doug Lauder
c943ed6859 Mono repo -> Master (#22553)
Combines the following repositories into one:

https://github.com/mattermost/mattermost-server
https://github.com/mattermost/mattermost-webapp
https://github.com/mattermost/focalboard
https://github.com/mattermost/mattermost-plugin-playbooks
2023-03-22 17:22:27 -04:00