Commit Graph

82 Commits

Author SHA1 Message Date
George Goldberg
185ed89978 MM-11243: Make Elasticsearch work after enabling without restart. (#9146)
* MM-11243: Make Elasticsearch work after enabling without restart.

* Also cope with config vars changing whilst enabled.
2018-07-26 15:50:38 +01:00
Emil Velikov
f17c15c9d8 Simplify oauth (#8972)
* Remove unused OauthProvider::GetIdentifier

Signed-off-by: Emil Velikov <emil.velikov@collabora.com>

* Reuse gitlab's getAuthData() instead of open-coding it

Signed-off-by: Emil Velikov <emil.velikov@collabora.com>

* Remove OauthProvider::GetAuthDataFromJson interface

The data is already available via GetUserFromJson().AuthData

Signed-off-by: Emil Velikov <emil.velikov@collabora.com>
2018-06-26 17:23:13 -07:00
Jesús Espino
d9390244af MM-8810: Add CSV Compliance export (#8966)
* MM-8810: Add CSV Compliance export

* Only allowing to schedule actiances export throught the cli

* De-duplicating some code

* Fixes on texts

* Fixes on translations
2018-06-22 11:15:19 +02:00
Harrison Healey
226d4b2ac8 MM-6992 Added highlighting to elasticsearch results (#8861)
* MM-6992 Added highlighting to elasticsearch results

* Added a unique type for post search matches

* Fixed Elasticsearch matches not being sent through API
2018-06-19 11:46:29 +02:00
Saturnino Abril
312edbe531 [MM-10718] Move custom branding to TE (#8871)
* move custom branding to TE
* move brand's enterprise code to server and remove BrandInterface
2018-06-02 06:33:59 +08:00
Jesús Espino
e88fe4bb1d MM-8853: Adding MANAGE_EMOJIS and MANAGE_OTHERS_EMOJIS permissions (#8860)
* MM-8853: Adding MANAGE_EMOJIS and MANAGE_OTHERS_EMOJIS permissions

* MM-8853: Removing unnecesary emoji enterprise feature

* Create emojis migration

* Adding MANAGE_EMOJIS and MANAGE_OTHERS_EMOJIS always to system admins

* Simplifing permissions checks

* Revert "Simplifing permissions checks"

This reverts commit e2cafc1905.
2018-05-29 15:58:12 +01:00
Jesse Hallam
847c181ec9 MM-8622: Improved plugin error reporting (#8737)
* allow `Wait()`ing on the supervisor

In the event the plugin supervisor shuts down a plugin for crashing too
many times, the new `Wait()` interface allows the `ActivatePlugin` to
accept a callback function to trigger when `supervisor.Wait()` returns.
If the supervisor shuts down normally, this callback is invoked with
a nil error, otherwise any error reported by the supervisor is passed
along.

* improve plugin activation/deactivation logic

Avoid triggering activation of previously failed-to-start plugins just
becase something in the configuration changed. Now, intelligently
compare the global enable bit as well as the each individual plugin's
enabled bit.

* expose store to manipulate PluginStatuses

* expose API to fetch plugin statuses

* keep track of whether or not plugin sandboxing is supported

* transition plugin statuses

* restore error on plugin activation if already active

* don't initialize test plugins until successfully loaded

* emit websocket events when plugin statuses change

* skip pruning if already initialized

* MM-8622: maintain plugin statuses in memory

Switch away from persisting plugin statuses to the database, and
maintain in memory instead. This will be followed by a cluster interface
to query the in-memory status of plugin statuses from all cluster nodes.

At the same time, rename `cluster_discovery_id` on the `PluginStatus`
model object to `cluster_id`.

* MM-8622: aggregate plugin statuses across cluster

* fetch cluster plugin statuses when emitting websocket notification

* address unit test fixes after rebasing

* relax (poor) racey unit test re: supervisor.Wait()

* make store-mocks
2018-05-23 14:26:35 -04:00
Christopher Speller
d8dd271e43 MM-4998 Adding LoginIdAttribute to allow LDAP users to change their login ID without losing their account (#8756)
* Adding LoginIdAttribute

* Modifying LDAP to use loginIDAttribute.

* Adding IDAttribute migration and AD objectGUID support.

* Removing unused idea.

* Fix typo.
2018-05-10 09:46:09 -07:00
Joram Wilander
09c6770885 Add IncrementClusterEventType function to metrics interface (#8728) 2018-05-04 14:59:28 -07:00
Christopher Speller
686c2fbab7 Structured logging (#8673)
* Implementing structured logging

* Changes to en.json to allow refactor to run.

* Fixing global logger

* Structured logger initalization.

* Add caller.

* Do some log redirection.

* Auto refactor

* Cleaning up l4g reference and removing dependancy.

* Removing junk.

* Copyright headers.

* Fixing tests

* Revert "Changes to en.json to allow refactor to run."

This reverts commit fd8249e99b.

* Fixing some auto refactor strangeness and typo.

* Making keys more human readable.
2018-04-27 12:49:45 -07:00
Joram Wilander
4a1802c039 MM-9664 Add invalidation metrics for store caches (#8340)
* Add invalidation metrics for store caches

* Increment session invalidation metric

* Fix tests
2018-03-05 10:35:26 -05:00
Jesús Espino
342d05bcfc Adding saml migration command (#8263)
* Adding saml migration command

* Texts fixed
2018-02-21 14:34:03 +01:00
Jonathan
375c0632fa PLT-7503: Create Message Export Scheduled Task and CLI Command (#7612)
* Created message export scheduled task

* Added CLI command to immediately kick off an export job

* Added email addresses for users joining and leaving the channel to the export

* Added support for both MySQL and PostgreSQL

* Fixing gofmt error

* Added a new ChannelMemberHistory store and associated tests

* Updating the ChannelMemberHistory channel as users create/join/leave channels

* Added user email to the message export object so it can be included in the actiance export xml

* Don't fail to log a leave event if a corresponding join event wasn't logged

* Adding copyright notices

* Adding message export settings to daily diagnostics report

* Added System Console integration for message export

* Cleaned up TODOs

* Made batch size configurable

* Added export from timestamp to CLI command

* Made ChannelMemberHistory table updates best effort

* Added a context-based timeout option to the message export CLI

* Minor PR updates/improvements

* Removed unnecessary fields from MessageExport object to reduce query overhead

* Removed JSON functions from the message export query in an effort to optimize performance

* Changed the way that channel member history queries and purges work to better account for edge cases

* Fixing a test I missed with the last refactor

* Added file copy functionality to file backend, improved config validation, added default config values

* Fixed file copy tests

* More concise use of the testing libraries

* Fixed context leak error

* Changed default export path to correctly place an 'export' directory under the 'data' directory

* Can't delete records from a read replica

* Fixed copy file tests

* Start job workers when license is applied, if configured to do so

* Suggestions from the PR

* Moar unit tests

* Fixed test imports
2017-11-30 09:07:04 -05:00
Chris
eb1a00ef5f Reorganize file util functionality (#7848)
* reorganize file util functionality

* fix api test compilation

* fix rebase issue
2017-11-16 16:04:27 -05:00
Corey Hulen
b3b46a01aa PLT-5813 support SAML sync via LDAP (#7668)
* PLT-5813 support SAML sync via LDAP

* Cleaning up based on review
2017-10-31 08:37:34 -07:00
George Goldberg
76bd1bb212 PLT-7705: API to get data retention policy. (#7539)
* PLT-7705: API to get data retention policy.

* Fix review comments.
2017-10-02 12:43:21 +01:00
Chris
4e79d2d4d0 remove jobs.Srv and other jobs-related globals (#7535) 2017-09-29 10:29:29 +01:00
George Goldberg
e8ca3d64f4 PLT-7470: Add metrics for searches. (#7507) 2017-09-25 13:44:44 -07:00
Corey Hulen
81c18a01bd PLT-7542 Converting LDAP sync to the job server (#7452)
* PLT-7542 Converting LDAP sync to the job server

* Fixing minor issues

* Fixing build failure

* Translate error message

* Translate error message

* Translate error message

* Translate error message

* Fixing merge

* Fixing bad merge
2017-09-25 13:30:33 -07:00
George Goldberg
265392fa48 PLT-7667: Clean up elasticsearch indexes when running data retention jobs. (#7481) 2017-09-21 17:54:31 -04:00
Chris
266ff86702 remove more global refs (#7480) 2017-09-21 10:13:34 +01:00
Christopher Speller
29fca51821 Renaming repo 2017-09-06 23:11:59 -07:00
Corey Hulen
7843dc3cfa Fixing leader election (#7267) 2017-09-05 07:58:30 -07:00
Joram Wilander
e2042c4b65 Add built-in plugin for getting LDAP attributes (#7317) 2017-09-01 14:28:15 -04:00
George Goldberg
22459ee17a PLT-7302: Aggregate Elasticsearch indexes over a certain age. (#7224)
* PLT-7302: Aggregate Elasticsearch indexes over a certain age.

This is done by a scheduled daily job, in order to keep the shard count
to a sensible level in Elasticsearch.

* Use map[string]string instead of StringMap
2017-08-17 10:05:17 -04:00
Corey Hulen
29a7e182a6 PLT-6079 Adding leader election (#7105) 2017-08-16 10:29:45 -07:00
George Goldberg
f30c0b5d38 PLT-6596: Elasticsearch interface changes. (#6993) 2017-07-21 08:38:44 +08:00
George Goldberg
fe368a7456 PLT-6595: API to purge Elasticsearch indexes. (#6971) 2017-07-19 09:43:05 +01:00
George Goldberg
83d53ea98c PLT-6475: Elasticsearch Indexing Worker. (#6879) 2017-07-11 09:09:15 +01:00
George Goldberg
39b071bc0e Add optional config parameter to test elasticsearch endpoint. (#6877) 2017-07-07 12:12:14 -04:00
George Goldberg
db2f6cf076 PLT-6976: Elasticsearch capitalisation and tests. (#6839)
* Fixes Elasticsearch to have consistent capitalisation everywhere
  across the code and UI (except the config file unfortunately).
* Adds basic unit tests for Elastichsearch.
* Adds a Elasticsearch docker image to the Makefile to enable testing
  the Elasticsearch feature.
2017-07-07 15:36:17 +01:00
George Goldberg
0495a51949 PLT-6916: Redesign the jobs package and Jobserver. (#6733)
This commit redesigns the jobserver to be based around an architecture
of "workers", which carry out jobs of a particular type, and "jobs"
which are a unit of work carried by a particular worker. It also
introduces "schedulers" which are responsible for scheduling jobs of a
particular type automatically (jobs can also be scheduled manually when
apropriate).

Workers may be run many times, either in instances of the platform
binary, or the standalone jobserver binary. In any mattermost cluster,
only one instance of platform OR jobserver must run the schedulers. At
the moment this is controlled by a config variable, but in future will
be controlled through the cluster leader election process.
2017-07-07 15:21:02 +01:00
George Goldberg
f2898927f1 PLT-6474: Server: Add elasticsearch/test endpoint to API. (#6792) 2017-06-29 17:40:14 -04:00
Corey Hulen
36f216cb7c PLT-6080 moving clustering to memberlist (#6499)
* PLT-6080 adding cluster discovery service

* Adding memberlist lib

* Adding memberlist lib

* WIP

* WIP

* WIP

* WIP

* Rolling back config changes

* Fixing make file

* Fixing config for cluster

* WIP

* Fixing system console for clustering

* Fixing default config

* Fixing config

* Fixing system console for clustering

* Tweaking hub setting

* Bumping up time

* merging vendor dir

* Updating vendor dir

* Fixing unit test

* Fixing bad merge

* Remove some testing code

* Moving comment

* PLT-6868 adding db ping retry

* Removing unused loc strings

* Adding defer to cancel
2017-06-19 08:44:04 -07:00
Christopher Speller
04088009f8 Adding force flag to migrate auth command (#6593) 2017-06-06 14:42:53 -04:00
Harrison Healey
577ed27f1b PLT-6408 Framework for job server (#6404)
* Added initial job server

* Added job server to be ran as part of platform

* Added test job to the enterprise repo

* Fixed job server not loading license

* Renamed job package to jobs

* Fixed TE not being buildable

* Added JobStatus table to database

* Changed fields used by JobStatus

* Added APIs to query job status

* Added config change listener to server

* Added option to run job server from Makefile

* Added ability to enable/disable jobs from config

* Commented out placeholder for search indexing job

* Fixed govet

* Removed debug messages and fixed job api init message
2017-05-18 15:05:57 -04:00
George Goldberg
0db5e3922f PLT-6472: Basic Elastic Search implementation. (#6382)
* PLT-6472: Basic Elastic Search implementation.

This currently supports indexing of posts at create/update/delete time.
It does not support batch indexing or reindexing, and does not support
any entities other than posts yet. The purpose is to more-or-less
replicate the existing full-text search feature but with some of the
immediate benefits of using elastic search.

* Alter settings for AWS compatability.

* Remove unneeded i18n strings.
2017-05-18 11:26:52 -04:00
Corey Hulen
e62afeace0 Adding slow pump detection to web_conn and better metrics (#6114)
* Adding slow pump detection to web_conn and better metrics

* Fixing bad merge

* Fixing typo
2017-04-21 12:38:26 -04:00
Harrison Healey
fb6f2a123c PLT-5860 Updated copyright date (#6058)
* PLT-5860 Updated copyright date in about modal

* PLT-5860 Updated copyright notice in JSX files

* PLT-5860 Updated copyright notice in go files

* Fixed misc copyright dates

* Fixed component snapshots
2017-04-12 08:27:57 -04:00
Harrison Healey
c7bdce8a66 PLT-5900 Removed automatic configuration of SiteURL (#5952)
* Revert "PLT-5765 Passed SiteURL to SendNotifications (#5705)"

This reverts commit a8e68bd890.

* Removed automatic configuration of SiteURL

* Reverted unintentional config changes

* Updated help text

* Added link to docs in Site URL warning

* Fixed merge conflict
2017-04-04 11:54:52 -04:00
George Goldberg
7f266c1951 Merge branch 'release-3.7' into master. 2017-03-17 16:29:26 +00:00
Joram Wilander
0bc3e46082 Implement GET /logs endpoint for APIv4 (#5778) 2017-03-16 11:59:44 -07:00
Harrison Healey
a8e68bd890 PLT-5765 Passed SiteURL to SendNotifications (#5705) 2017-03-10 10:18:18 +00:00
Joram Wilander
83291e4c8f Merge 3.7 changes into master (#5679)
* Save schema version for 3.7 (#5675)

* Adding index and cache to reactinos store (#5654)

* Fix badge count for push notifications (#5672)
2017-03-07 16:05:01 -05:00
Christopher Speller
33b8b72a02 Adding index and cache to reactinos store (#5654) 2017-03-07 09:37:00 -05:00
Corey Hulen
847c716ea2 Example of how to call metrics (#5621) 2017-03-06 10:15:37 -05:00
Christopher Speller
83c113595a Revert "Adding caching to get channel member (#5518)"
This reverts commit ba028ed74b.
2017-02-27 17:43:23 -05:00
Christopher Speller
ba028ed74b Adding caching to get channel member (#5518) 2017-02-24 09:15:36 -05:00
enahum
748a416961 PLT-3193 Add channel notification preferences for push and email noti… (#5500)
* PLT-3193 Add channel notification preferences for push and email notifications

* unit tests, model validation and localization

* Feedback review

* Adding back allowFromCache check

* Setting push and email to use default settings

* Move props as constants

* address feedback
2017-02-23 11:08:48 -03:00
Christopher Speller
2a9c1afd44 Adding webhook metrics (#5348) 2017-02-10 07:05:12 -08:00