Commit Graph

39 Commits

Author SHA1 Message Date
Christopher Speller
686c2fbab7 Structured logging (#8673)
* Implementing structured logging

* Changes to en.json to allow refactor to run.

* Fixing global logger

* Structured logger initalization.

* Add caller.

* Do some log redirection.

* Auto refactor

* Cleaning up l4g reference and removing dependancy.

* Removing junk.

* Copyright headers.

* Fixing tests

* Revert "Changes to en.json to allow refactor to run."

This reverts commit fd8249e99b.

* Fixing some auto refactor strangeness and typo.

* Making keys more human readable.
2018-04-27 12:49:45 -07:00
Jesse Hallam
8491ba5740 Relax 4k post message limit (#8478)
* MM-9661: rename POST_MESSAGE_MAX_RUNES to \0_v1

* MM-9661: s/4000/POST_MESSAGE_MAX_RUNES_V1/ in tests

* MM-9661: introduce POST_MESSAGE_MAX_RUNES_V2

* MM-9661: migrate Postgres Posts.Message column to TEXT from VARCHAR(4000)

This is safe to do in a production instance since the underyling type is
not changing. We explicitly don't do this automatically for MySQL, but
also don't need to since the ORM would have already created a TEXT column
for MySQL in that case.

* MM-9661: emit MaxPostSize in client config

This value remains unconfigurable at this time, but exposes the current
limit to the client. The limit remains at 4k in this commit.

* MM-9661: introduce and use SqlPostStore.GetMaxPostSize

Enforce a byte limitation in the database, and use 1/4 of that value as
the rune count limitation (assuming a worst case UTF-8 representation).

* move maxPostSizeCached, lastPostsCache and lastPostTimeCache out of the global context and onto the SqlPostStore

* address feedback from code review:

* ensure sqlstore unit tests are actually being run
* move global caches into SqlPostStore
* leverage sync.Once to address a race condition

* modify upgrade semantics to match new db semantics

gorp's behaviour on creating columns with a maximum length on Postgres
differs from MySQL:
* Postgres
  * gorp uses TEXT for string columns without a maximum length
  * gorp uses VARCHAR(N) for string columns with a maximum length of N
* MySQL
  * gorp uses TEXT for string columns with a maximum length >= 256
  * gorp uses VARCHAR(N) for string columns with a maximum length of N
  * gorp defaults to a maximum length of 255, implying VARCHAR(255)

So the Message column has been TEXT on MySQL but VARCHAR(4000) on
Postgres. With the new, longer limits of 65535, and without changes to
gorp, the expected behaviour is TEXT on MySQL and VARCHAR(65535) on
Postgres. This commit makes the upgrade semantics match the new database
semantics.

Ideally, we'd revisit the gorp behaviour at a later time.

* allow TestMaxPostSize test cases to actually run in parallel

* default maxPostSizeCached to POST_MESSAGE_MAX_RUNES_V1 in case the once initializer panics

* fix casting error

* MM-9661: skip the schema migration for Postgres

It turns out resizing VARCHAR requires a rewrite in some versions of
Postgres, but migrating VARCHAR to TEXT does not. Given the increasing
complexity, let's defer the migration to the enduser instead.
2018-03-26 17:55:35 -04:00
Jesse Hallam
87762ae62e Improved bulkload error handling (#8491)
* log the config file path used by the server on startup

* return an err if the bulk import command fails

* log the underlying errors that occur when importing

The code assumed all errors meant a missing resource, but it's possible
something else is at fault. Including the error helps pinpoint that more
readily.
2018-03-23 03:33:02 +08:00
Jesús Espino
c158e9a5a0 XYZ-114 Adding mandatory membership to town-square in bulk-import (#8274) 2018-02-15 15:31:48 +00:00
Jesús Espino
6990d052d5 [XYZ-6] Add sampledata platform command (#8027)
* Add fake dependency

* [XYZ-6] Add sampledata platform command

* Creating EMOJI_NAME_MAX_LENGTH as a constant and using it where needed
2018-01-11 10:57:47 -05:00
Chris
816a30397d Role refactor (#7867)
* role refactor

* add missing file

* fix web test
2017-11-21 11:08:32 -08:00
Chris
eb1a00ef5f Reorganize file util functionality (#7848)
* reorganize file util functionality

* fix api test compilation

* fix rebase issue
2017-11-16 16:04:27 -05:00
Joram Wilander
06ec648cf3 PLT-7978 Add websocket event for user role update (#7745)
* Add websocket event for user role update

* Fix tests

* More test fixes
2017-10-31 16:00:21 +00:00
Chris
c5e8cb25ca simplify things (#7735) 2017-10-30 11:57:24 -05:00
Chris
d39a722a81 refactor max channels per team enforcement (#7690) 2017-10-20 20:33:20 -04:00
Chris
8e19ba029f Reduce utils.Cfg references (#7650)
* app.UpdateConfig method

* test fix

* another test fix

* the config override option as-was is just error prone, remove it for now

* derp
2017-10-18 15:36:43 -07:00
George Goldberg
fe41022353 PLT-7883: Fix database race in bulk importer in GetByUsername. (#7611) 2017-10-12 10:00:05 -07:00
George Goldberg
e16bdf8d1d PLT-7789: Fix raciness on Aurora with importing team members. (#7573)
* PLT-7789: Fix raciness on Aurora with importing team members.

* Fix and document return values.
2017-10-04 10:17:58 -07:00
Christopher Speller
29fca51821 Renaming repo 2017-09-06 23:11:59 -07:00
Chris
1adfd0e9be app type transition (#7167) 2017-09-06 17:12:54 -05:00
George Goldberg
e85b5fb988 App: NewLocAppError -> NewAppError (#7327)
* App: NewLocAppError -> NewAppError

* Remove statuscode that got missed.
2017-09-01 11:42:02 -04:00
George Goldberg
50fc6e1e9e PLT-???? Prepare file upload infrastructure for Data Retention. (#7266)
* Prepare file upload infrastructure for Data Retention.

This commit prepares the file upload infrastructure for the data
retention feature that is under construction. Changes are:

* Move file management code to utils to allow access to it from jobs.

* From now on, store all file uploads in a top level folder which is the
  date of the day on which they were uploaded.

This commit is based on Harrison Healey's branch, but updated to work
with the latest master.

* Use NewAppError
2017-08-25 10:38:13 -04:00
George Goldberg
4132a02319 PLT-7183: User/Channel NotifyProps Bulk Import. (#7019) 2017-07-26 08:51:25 +01:00
George Goldberg
02abaabf1f PLT-7123: Add tutorial, flags and favorites to bulk importer. (#7000) 2017-07-21 09:23:27 -04:00
George Goldberg
65289f26e7 Fix some duplicate message IDs in Bulk Importer. (#6850) 2017-07-05 07:49:30 -07:00
George Goldberg
37642a4f1e PLT-6937: Bulk Importing of Direct/Group channels and posts. (#6761)
* PLT-6937: Bulk Importing of Direct/Group channels and posts.

* Show group/direct channels in sidebar.
2017-06-28 07:26:38 -07:00
George Goldberg
a678981861 PLT-6488: Reduce database queries in user bulk import. (#6371) 2017-05-10 09:08:06 -04:00
George Goldberg
6f6b7e4e97 Add Password field to Bulk Importer. (#6268) 2017-04-28 15:01:56 -07:00
George Goldberg
302ec17bee Parallelise Bulk Import. (#6267)
* Parallelise Bulk Import.

* Set worker count through command line flag.
2017-04-28 17:54:04 +01:00
Harrison Healey
fb6f2a123c PLT-5860 Updated copyright date (#6058)
* PLT-5860 Updated copyright date in about modal

* PLT-5860 Updated copyright notice in JSX files

* PLT-5860 Updated copyright notice in go files

* Fixed misc copyright dates

* Fixed component snapshots
2017-04-12 08:27:57 -04:00
George Goldberg
32460bf63b PLT-6198: Use added to channel system message on default channels. (#5969)
Use a different sytem message when a user was added to a default channel
by someone else than when they joined themselves.
2017-04-04 18:19:42 +01:00
Harrison Healey
c7bdce8a66 PLT-5900 Removed automatic configuration of SiteURL (#5952)
* Revert "PLT-5765 Passed SiteURL to SendNotifications (#5705)"

This reverts commit a8e68bd890.

* Removed automatic configuration of SiteURL

* Reverted unintentional config changes

* Updated help text

* Added link to docs in Site URL warning

* Fixed merge conflict
2017-04-04 11:54:52 -04:00
Harrison Healey
06a7c3ba8e PLT-5800 Cleaned up duplicated code for adding slack attachments to posts (#5711) 2017-03-10 10:22:14 +00:00
Harrison Healey
a8e68bd890 PLT-5765 Passed SiteURL to SendNotifications (#5705) 2017-03-10 10:18:18 +00:00
George Goldberg
f7682ad11a PLT-5367: Basic post bulk importing. (#5562) 2017-02-28 20:55:21 -03:00
enahum
748a416961 PLT-3193 Add channel notification preferences for push and email noti… (#5500)
* PLT-3193 Add channel notification preferences for push and email notifications

* unit tests, model validation and localization

* Feedback review

* Adding back allowFromCache check

* Setting push and email to use default settings

* Move props as constants

* address feedback
2017-02-23 11:08:48 -03:00
George Goldberg
d60d05f2d8 PLT-5521: Mandatory version in line 1 of bulk import. (#5400) 2017-02-16 09:49:01 -05:00
George Goldberg
6251e836b0 PLT-5398 Bulk loading of user display preferences. (#5382) 2017-02-14 08:38:48 -05:00
George Goldberg
8277945363 PLT-5427: Import ChannelMember Notify Props. (#5329) 2017-02-08 08:19:38 -08:00
George Goldberg
28e445c6e0 PLT-5407: Bulk importing of User memberships. (#5273) 2017-02-02 10:03:41 -05:00
George Goldberg
67739cb516 PLT-5365 Import of basic user properties. (#5231) 2017-01-31 10:04:17 -03:00
Joram Wilander
097289f8e4 Merge 3.6.2 into master (#5211)
* Add webhook cache

* Add channel by name cache

* Fxing profiles in channels cache

* Fix merge
2017-01-27 14:07:34 -05:00
George Goldberg
e07e9937e0 PLT-5366, PLT-5364, PLT-5363: Bulk Import Part 1. (#5204)
This commit provides the first part of the bulk import system.

The CLI command is provided, complete with validation & apply modes. All
the basic properties of Teams and Channels can be imported. Users &
Posts will follow separately in a future commit.
2017-01-27 10:14:54 -05:00
George Goldberg
2de6c5394e Move Slack Import to App Layer. (#5135) 2017-01-20 09:47:14 -05:00