Commit Graph

432 Commits

Author SHA1 Message Date
Guo Xiang Tan
eb9565d8fc DEV: Rescue from readonly error in lograge. 2019-01-21 14:20:19 +08:00
Guo Xiang Tan
ee7ab3e2ec Pause MiniScheduler when Sidekiq is paused. 2019-01-18 17:50:24 +08:00
Guo Xiang Tan
234e0f35c5 Fix the build.
6edf285c89
2019-01-17 17:28:09 +08:00
Guo Xiang Tan
6edf285c89 FIX: Avoid rescuing error in login hint initializer.
On the first migration, trying to access the users table will throw an
error in PostgreSQL's log which has been confusing since users will
report it to us when rebuild fails.
2019-01-17 17:22:15 +08:00
Sam
a19170a4c2 DEV: avoid require_dependency for some libs
This avoids require dependency on method_profiler and anon cache.

It means that if there is any change to these files the reloader will not pick it up.

Previously the reloader was picking up the anon cache twice causing it to double load on boot.

This caused warnings.

Long term my plan is to give up on require dependency and instead use:

https://github.com/Shopify/autoload_reloader
2018-12-31 10:53:30 +11:00
Guo Xiang Tan
e564fe1866 FIX: Sidekiq fails to start if any of the multisite has problems. 2018-12-12 11:30:14 +08:00
Sam
a34bc92e1a DEV: update mini profiler
This provides us with instrumentation missing after rails upgrade

Latest version of rails uses exec_params internally which is no longer
routed to intercepted methods in mini profiler 1.0.0
2018-12-10 14:29:20 +11:00
Sam
1a71f98d28 DEV: only publish logs error count to admins 2018-12-05 17:03:37 +11:00
Sam
12f5889c85 DEV: stop logging warning when overriding open scope
ActiveRecord defines automatic scopes for enums, the Poll model defines
an enum for `{open: 1}` this mean Rails wants the scope `Poll.all.open`
to work which in turn means it has to override `open` which is defined
privately.

Rails feature req exists for: https://github.com/rails/rails/issues/34599
which will allow us to define enums without scopes which would resolve this
a lot more cleaner.
2018-12-03 13:52:09 +11:00
Penar Musaraj
03deda2147
Upgrade to FontAwesome 5 (take two) (#6673)
* Add missing icons to set

* Revert FA5 revert

 This reverts commit 42572ff

* use new SVG syntax in locales

* Noscript page changes (remove login button, center "powered by" footer text)

* Cast wider net for SVG icons in settings

- include any _icon setting for SVG registry (offers better support for plugin settings)

- let themes store multiple pipe-delimited icons in a setting

- also replaces broken onebox image icon with SVG reference in cooked post processor

* interpolate icons in locales

* Fix composer whisper icon alignment

* Add support for stacked icons

* SECURITY: enforce hostname to match discourse hostname

This ensures that the hostname rails uses for various helpers always matches
the Discourse hostname

* load SVG sprite with pre-initializers

* FIX: enable caching on SVG sprites

* PERF: use JSONP for SVG sprites so they are served from CDN

This avoids needing to deal with CORS for loading of the SVG

Note, added the svg- prefix to the filename so we can quickly tell in
dev tools what the file is

* Add missing SVG sprite JSONP script to CSP

* Upgrade to FA 5.5.0

* Add support for all FA4.7 icons

- adds complete frontend and backend for renamed FA4.7 icons

- improves performance of SvgSprite.bundle and SvgSprite.all_icons

* Fix group avatar flair preview

- adds an endpoint at /svg-sprites/search/:keyword

- adds frontend ajax call that pulls icon in avatar flair preview even when it is not in subset

* Remove FA 4.7 font files
2018-11-26 16:49:57 -05:00
Sam
42572ff138 Revert font awesome 5 changes
We are still pushing ahead on this 100% just need a bit longer to prepare
all plugins
2018-11-08 16:12:18 +11:00
Penar Musaraj
005e1ecb9b
FEATURE: Update Font Awesome to v5.4.1 and SVGs (#6557)
* First take on subsetting svg icons

* FontAwesome 5 svg subset WIP

* Include icons from plugins/badges into svg sprite subset

* add svg icon support to themes

* Add spec for SvgSprite

* Misc. SVG icon fixes

* Use FA5 svgs in local-dates plugin

* CSS adjustments, fix SVG icons in group flair

* Use SVG icons in poll plugin

* Add SVG icons to /wizard
2018-11-07 13:05:43 -05:00
Kyle Zhao
dca830cb73 Revert "FEATURE: [Experimental] Content Security Policy (#6504)"
This reverts commit fb8231077a.
2018-10-19 11:53:29 -04:00
Kyle Zhao
fb8231077a
FEATURE: [Experimental] Content Security Policy (#6504) 2018-10-19 10:39:22 -04:00
Davide Porrovecchio
005e1f5373 Add Cache-Control header to CORS (#6490) 2018-10-16 10:46:55 +11:00
Guo Xiang Tan
1c9b5e75e7 DEV: Support post deployment migrations for plugins. 2018-10-09 13:11:45 +08:00
Guo Xiang Tan
40fa96777d
FEATURE: Post deployment migrations. (#6406)
This moves us away from the delayed drops pattern which
was problematic on two counts. First, it uses a hardcoded "delay for"
duration which may be too short for certain deployment strategies.
Second, delayed drop doesn't ensure that it only runs after
the latest application code has been deployed. If the migration runs
and the application code fails to deploy, running the migration after
"delay for" has been met will cause the application to blow up.

The new strategy allows post deployment migrations to be skipped if the
env `SKIP_POST_DEPLOYMENT_MIGRATIONS` is provided.

```
SKIP_POST_DEPLOYMENT_MIGRATIONS=1 rake db:migrate
-> deploy app servers
SKIP_POST_DEPLOYMENT_MIGRATIONS=0 rake db:migrate
```

To aid with the generation of a post deployment migration, a generator
has been added. Simply run `rails generate post_migration`.
2018-10-08 15:47:38 +08:00
Vinoth Kannan
8430ea927e FIX: Generate webhook payloads before destroy events (#6325) 2018-10-05 16:53:59 +08:00
CheshireVillageSoftware
f3214889dc FEATURE: Updated CORS config to explicitly specifyhttp methods
See: https://stackoverflow.com/questions/20478312/default-value-for-access-control-allow-methods 

In particular we now explicitly allow DELETE and PUT which is inconsistently allowed depending on browser
2018-09-17 11:01:08 +10:00
Guo Xiang Tan
8dc1463ab3 Enable Lint/ShadowingOuterLocalVariable for Rubocop. 2018-09-04 10:16:42 +08:00
Guo Xiang Tan
0a14e0a256 Ensure params[:files] responds to map in Lograge. 2018-09-04 09:22:54 +08:00
Davide Porrovecchio
1826626272 FEATURE: Add Content-Type header to CORS
- add Content-Type to Access-Control-Allow-Headers
- update test accordingly
2018-08-28 11:19:38 +10:00
Guo Xiang Tan
6d588e43cc FIX: Avoid lograge error when controller doesn't respond to current_user.
https://meta.discourse.org/t/broken-third-party-plugins-can-break-logging/94256
2018-08-20 10:59:56 +08:00
Guo Xiang Tan
402e570c77 Logster::Logger#chained is now readable in new version of logster. 2018-08-14 09:57:09 +08:00
Sam
b504398a91 DEV: keep bin/unicorn clean from log noise 2018-08-14 08:49:57 +10:00
Guo Xiang Tan
dadbf2edb4 DEV: Log to STDOUT in development like how rails s used to. 2018-08-13 17:05:42 +08:00
Sam
c1e7d6e39a move patch cause it was causing a side effect 2018-08-07 17:25:28 +10:00
Sam
f3549291a3 DEV: use unicorn in development
This commit also cleans up a bunch of pointless noise each time we boot app

- narrative was loading i18n cause redefinition of consts
- discourse.rb was loaded twice as was auth
- bin/unicorn now does all the smart things and boots unicron in dev
- bin/rails s will boot unicorn with no params
- remove bin/puma which only causes confusion
2018-08-07 17:13:47 +10:00
Neil Lalonde
0f6dcb8c40 remove unnecessary require 2018-08-01 10:37:10 -04:00
Neil Lalonde
4ad7ce70ce REFACTOR: extract scheduler to the mini_scheduler gem 2018-07-31 17:12:55 -04:00
Vinoth Kannan
af5b88f8e2 Rename approval web hook event type to queued post 2018-07-26 10:29:38 +05:30
Vinoth Kannan
5059dad8f0 FEATURE: Webhook for post approval events 2018-07-25 23:43:19 +05:30
Davide Porrovecchio
dd9d815178 FIX: Add User Api Key headers to CORS
- add User-Api-Key and User-Api-Client-Id to Access-Control-Allow-Headers
- update test
2018-07-24 10:28:23 +10:00
Vinoth Kannan
84ab825e41
FEATURE: Webhook for user destroyed event (#6124) 2018-07-23 13:19:49 +05:30
Guo Xiang Tan
125126c893 Fix typo. 2018-07-20 07:57:00 +08:00
Guo Xiang Tan
9b18ee4bba Improve handling of file params in lograge. 2018-07-19 08:23:59 +08:00
Sam
665ade50c6 SECURITY: extra CORS headers should be set on correct host 2018-07-11 09:29:02 +10:00
Sam
b97b426d34 Revert "FIX: wrong site origin in message bus header"
This reverts commit 93b49a0dd5.
2018-07-11 09:26:43 +10:00
Neil Lalonde
93b49a0dd5 FIX: wrong site origin in message bus header 2018-07-10 15:01:03 -04:00
Dax74
ccc2d94ae8
Update link
See https://meta.discourse.org/t/wrong-link-on-manual-admin-creation/90849
2018-06-27 11:38:01 +02:00
Sam
5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
Guo Xiang Tan
8a2c5fbebb Remove unused lines. 2018-06-11 08:44:41 +08:00
Guo Xiang Tan
2aad91d4a2 PERF: Don't bloat the Sidekiq queue with Jobs::SendPushNotification. 2018-05-24 10:04:09 +08:00
Sam
6974b7d6a8 FIX: run deferred jobs inline in sidekiq 2018-05-23 12:05:37 +10:00
Guo Xiang Tan
467d91347a Missing specs for Group, Tag, Category and Flag web hooks. 2018-05-21 17:29:58 +08:00
Guo Xiang Tan
bf84037f79 FIX: Payload for webhooks should be current as of the time the event was triggered.
https://meta.discourse.org/t/group-category-tag-user-deleted-webhooks-not-firing/87752
2018-05-21 17:29:58 +08:00
Jeff Wong
2002a7c4ef FIX: regenerate vapid keys when the public key bytes is blank 2018-05-07 17:01:56 -07:00
Jeff Wong
91b31860a1
Feature: Push notifications for Android (#5792)
* Feature: Push notifications for Android

Notification config for desktop and mobile are merged.

Desktop notifications stay as they are for desktop views.

If mobile mode, push notifications are enabled.

Added push notification subscriptions in their own table, rather than through
custom fields.

Notification banner prompts appear for both mobile and desktop when enabled.
2018-05-04 15:31:48 -07:00
Sam
26ce930ac6 FIX: remove auth cookie if we see InvalidAccess 2018-04-20 11:21:51 +10:00
Guo Xiang Tan
1b9a38c5e2 FIX: Incorrect formatter used when logstash formatter is enabled. 2018-04-17 12:07:52 +08:00
Guo Xiang Tan
a902d5b27f Replace ActiveSupport::Logger with DiscourseLogstashLogger when enabled. 2018-04-13 12:08:27 +08:00
Arpit Jalan
a16b616861 FEATURE: webhook for flag events 2018-04-13 07:47:58 +05:30
Vinoth Kannan
434cbc649f FEATURE: Webhook for tag events 2018-04-04 17:49:20 +05:30
Guo Xiang Tan
e04d09d148 Add Process pid to lograge output as well. 2018-04-04 18:31:29 +08:00
Guo Xiang Tan
746889bbc1 Include hostname in lograge payload. 2018-04-04 18:24:30 +08:00
Vinoth Kannan
e7407d0adc FEATURE: Webhook for group and category events 2018-03-27 11:53:35 +05:30
Kyle Zhao
f7bd05e534 FEATURE: set 'Retry-After' header for 429 responses (#5659) 2018-03-13 23:12:41 +08:00
Sam
0134e41286 FEATURE: detect when client thinks user is logged on but is not
This cleans up an error condition where UI thinks a user is logged on
but the user is not. If this happens user will be prompted to refresh.
2018-03-06 16:49:31 +11:00
Guo Xiang Tan
66d620f7b1 FEATURE: Trigger topic webhook when topic status is updated. 2018-02-27 11:07:37 +08:00
scossar
9d0807224b Don't enqueue topic webhook unless a post has a topic 2018-02-22 14:34:59 +08:00
Sam
ca1a3f37e3 FEATURE: add instrumentation for all external net calls 2018-02-21 15:20:29 +11:00
Gerhard Schlager
ce060e2b86 FIX: Server didn't use default_locale as fallback locale 2018-01-25 14:57:41 +01:00
Guo Xiang Tan
54dc191a91 Update rails_multisite to 2.0.1. 2018-01-19 10:19:16 +08:00
Sam
442a17bfb2 PERF: bypass omniauth unless in an auth path 2018-01-15 12:44:54 +11:00
Sam
ea63abf0f7 bypass mini profiler for locales
bypass cdn for now
2018-01-09 11:30:59 +11:00
Arpit Jalan
b205910faa FIX: allow upto three url redirects in onebox (#5457) 2017-12-27 10:30:06 +11:00
Guo Xiang Tan
6c4ee9d5b5 FEATURE: Trigger user webhook when a user logs in/out.
https://meta.discourse.org/t/how-do-you-learn-how-to-build-these-plugins/57946/10?u=tgxworld
2017-12-20 17:47:30 +08:00
Guo Xiang Tan
7d8cd84fa6 Improve lograge custom payload error handling. 2017-12-12 17:27:40 +08:00
Sam
63bdc4056d More log suppressions for rate limit exceeded 2017-12-11 17:52:57 +11:00
Guo Xiang Tan
2ff3444568 Rescue from errors when trying to append custom payloads to lograge output. 2017-12-08 08:31:58 +08:00
Sam
0693ed10d1 remove logging code for now 2017-12-08 08:36:35 +11:00
Vinoth Kannan
58aeaef0c4 FIX: Rescue if default_current_user_provider raised an error 2017-12-08 01:42:55 +05:30
Vinoth Kannan
19247f9087 FIX: current_user should be called only if controller have that method 2017-12-07 23:12:34 +05:30
Sam
90a55d6f7c FIX: handle CORS in hijacked requests 2017-12-07 10:31:04 +11:00
Guo Xiang Tan
6cb2b68b22 current_username -> username. 2017-12-05 12:29:02 +08:00
Guo Xiang Tan
dc858ec120 Include request.remote_ip and current_username in lograge logs. 2017-12-05 11:57:41 +08:00
Guo Xiang Tan
900dec5bc1 Make logster ignore Rack::QueryParser::InvalidParameterError errors. 2017-11-30 15:29:19 +08:00
Guo Xiang Tan
46f8a6c97d Fix broken lograge logs. 2017-11-28 14:00:13 +08:00
Guo Xiang Tan
b907f7d307 Add more metrics to lograge's output if MethodProfiler is enabled. 2017-11-25 08:16:40 +08:00
Sam
2b8d4508e5 PERF: stop running background work between requests
Use a dedicated thread to run Scheduler::Defer

This avoids blocking of a worker during operations that require waiting.

In particular uploads risked blocking a unicorn.

This also add a queue "length" that discourse prometheus consumes.
2017-11-23 15:48:47 +11:00
Guo Xiang Tan
89d9ffa884 EXPERIMENTAL: Allow logs to be shipped via different methods.
See https://github.com/dwbutler/logstash-logger#uri-configuration for
a list of available methods.
2017-11-15 09:11:33 +08:00
Guo Xiang Tan
0d8e9a88c9 FIX: Disable rails_multisite formatter if lograge is enabled. 2017-11-15 09:03:50 +08:00
Sam
86e6732f78 FEATURE: update rails multisite
always allow /srv/status through even if host does not match
2017-11-09 15:40:56 +11:00
Guo Xiang Tan
b840971b77 Convert params hash to a query string instead of a hash when logging. 2017-11-08 15:19:45 +08:00
Guo Xiang Tan
09cee4c315 Don't add time to logs when using logstash formatter. 2017-11-02 14:40:18 +08:00
Guo Xiang Tan
d888d3c54c EXPERIMENTAL: Allow Logstash formatter to be enable for lograge logs. 2017-11-01 08:38:06 +08:00
Guo Xiang Tan
d15068da70 Remove experimental logstash feature.
* We'll most likely be extracting it into a plugin.
2017-10-31 10:15:22 +08:00
Guo Xiang Tan
b8e580b0e0 Set logstash-logger file output to sync. 2017-10-30 20:57:02 +08:00
Guo Xiang Tan
9112d2da3b Add timestamp to lograge output. 2017-10-28 00:46:49 +08:00
Guo Xiang Tan
fba0c9e51e Continue to log to Rails default log file when LogStashLogger is
enabled.
2017-10-28 00:30:11 +08:00
Guo Xiang Tan
8b12a906c6 Fix incorrect URI for LogstashLogger. 2017-10-27 23:52:36 +08:00
Guo Xiang Tan
afdd31473d Add type when shipping logs to logstash. 2017-10-27 23:31:17 +08:00
Guo Xiang Tan
d487be759c EXPERIMENTAL: Allow app to Logstash directly. 2017-10-27 22:56:32 +08:00
Guo Xiang Tan
90d6677d97 EXPERIMENTAL: Allow lograge to be selected as the logging library. 2017-10-27 17:54:45 +08:00
Guo Xiang Tan
d2ac7a9476 Partially revert 4e89d3f778.
* SiteSetting uses DistributedCache which depends on MessageBus.
  Therefore, we need to initialize MessageBus first.
2017-10-12 15:15:24 +08:00
Guo Xiang Tan
23415caecd Print out backtrace as well. 2017-10-12 14:52:18 +08:00
Guo Xiang Tan
4e89d3f778 Only refresh site settings once on initialization. 2017-10-12 14:12:18 +08:00
Guo Xiang Tan
630f7bfba2 Remvoe unused file. 2017-10-12 13:45:00 +08:00
Guo Xiang Tan
69f56e8745 Remove unused file. 2017-10-11 16:05:41 +08:00
Sam
9b4fd0b26b correct multisite issues with scheduler 2017-10-11 18:46:53 +11:00