Commit Graph
100 Commits
Author SHA1 Message Date
David Taylor e6e47f2fb2 SECURITY: Add confirmation screen when logging in via user-api OTP 2019-06-17 16:18:44 +01:00
David Taylor 52387be4a4 SECURITY: Add confirmation screen when logging in via email link 2019-06-17 16:18:37 +01:00
David Taylor 5f6f707080 Revert "Merge pull request from GHSA-hv9p-jfm4-gpr9"
This reverts commit b8340c6c8e.
2019-06-17 16:17:10 +01:00
David Taylor b8340c6c8e Merge pull request from GHSA-hv9p-jfm4-gpr9
* SECURITY: Add confirmation screen when logging in via email link

* SECURITY: Add confirmation screen when logging in via user-api OTP

* FIX: Correct translation key in session controller specs

* FIX: Use .email-login class for page
2019-06-17 15:59:41 +01:00
David Taylor ed21128ee6 FIX: Do not change directory when decompressing S3 inventory
In sidekiq, jobs are run in multiple threads within the same process. `cd` affects the entire process, so can cause unexpected issues in other running jobs.
2019-06-13 17:13:50 +01:00
David Taylor f60451a89d DEV: Fix linting error 2019-06-12 00:19:52 +01:00
David Taylor 0ebe5ec1f8 FIX: Check postStream.gaps exists before trying postSteam.gaps.after 2019-06-11 23:48:21 +01:00
David Taylor f4fd75aea4 DEV: Rename variable to avoid conflict 2019-06-11 13:02:40 +01:00
David Taylor f1d5b992bf DEV: Correct linting error 2019-06-11 12:51:18 +01:00
David Taylor 61b587f66e FIX: Mark ignored posts as 'read', if last visible post is read (#7739) 2019-06-11 12:16:28 +01:00
David Taylor 000a35b219 FIX: Do not live-load posts from ignored users 2019-06-11 12:07:14 +01:00
David Taylor e3a9a2d2dd FIX: Avoid infinite loop if disk space is low
We now continue to enqueue the pull_hotlinked_images job for optimized images, even if disk space is low
2019-06-07 14:24:22 +01:00
David Taylor 65b0cafc03 FIX: Always schedule pull_hotlinked_images in cooked_post_processor
The job is now used to pull optimized images, and images from other sites on the same CDN. This needs to run even if download_remote_images is false
2019-06-07 13:08:23 +01:00
David Taylor 54afa314fb FIX: Do not download emojis in pull_hotlinked_images 2019-06-07 13:00:52 +01:00
David Taylor 0508546fd2 FIX: Accept github theme urls with a trailing slash 2019-06-04 10:28:36 +01:00
David Taylor 6afd7f0eae FIX: Clean broken relations before adding javascript_caches foreign key 2019-06-04 10:01:20 +01:00
David Taylor 8511bfe583 Bump omniauth-google-oauth2 gem version
Pinning to an old version is no longer required following https://github.com/zquestz/omniauth-google-oauth2/commit/8b4d6dafea1e741a34c92d595ffb2e7b99acd874
2019-06-03 19:17:00 +01:00
David Taylor 81dcadf788 DEV: Stop compiling themes during DB migration. Recompile on cdn change. (#7676)
This is an improved implementation for bc8b7b13
2019-06-03 16:38:02 +01:00
David Taylor 5e61893cb2 FIX: Display generic descriptor for associated account with no info 2019-06-03 12:14:02 +01:00
David Taylor 7500eed4c0 FEATURE: Multi-file javascript support for themes (#7526)
You can now add javascript files under `/javascripts/*` in a theme, and they will be loaded as if they were included in core, or a plugin. If you give something the same name as a core/plugin file, it will be overridden. Support file extensions are `.js.es6`, `.hbs` and `.raw.hbs`.
2019-06-03 10:41:00 +01:00
David Taylor ff17177bf9 DEV: Update specs for new stylesheet location
Followup to 75fc126224
2019-05-31 16:15:08 +01:00
David Taylor 75fc126224 DEV: Change path for extra scss files in themes (#7664)
The preferred path is `/stylesheets/...`, to match core and plugins. The old `/scss/...` path will continue to function
2019-05-31 15:40:41 +01:00
David Taylor 03363d03e8 FEATURE: Load theme setting descriptions from theme locale files
Previously theme setting descriptions were defined in the `settings.yml` file like this:
```
setting_name:
  default: "My Default Value"
  description:
    en: "English description"
    fr: "French description"
```

This commit allows developers to store the localised descriptions in the theme locale files instead:
```
en:
  theme_metadata:
    description: Theme Description
    settings:
      setting_name: "The localised description for setting_name"
```
2019-05-31 14:49:59 +01:00
David Taylor 0fbff66d97 DEV: Correct rake task syntax error 2019-05-20 17:43:30 +01:00
David Taylor 31902159af DEV: Allow skipping failed migrations when running S3 migration
Use the SKIP_FAILED environment variable to skip failed sites. Use with caution - make sure you go back and re-run the failed migrations afterwards.
2019-05-20 17:25:56 +01:00
David Taylor a15cca9a0f DEV: Improve error message for posts:missing_uploads during S3 migration 2019-05-20 16:09:22 +01:00
David Taylor 41bc90dd3e DEV: Add progress indicator for post rebake during s3 migration
Now that we run sidekiq jobs synchronously, this is important
2019-05-20 14:19:58 +01:00
David Taylor 77a06209cb DEV: Skip S3 migration if all uploads are already migrated
This makes the task resumable in a multisite context
2019-05-20 14:17:37 +01:00
David Taylor ef660d5a3e FIX: Return consistent character encodings when downloading S3 uploads
Net::HTTP always returns ASCII-8BIT encoding. File.read auto-detects the encoding. This leads to an encoding inconsistency between a fresh download, and a cached download. This commit ensures all downloaded files are treated equally, by always returning the cached version from the filesystem, even during initial download.

One symptom of this problem is during theme exports: https://meta.discourse.org/t/116907

Related ruby ticket: https://bugs.ruby-lang.org/issues/2567
2019-05-17 11:27:00 +01:00
David Taylor 105359e54a FIX: Only delete theme folder if it has been created
If an exception is raised before the folder is created, then this `ensure` block is still run
2019-05-17 09:45:11 +01:00
David Taylor 5666316285 FEATURE: Display 'last updated' on dashboard, improve release notes link (#7560) 2019-05-17 15:42:45 +10:00
David Taylor b5dd4478e5 FIX: Blocked watched words should apply to staff (#7547)
Having different behavior for staff and regular users can make it confusing for admins to understand how their configuration changes affect regular users
2019-05-16 15:19:41 +01:00
David Taylor 1299c94a52 FIX: Make serverside and clientside omniauth origin redirects consistent
Previously external domains were allowed in the client-side redirects, but not the server-side redirects. Now the behavior is to only allow local origins.
2019-05-15 12:40:51 +01:00
David Taylor 33bd47164f DEV: Re-enable uglifier for non-precompiled assets
Keep `harmony` disabled
2019-05-14 10:28:18 +01:00
David Taylor f12709e24c DEV: Temporarily disable uglifier for non-precompiled assets
This is causing performance issues which require further investigation
2019-05-14 09:34:38 +01:00
David Taylor 5605dba85c DEV: Automatically annotate plugin models alongside core models 2019-05-13 16:37:47 +01:00
David Taylor 750c125707 DEV: Include foreign keys in model annotations 2019-05-13 15:53:42 +01:00
David Taylor 1bff81419e DEV: Update annotations 2019-05-13 15:24:24 +01:00
David Taylor 2b28abdc21 FIX: Allow underscores in s3_upload_bucket setting
Underscores are not allowed in s3 bucket names, but they should be allowed in the subfolder name
2019-05-13 11:18:24 +01:00
David Taylor 20daa76829 DEV: Change priority to ultra_low for post-s3-migration rebake 2019-05-10 18:37:45 +08:00
David Taylor abbe812a0d FIX: Correct CSS for instagram video play icon 2019-05-10 10:46:07 +01:00
David Taylor 2c6b595eed FIX: Process image onebox correctly when image is wrapped in a link
The instagram onebox sometimes surrounds the image with an `<a>` tag, which was breaking the aspect ratio logic, and therefore causing posts to change height on load.
2019-05-10 10:02:40 +01:00
David Taylor e84531a6a6 FIX: Correctly clear theme stylesheet cache when changing color scheme 2019-05-08 16:02:55 +01:00
David Taylor b717e23ceb FIX: Dynamically adjust scale when rendering wizard canvases
This was a typo in fd0de64
2019-05-08 13:14:28 +01:00
David Taylor d82da69c2c FIX: Fail gracefully if username parameter is missing when creating user 2019-05-03 17:17:06 +01:00
David Taylor 0644c10bfe FIX: Prioritize VERP key over final_recipient header in bounced email
Per RFC3464 2.3.2, the final_recipient header may not match the address we originally sent the email to.
2019-05-03 12:12:44 +01:00
David Taylor 6fdbf15d6b DEV: Remove Danger from travis build 2019-05-01 15:15:28 +01:00
David Taylor 0e303c7f5d FEATURE: Automatically generate optimized site metadata icons (#7372)
This change automatically resizes icons for various purposes. Admins can now upload `logo` and `logo_small`, and everything else will be auto-generated. Specific icons can still be uploaded separately if required.

## Core

- Adds an SiteIconManager module which manages automatic resizing and fallback

- Icons are looked up in the OptimizedImage table at runtime, and then cached in Redis. If the resized version is missing for some reason, then most icons will fall back to the original files. Some icons (e.g. PWA Manifest) will return `nil` (because an incorrectly sized icon is worse than a missing icon). 

- `SiteSetting.site_large_icon_url` will return the optimized version, including any fallback. `SiteSetting.large_icon` continues to return the upload object. This means that (almost) no changes are required in core/plugins to support this new system.

- Icons are resized whenever a relevant site setting is changed, and during post-deploy migrations

## Wizard

- Allows `requiresRefresh` wizard steps to reload data via AJAX instead of a full page reload

- Add placeholders to the **icons** step of the wizard, which automatically update from the "Square Logo"

- Various copy updates to support the changes

- Remove the "upload-time" resizing for `large_icon`. This is no longer required.

## Site Settings UX

- Move logo/icon settings under a new "Branding" tab

- Various copy changes to support the changes

- Adds placeholder support to the `image-uploader` component

- Automatically reloads site settings after saving. This allows setting placeholders to change based on changes to other settings

- Upload site settings will be assigned a placeholder if SiteIconManager `responds_to?` an icon of the same name

## Dashboard Warnings

- Remove PWA icon and PWA title warnings. Both are now handled automatically.

## Bonus

- Updated the sketch logos to use @awesomerobot's new high-res designs
2019-05-01 14:44:45 +01:00
David Taylor 5edb5c8b03 FIX: Mark invited admins as 'approved' (#7459)
This prevents invited admins appearing as no-op reviewables in the queue when invite_only or require_approval is enabled.
2019-04-30 17:26:39 +01:00
David Taylor 583226d295 FIX: Linking to /new-topic from /latest
`/new-topic` redirects back to `/latest`, so the ember router considers this to be a 'refresh'. This triggers the `resetParams()` logic, which would cause the transition to abort, and the composer would never open.

This commit fixes the `resetParams()` logic so that it uses the default values, instead of setting everything to `null`. Therefore the transition is no longer aborted, and the composer opens successfully.

Unfortunately I have not been able to reproduce the issue in a QUnit test.
2019-04-30 16:25:25 +01:00
David Taylor bc8b7b13ef DEV: Recompile theme fields after db:migrate 2019-04-30 11:52:30 +01:00
David Taylor a2ddb6cf2d DEV: Increase max length of theme_field name column
Now that we have custom SCSS files with arbitrary names, it's easy to exceed 30 characters
2019-04-23 12:34:32 +01:00
David Taylor 7826acc4a7 DEV: Replace site_setting_saved DiscourseEvent with site_setting_changed (#7401)
* DEV: Replace site_setting_saved DiscourseEvent with site_setting_changed

site_setting_saved is confusing for a few reasons:
- It is attached to the after_save of the ActiveRecord model. This is confusing because it only works 'properly' with the db_provider
- It passes the activerecord model as a parameter, which is confusing because you get access to the 'database' version of the setting, rather than the ruby setting. For example, booleans appear as 'y' or 'n' strings.
- When the event is called, the local process cache has not yet been updated. So if you call SiteSetting.setting_name inside the event handler, you will receive the old site setting value

I have deprecated that event, and added a new site_setting_changed event. It passes three parameters:
- Setting name (symbol)
- Old value (in ruby format)
- New value (in ruby format)

It is triggered after the setting has been persisted, and the local process cache has been updated.

This commit also includes a test case which describes the confusing behavior. This can be removed once site_setting_saved is removed.
2019-04-18 16:48:01 +01:00
David Taylor 7d301cd0dd DEV: Remove empty DiscourseEvent handler
Logic was moved in ba6d4b2
2019-04-18 12:25:28 +01:00
David Taylor c687e2b921 FIX: Use saved_change_to_value? in site_setting_saved event
Since Rails 5.2, the behavior of `attribute_changed?` inside `after_save` callbacks has changed, so we need to use `saved_change_to_attribute` instead. The site setting local_process_provider in test mode was covering up the issue.
2019-04-17 10:07:00 -04:00
David Taylor af86cf46dc FEATURE: Allow API requests to specify the DISCOURSE_VISIBLE header
This allows API consumers to automatically update the user's last_seen time. The default behaviour is unchanged.
2019-04-15 17:34:34 +01:00
David Taylor 428536788c FIX: Do not error when importing a theme with empty files 2019-04-15 15:46:47 +01:00
David Taylor fd0de64e0d UX: Render wizard previews for high-DPI displays (#7371)
* UX: Render wizard previews for high-DPI displays

Sets up a canvas element of twice the required dimensions, scales all coordinates by 2x, then shrinks the display in css.

* Use window.devicePixelRatio to determine scale factor
2019-04-15 16:16:05 +10:00
David Taylor 268d4d4c82 FEATURE: Multiple SCSS file support for themes (#7351)
Theme developers can include any number of scss files within the /scss/ directory of a theme. These can then be imported from the main common/desktop/mobile scss.
2019-04-12 11:36:08 +01:00
David Taylor 24ef4f7b2b Remove support for disable_jump_reply user setting (#7359) 2019-04-12 09:03:06 +10:00
David Taylor dc703adad7 FEATURE: Add user preference for title counter mode (#7364) 2019-04-12 09:02:18 +10:00
David Taylor 3987691b5e DEV: Correct specs which had exceptions raised in jobs 2019-04-08 16:00:33 +01:00
David Taylor 20cda5d689 DEV: Raise exceptions when jobs fail in test mode
Previously jobs would fail silently in test mode. Now they will raise the exception and cause the relevant test to fail. This identified a few broken tests, which I will fix in a followup commit
2019-04-08 15:59:04 +01:00
David Taylor f524f8f811 Remove Yahoo login support from core and deprecate OpenID2.0 (#7310)
- Plugin developers using OpenID2.0 should migrate to OAuth2 or OIDC. OpenID2.0 APIs will be removed in v2.4.0

- For sites requiring Yahoo login, it can be implemented using the OpenID Connect plugin: https://meta.discourse.org/t/103632

For more information, see https://meta.discourse.org/t/113249
2019-04-08 10:38:25 +01:00
David Taylor 6a05f190c6 PERF: Do not create staged users for most rejected incoming emails (#7301)
Previously we would create users, then destroy them at the end of the job if the post was rejected. Now we do not create users unless required.
2019-04-08 10:36:39 +01:00
David Taylor ef3dd83367 DEV: Remove trailing blank line 2019-04-02 15:49:23 +01:00
David Taylor 02ed5e78e1 DEV: Use parallel-compatible formatter for logging autospec failures
The old method would cause the parallel processes to overwrite each other. The parallel formatter allows multiple processes to write to the same file.
2019-04-02 15:34:30 +01:00
David Taylor 25feb287b8 DEV: Add parallel spec support to autospec
Set PARALLEL_SPEC=1 to use this functionality
2019-04-01 11:06:47 -04:00
David Taylor b375dcb14a DEV: Introduce parallel rspec testing
Adds the parallel_tests gem, and redis/postgres configuration for running rspec tests in parallel. To use:

```
rake parallel:rake[db:create]
rake parallel:rake[db:migrate]
rake parallel:spec
```

This brings the test suite from 12m20s to 3m11s on my macOS machine
2019-04-01 11:06:47 -04:00
David Taylor a6617a63af FIX: Expire theme cache after remap 2019-03-28 22:31:58 +00:00
David Taylor 95d5819218 FIX: Re-download hotlinked optimized images (#7249)
* FIX: Download local images, even if download remote is disabled
2019-03-27 21:31:12 +01:00
David Taylor 0d3531c2f1 FEATURE: Allow auth providers to define specific icons
Previously we relied on the provider name matching the name of the icon. Now icon names are explicitly set. Plugin providers which do not define an icon will get the default "sign-in-alt" icon
2019-03-27 13:25:04 +00:00
David Taylor bb3f8e32e5 DEV: Run pull_hotlinked_images onebox specs without synchronous jobs
This was changed in fa5a1586, and caused a mutex to lock up, adding 60 seconds to the test suite.
2019-03-22 19:27:37 +00:00
David Taylor a9d5ffbe3d FIX: Prevent critical emails bypassing disable, and improve email test logic
- The test_email job is removed, because it was always being run synchronously (not in sidekiq)
- 34b29f62 added a bypass for critical emails, to match the spec. This removes the bypass, and removes the spec.
- This adapts the specs for 72ffabf6, so that they check for emails being sent
- This reimplements c2797921, allowing test emails to be sent even when emails are disabled
2019-03-22 17:28:43 +08:00
David Taylor 59491f3047 FIX: Add sassc-rails for plugins using sprockets to compile scss
This did not affect core because we have a custom stylesheet pipeline
2019-03-21 23:23:29 +00:00
David Taylor 3f9e7eb326 FIX: Respect the disable_emails=non-staff site setting correctly
This reverts commit c279792130.

This commit inadvertently removed all of the non-staff email logic, rather than just for the 'test email' button. 

https://meta.discourse.org/t/112231/5
2019-03-21 21:44:14 +00:00
David Taylor 56b4ee43d3 DEV: Update sassc gem 2019-03-20 16:04:26 +00:00
David Taylor 01dd96d178 FIX: Use default frame size for Instagram login
I don't know why this was set to 1x1 pixels, but it shouldn't have been
2019-03-19 18:57:57 +00:00
David Taylor e2d6edd5ff DEV: Correct email disabled banner tests
Followup to acc121fd27
2019-03-18 17:06:29 +00:00
David Taylor acc121fd27 FIX: Show 'emails disabled' to staff users when disabled for non-staff (#7187) 2019-03-18 16:26:18 +01:00
David Taylor 4e8c174ee5 FEATURE: Display notification count in title for logged in users (#7184)
This replaces the 'contextual' counters for logged in users. We will still use the old method for anon users
2019-03-18 12:59:47 +00:00
David Taylor ddd3e43326 FIX: Skip create_post_notice when importing
Imports don't necessarily happen in chronological order, so this can cause some very unusual behavior
2019-03-15 13:48:35 +00:00
David Taylor fc0cf3ecd1 FEATURE: Fetch email from auth provider if current user email is invalid (#7163)
If the existing email address for a user ends in `.invalid`, we should take the email address from an authentication payload, and replace the invalid address. This typically happens when we import users from a system without email addresses.

This commit also adds some extensibility so that plugin authenticators can define `always_update_user_email?`
2019-03-14 11:33:30 +00:00
David Taylor 420c6f8102 FEATURE: Skip sending emails to domains on the .invalid TLD (#7162)
This is a reserved TLD which we use when importing users without an email address. https://tools.ietf.org/html/rfc2606
2019-03-13 16:17:59 +00:00
David Taylor 0a4562253e DEV: Add 'starting' event to sidekiq log when interval logging enabled 2019-03-08 10:56:36 +00:00
David Taylor e2510d79cc DEV: Improve thread-safety of sidekiq logging 2019-03-08 10:31:49 +00:00
David Taylor 9db05a895a DEV: Add job_id to sidkiq log
This makes it easier to correlate 'pending' logs from the same job
2019-03-08 09:16:13 +00:00
David Taylor b101065bad FIX: Sanitize theme settings properly before injecting into stylesheets (#7031) 2019-03-08 08:58:06 +00:00
David Taylor bc52437153 FIX: Add additional rescue for assets:precompile
This error can be caused if assets are precompiled before the database has been migrated
2019-03-07 14:03:05 +00:00
David Taylor fc7938f7e0 REFACTOR: Migrate GoogleOAuth2Authenticator to use ManagedAuthenticator (#7120)
https://meta.discourse.org/t/future-social-authentication-improvements/94691/3
2019-03-07 11:31:04 +00:00
David Taylor df474bceee DEV: Further sidekiq logging stability improvements
- Open the log file in "append" mode. This avoids issues if the file does not exist (and matches standard rails log behavior)
- Correctly parse the interval logging environment variable
2019-03-06 12:50:15 +00:00
David Taylor fe62de68dd DEV: Correct sidekiq logging to avoid thread leak 2019-03-06 10:11:31 +00:00
David Taylor 8b30ed5b7a DEV: Serialize the job parameters in sidekiq logs
Otherwise this can lead to some very large data structures when processing the logs later
2019-03-05 17:44:49 +00:00
David Taylor 35275c137b PERF: Remove final N+1 from theme admin page
The rails bug previously mentioned only causes an issue when using `pluck`, so this controller is not affected.

Followup to a8ffc02d06
2019-03-05 12:10:53 +00:00
David Taylor 8963f1af30 FEATURE: Optional detailed performance logging for Sidekiq jobs (#7091)
By default, this does nothing. Two environment variables are available:

- `DISCOURSE_LOG_SIDEKIQ`

  Set to `"1"` to enable logging. This will log all completed jobs to `log/rails/sidekiq.log`, along with various db/redis/network statistics. This is useful to track down poorly performing jobs.

- `DISCOURSE_LOG_SIDEKIQ_INTERVAL`

  (seconds) Check running jobs periodically, and log their current duration. They will appear in the logs with `status:pending`. This is useful to track down jobs which take a long time, then crash sidekiq before completing.
2019-03-05 11:19:11 +00:00
David Taylor e67474daf4 UX: Don't link to theme source unless it's http(s) 2019-02-28 17:01:59 +00:00
David Taylor 3517103398 UX: Correct locale key for theme compatibility error 2019-02-28 14:40:33 +00:00
David Taylor 6296554ed1 DEV: Add additional hook for theme-creator 2019-02-28 10:48:30 +00:00
David Taylor b907924c93 FIX: ManagedAuthenticator should automatically update last_used time 2019-02-27 15:29:26 +00:00
David Taylor a8ffc02d06 PERF: Reduce N+1s on theme admin page 2019-02-26 14:22:02 +00:00