Commit Graph
100 Commits
Author SHA1 Message Date
David Taylor 2f96474155 FIX: Ignore empty search terms in topic queries
Previously an empty search term would cause an invalid tsquery, and led to a 500 error. Now an empty string will be ignored.
2020-09-10 15:49:11 +01:00
David Taylor d84dee4519 FIX: Account for different composer 'edit' states in discourse-presence (#10631)
Previously we were manually comparing the composer 'action' to the REPLY and EDIT constants. This meant that other forms of editing (e.g. shared drafts) were not included. This commit changes the presence logic to use the existing composer computed variables.
2020-09-10 10:52:51 +01:00
David Taylor 6d2b33035a FIX: Bump onebox to fix soundcloud oneboxes
Followup to a3577435f7
2020-08-27 22:46:46 +01:00
David Taylor a3577435f7 FEATURE: Additional control of iframes in oneboxes (#10523)
This commit adds a new site setting "allowed_onebox_iframes". By default, all onebox iframes are allowed. When the list of domains is restricted, Onebox will automatically skip engines which require those domains, and use a fallback engine.
2020-08-27 20:12:13 +01:00
David Taylor 89cb537fae DEV: Switch letter avatar service to new domain (#10546) 2020-08-27 12:05:36 +01:00
David Taylor 4351fa435e FIX: Correctly redirect after external login on subfolder sites (#10529) 2020-08-26 15:52:21 +01:00
David Taylor 9debfed060 FIX: Add cache-busting parameter to ace-editor javascript
Long-term we should consider adding the file hash to javascript URLs, so that the URL changes whenever the file changes
2020-08-24 12:29:46 +01:00
David Taylor ac80109705 DEV: Improve user card styling extensibility (#10508)
- Add a metadata-row class
- Remove wrapper tags from user-card-after-metadata and user-card-before-badges outlets
- Correct max-height for mobile card
2020-08-24 11:02:40 +01:00
David Taylor 629ee5494d FEATURE: Allow plugins to register parameter-based API routes (#10505)
Example usage:

```
add_api_parameter_route(
  method: :get,
  route: "users#bookmarks",
  format: :ics
)
```
2020-08-24 10:24:52 +01:00
David Taylor 33c21c5921 DEV: Skip readonly tables when remapping content (#10461) 2020-08-18 08:53:12 +01:00
David Taylor 1671cd924d DEV: Bump onebox to 2.0.2 2020-08-18 01:19:52 +01:00
David Taylor 645b65adfa FIX: Add playsinline to twitter GIFs 2020-08-18 00:26:51 +01:00
David Taylor eb981b8fbb FEATURE: Autoplay oneboxed twitter GIF media 2020-08-17 23:56:41 +01:00
David Taylor 5f7839f0a4 FIX: Allow safari to load and autoplay videos in posts
This improves the reloading workaround in a few ways:
  - Multiple videos in posts are now reloaded. Previously only the first was reloaded.
  - An empty `poster` string is treated the same as a missing attribute
  - If the video is set to autoplay, it will be reloaded (and therefore autoplayed correctly)
2020-08-17 23:48:36 +01:00
David Taylor a895e4a15b FIX: Skip rendering twitter video if matching format not found
Followup to d5a4318a
2020-08-17 20:53:14 +01:00
David Taylor d5a4318ac1 FIX: Replace iframe with <video> for twitter videos
This avoids the samesite cookie related error on chrome. It also adds support for twitter 'GIF' content, and allows videos to resize smoothly for narrow devices.
2020-08-17 20:37:36 +01:00
David Taylor 0a5376084d FIX: Ensure auto close notice is posted with system locale
Previously it was created with the locale of the user who created the final post in the topic
2020-08-17 15:40:47 +01:00
David Taylor 8ac85f54fb DEV: Remove blob: workers from CSP (#10440)
Ace editor is reconfigured to load workers directly from their JS URL. Workers must be on the same origin as the site, so they will not use the CDN.
2020-08-14 18:15:30 +01:00
David Taylor 3cf93e9a8f Revert "DEV: Disable ACE editor worker blobs"
This reverts commit d5463d2a4d.

With S3 assets, CORS prevents loading worker assets directly.
2020-08-14 14:12:11 +01:00
David Taylor a5608025aa FIX: Add script asset locations to worker-src CSP directives
We no longer need :blob worker src since d5463d2a. But we do want to allow workers to be loaded from all our existing script-src options.
2020-08-14 12:31:50 +01:00
David Taylor d5463d2a4d DEV: Disable ACE editor worker blobs
This will allow us to remove `blob:` URLs from the worker-src CSP directive
2020-08-14 12:01:46 +01:00
David Taylor bd0a7553c4 DEV: Detect when s3 inventory failure is caused by etag difference (#10427) 2020-08-13 09:30:28 +10:00
David Taylor 451b9b245f DEV: Rename new upload rake tasks
These tasks are s3-specific, so update the names to make that very clear
2020-08-12 23:26:13 +01:00
David Taylor 4476680d4b UX: Set silence_reason using the system locale
Previously we would use the user's locale, which can be confusing for moderators.
2020-08-10 18:51:54 +01:00
David Taylor fe7a7ecf6c FIX: Include secure media URLs when linking post uploads (#10404)
Normally, secure media urls are linked like `/secure-media-uploads/...`. In this case, uploads were already being linked correctly.

But sometimes (e.g. when pulling hotlinked onebox images) secure media is referenced with a full domain name (`//example.com/secure-media-uploads`). This commit ensures that those uploads are also linked correctly.
2020-08-10 17:59:29 +01:00
David Taylor 76c02cac65 DEV: Ensure the correct plugin stylesheets are reloaded in development
If a plugin name contained the name of another plugin, the wrong stylesheets would be reloaded. For example, working on discourse-prometheus-alert-receiver would cause discourse-prometheus stylesheets to be reloaded.
2020-08-06 16:10:57 +01:00
David Taylor 8c03868808 DEV: Correctly render data- attributes in widget hbs templates (#10376)
In virtualdom, element 'properties' are not completely synonymous with element 'attributes'. In particular, `data-` properties will not be rendered as attributes. To ensure all attributes are passed through, we need to include them under an `attributes` key. For more info, see https://github.com/Matt-Esch/virtual-dom/blob/master/docs/vnode.md#custom-attributes-data-
2020-08-06 14:33:09 +01:00
David Taylor df39e372d7 DEV: Add spec for removing and re-adding hotlinked images
Before the recent refactor, this would fail. https://meta.discourse.org/t/154184
2020-08-06 10:01:53 +01:00
David Taylor ceb858c70a PERF: Release post_upload records when downloaded image is removed (#10379)
Previously we would unconditionally keep all images downloaded via pull_hotlinked_images, even if they are later removed from the post. This commit removes that logic, and relies on the existing link_post_uploads process to pick up the downloaded images in `cooked`. Specs are added to ensure this is working correctly for regular hotlinked images, and for oneboxes.
2020-08-06 10:06:34 +10:00
David Taylor cb12a721c4 REFACTOR: Refactor pull_hotlinked_images job
This commit should cause no functional change
- Split into functions to avoid deep nesting
- Register custom field type, and remove manual json parse/serialize
- Recover from deleted upload records

Also adds a test to ensure pull_hotlinked_images redownloads secure images only once
2020-08-05 12:14:59 +01:00
David Taylor 4ef389bc04 FIX: Define s3_helper in EnsureS3UploadsExistence job
Followup to 16c65a94
2020-07-29 12:23:04 +01:00
David Taylor 16c65a94f7 PERF: Preload S3 inventory data for multisite clusters 2020-07-29 10:31:55 +01:00
David Taylor f25fa83b6d FIX: Ensure correct locale is set during RenderEmpty responses
Follow-up to bcb0e623
2020-07-28 22:20:38 +01:00
David Taylor bc11769118 FIX: Bump onebox to 2.0.1 for engine priority fix 2020-07-28 13:29:15 +01:00
David Taylor 407bb96a22 FIX: Avoid validation error when deleting users with locked trust level 2020-07-27 17:40:10 +01:00
David Taylor 61d3640643 DEV: Improve HTML structure for group-reports-nav-item outlet 2020-07-23 12:44:11 +01:00
David Taylor 5b7c016ebc FIX: Restore navigation-bar on tag topic list
This was accidentally removed in ac0d5b1d
2020-07-22 18:26:37 +01:00
David Taylor c09b5807f3 FIX: Include resolved locale in anonymous cache key (#10289)
This only applies when set_locale_from_accept_language_header is enabled
2020-07-22 18:00:07 +01:00
David Taylor bcb0e62363 FIX: Make set_locale an around_action to avoid leaking between requests (#10282) 2020-07-22 17:30:26 +01:00
David Taylor ec4024fe6d FIX: Keep by_users check in S3 inventory
Partial revert of 8515d8fa - the by_users check is ensuring we don't raise errors for fixtures
2020-07-21 17:19:56 +01:00
David Taylor 8515d8fae5 FIX: Improve S3 inventory logic
Previously we considered 'upload rows without etags' to be exempt from the check. This is bad, because older/migrated sites might not have etags on all their uploads. We should consider rows without etags to be broken, since we can't check them against the inventory.

This also removes the `by_users` scope. We need all uploads to be working, even ones created by the system user.
2020-07-21 15:55:53 +01:00
David Taylor 5f3dfce4eb FIX: Listing topics with muted mixed-case tags (#10268)
When visiting a tag page directly, we should display all topics, even if that tag is muted. This was not working for mixed-case tags.
2020-07-20 11:01:29 +01:00
David Taylor 85d1677b26 DEV: Drop unsupported-browser plugin (#10261)
Discourse 2.6 will not have support for older browsers (e.g. IE11)
2020-07-17 15:04:06 +01:00
David Taylor fab8b8649e PERF: Combine avatar_lookup and primary_group_lookup into user_lookup (#10253)
These two classes were running very similar queries, which could be expensive on large topics
2020-07-17 10:48:08 +01:00
David Taylor 7d300006a1 Revert "PERF: Move highlightjs to a background worker, and add result cache (#10191)"
This caused a CORS error when used with S3 asset storage

This reverts commit d09f283e91.
2020-07-15 13:52:35 +01:00
David Taylor c802c7367a FIX: Allow highlightjs-worker to be compiled successfully 2020-07-15 13:17:52 +01:00
David Taylor d09f283e91 PERF: Move highlightjs to a background worker, and add result cache (#10191)
Syntax highlighting is a CPU-intensive process which we run a lot while rendering posts and while using the composer preview. Moving it to a background worker releases the main thread to the browser, which makes the UX much smoother.
2020-07-15 12:48:07 +01:00
David Taylor cc6d722de1 FIX: Correct user profile URLs in /about crawler view 2020-07-14 16:09:27 +01:00
David Taylor 3d65678a13 DEV: Add timestamp columns to optimized_images table (#10199)
This allows us to filter by created/updated date when comparing to an S3 inventory.
2020-07-14 11:50:33 +01:00
David Taylor fbf29443bd FIX: Allow signup when auth provider supplies no email 2020-07-10 17:56:33 +01:00
David Taylor 4b0d86d8d3 FIX: Do not display enabled inputs when username/name are locked 2020-07-10 10:49:32 +01:00
David Taylor cb1f891392 Revert "FIX: Incorrect search blurb when advanced search filters are used."
This change was causing advanced search filters to disappear from the search input

This reverts commit 2e1eafae06.
2020-07-09 16:19:18 +01:00
David Taylor da0fc0a9d3 DEV: Cleanup PostActionType and ReviewableScore changes in tests
Followup to 2df388ffd7
2020-07-06 17:12:23 +01:00
David Taylor 7f2b5a446a PERF: Remove post_upload recovery in daily EnsureS3UploadsExistence job (#10173)
This is a very expensive process, and it should only be required in exceptional circumstances. It is possible to run a similar recovery using `rake uploads:recover` (https://github.com/discourse/discourse/blob/5284d41a8e8cfca1ab5ca21574429f082fb187dd/lib/upload_recovery.rb#L135-L184)
2020-07-06 16:26:40 +01:00
David Taylor 98786cb037 DEV: Refactor basic-topic-list to use topic-list-item on mobile (#10105)
Previously, basic-topic-list had its own implementation of topic-list-item on mobile, which made it more difficult to maintain and extend. The visible difference was that the basic-topic-list implementation had no large avatar on the left. This commit adds a new hideMobileAvatar parameter to topic-list-item and topic-list, and sets it to `true` for the basic-topic-list component.
2020-07-06 13:58:57 +01:00
David Taylor cb1b472a0f PERF: Do not include thumbnail information in default topic list payload (#10163)
Now it is only included when a theme/plugin has requested it.
2020-07-06 10:59:21 +01:00
David Taylor 5284d41a8e FEATURE: Optionally skip the create account popup for external auth 2020-07-06 10:18:57 +01:00
David Taylor 041d28295f UX: Hide login button during externally authenticated account creation
Also switch to use the new d-button loading property for the create account button
2020-07-06 10:18:49 +01:00
David Taylor 977766e7a8 FEATURE: sso_overrides_(email|username|name) for all auth methods
These settings previously applied only to discourse-sso. Now they work for all external authentication methods.
2020-07-06 10:18:45 +01:00
David Taylor ec448a1516 DEV: Refactor Auth::Result for readability, recreate during signup flow 2020-07-06 10:18:41 +01:00
David Taylor 4c1e690e32 DEV: after_extract_linked_users hook for plugins to modify user list
This allows plugins to change the list of users which will be sent 'linked' notifications
2020-07-03 14:52:49 +01:00
David Taylor 95153356ea PERF: Refactor lightbox decorator to use querySelectorAll (#10158)
Previously we were using `$elem.find(...).not($elem.find(...))`. This took approximately 2ms on my machine with a test post.

This commit switches to using a native querySelectorAll method, which takes less than 0.5ms on the same test post.
2020-07-02 11:04:19 +01:00
David Taylor 76d5e54aab PERF: Check for modal visibility in a more efficient way
This code runs on every keyup event in the application, so it needs to be efficient. Previously we were iterating over the whole document using the JQuery :visible selector. Per the JQuery docs at https://api.jquery.com/visible-selector/

> Using this selector heavily can have performance implications, as it may force the browser to re-render the page before it can determine visibility. Tracking the visibility of elements via other methods, using a class for example, can provide better performance.

We already had a `hidden` class on the modal element which we can check, so we can check that instead.
2020-07-01 17:49:23 +01:00
David Taylor 4db9f7bb73 FIX: Hide publish_read_state option from non-admin users (#10148) 2020-06-30 16:49:04 +01:00
David Taylor 0edffcc47d FIX: Correct version comparison logic when comparing stable to beta (#10135)
* FIX: Correct version comparison logic when comparing stable to beta

For example, version 1.3.0 should be considered higher than 1.3.0.beta3. So `Discourse.has_needed_version?('1.3.0', '1.3.0.beta3')` should return true

* Switch to use Gem::Version to compare versions
2020-06-29 17:52:33 +10:00
David Taylor c5078e5dc1 DEV: Remove accidentally committed puts statements 2020-06-23 12:41:47 +01:00
David Taylor d473ddd02c FIX: Use raw content when checking for watched words to flag (#10104)
This makes flagged watched words match the behavior of censor, requires_approval and blocked watched words.
2020-06-22 19:48:06 +01:00
David Taylor 17c4f76eac FIX: Do not attempt to pull_hotlinked on emoji images when CDN enabled (#10091) 2020-06-19 20:21:05 +01:00
David Taylor a99bb0ded4 Revert "FIX: Do not attempt to pull_hotlinked on emoji images when CDN enabled"
This changed cause plugin spec failures and needs further investigation

This reverts commit 78626d2832.
2020-06-19 14:39:16 +01:00
David Taylor 9f2e7e4651 FIX: Handle invalid URLs gracefully when pulling hotlinked images 2020-06-19 12:52:51 +01:00
David Taylor 78626d2832 FIX: Do not attempt to pull_hotlinked on emoji images when CDN enabled 2020-06-19 12:45:06 +01:00
David Taylor e29afa200a FIX: Cleanup migrations with timestamps in the future
A future-dated migration was accidently introduced by me in 45c399f0. This was removed in b9762afc, but other migrations had already been generated based on its incorrect date. This commit removes the offending data in the schema_migrations table, and corrects the version in the published_pages migration.

This commit also adds a check to db:migrate which raises an error when invalid migration timestamps are used.
2020-06-17 15:58:22 +01:00
David Taylor 159fc41f40 FIX: Restore missing translation keys
These were accidentally removed/renamed in 5bfe1ee4
2020-06-17 14:24:22 +01:00
David Taylor 6caad5c083 FIX: Do not send staff welcome message if user already has role 2020-06-17 12:12:55 +01:00
David Taylor a2f80670e1 FIX: Do not count youtube thumbnail when counting post images (#10049) 2020-06-15 20:25:30 +01:00
David Taylor 45c399f0d7 DEV: Correct historical badge and user_badge index discrepancies (#10017)
Badge and user_badge tables were created using add_column index: true
When the migration was written, `index: true` was a no-op for non-reference columns
Since then, rails made it work https://github.com/rails/rails/commit/9a0d35e820464f872b0340366dded639f00e19b9
This migration adds the index to very old sites, so that we have a consistent state
2020-06-11 11:46:28 +01:00
David Taylor c74f2b949f DEV: Correct historical posts table schema discrepancies (#10014)
This is a no-op for modern installations. It only affects older sites.

Follow-up to f7d676dce1

This needs to be done in a transaction, because we need to drop and recreate the badge_posts view.
2020-06-11 11:45:46 +01:00
David Taylor 9e98c02dd7 FIX: Don't call cooked decorators in composer if cooked is unchanged
If cooked is unchanged, ember will not re-render the preview area, so we should not re-run the post-processing decorators. This can cause issues when decorateCooked functions are not idempotent.
2020-06-09 23:01:07 +01:00
David Taylor ae6c4cd237 FIX: Allow merging users when email domains are restricted 2020-06-09 18:51:59 +01:00
David Taylor f1fc6cd40d DEV: Add user-card-additional-buttons plugin outlet
The user-card-additional-controls outlet is outside the `<li>` element, which makes styling difficult. Placing an outlet inside the button list means that themes/plugins can easily reuse core styling.
2020-06-09 14:33:24 +01:00
David Taylor 472862e87d DEV: Do not attempt to change schema of posts table
Changing this requires rebuilding the badge_posts view, so will need to be done separately

Followup to f7d676dc
2020-06-08 13:52:31 +01:00
David Taylor 556380f970 DEV: Do not attempt to change schema badge_posts view
Followup to f7d676dc
2020-06-08 13:08:28 +01:00
David Taylor f7d676dce1 DEV: Correct historical schema discrepancies (#9955)
Rails schema defaults have changed over the years. This migration will correct discrepancies, and make all databases match the modern schema.
2020-06-08 12:20:27 +01:00
David Taylor 833b5d89e0 PERF: Cache PrettyText instance for rendering composer preview (#9987)
Previously we were building pretty-text from scratch on every keypress
2020-06-08 10:42:59 +01:00
David Taylor 293467a37a REFACTOR: Remove jquery usage from resolveAllShortUrls, and fix debounce
- This function now requires an explicit scope. It will never run on the entire document.

- Previously debounce was being used with an anonymous function, which means it was having no effect.
2020-06-05 17:15:14 +01:00
David Taylor 0f09fd22f3 DEV: Remove attachment css handling from resolveAllShortUrls
This was moved inside the markdown processor in 65481858, so there is no longer any need to run this logic after cooking
2020-06-05 15:12:30 +01:00
David Taylor be5974734d PERF: Pass element to resolveAllShortUrl function, not text selector (#9988)
We already have a reference to the preview element, so there is no need to look it up again. This saves about 1ms per keypress in my testing.
2020-06-05 13:37:07 +01:00
David Taylor 731263b81c PERF: Correctly memoize baseUri value in javascript app (#9986)
An empty string is a falsey value in javascript, so we were looking for the meta tag every time getURL was called, which took approximately 1.5ms every time.
2020-06-05 13:36:48 +01:00
David Taylor 6c7e9d3255 DEV: Remove htmlparser dependency (#9981)
We stopped using htmlparser in b5008586. There is no need to list it as a dependency
2020-06-05 13:29:50 +01:00
David Taylor 75b1298e99 DEV: Drop unused image_url column from posts and topics (#9953)
This has been superseded by image_upload_id. The image_url value in API responses is now generated dynamically from the upload record.
2020-06-02 16:21:38 +10:00
David Taylor 3106f85983 FIX: Support exporting reports which reference topics (#9957) 2020-06-01 18:23:58 +01:00
David Taylor 7b9d207175 DEV: Restore default_list_filter column migration
This migration was added to master temporarily, then removed. This resulted in some sites being in an inconsistent state. Adding the migration back for now. If we decide the column is unneeded, then we should add a second migration to remove the column.

This reverts part of commit e1af91f5ae.
2020-06-01 18:16:00 +01:00
David Taylor 13d00eaad0 Drop support for outdated browsers (#9956)
In particular, this includes Internet Explorer 11. Code which existed to support old browsers will be removed over the coming weeks.

https://meta.discourse.org/t/discourse-is-ending-support-for-internet-explorer-11-ie11-on-june-1-2020/137984
2020-06-01 16:38:44 +01:00
David Taylor ecfce93f28 FIX: Support IRIs (unicode URIs) when pulling hotlinked images (#9928) 2020-05-29 17:47:05 +01:00
David Taylor e159fb06df FEATURE: Download remote images even for old posts (#9925)
When a post is rebaked, the admins expect it to work the same regardless of how old the post is.
2020-05-29 17:13:55 +01:00
David Taylor 28f46c171c FIX: Pull hotlinked images even when edited by system users (#9890)
Previously the pull hotlinked images job was skipped after system edits. This ensured that we never had an infinite loop of system-edit/pull-hotlinked/system-edit/pull-hotlinked etc.

A side effect was that edits made by system for any other reason (e.g. API, removing full quotes) would prevent pulling hotlinked images. This commit removes the system edit check, and replaces it with another method to avoid an infinite job scheduling loop.
2020-05-29 13:07:47 +01:00
David Taylor d29d69e10d FIX: Invalidate database theme cache when hostname changes (#9908)
Hostname can vary per-site on a multisite cluster, so this change requires converting the compiler_version from a constant into a class method which is evaluated at runtime. The value is stored in the theme DistributedCache, so performance impact should be negligible.
2020-05-29 13:04:51 +01:00
David Taylor 8a3d9d7036 DEV: Run jobs sequentially in test mode (#9897)
When running jobs in tests, we use `Jobs.run_immediately!`. This means that jobs are run synchronously when they are enqueued. Jobs sometimes enqueue other jobs, which are also executed synchronously. This means that the outermost job will block until the inner jobs have finished executing. In some cases (e.g. process_post with hotlinked images) this can lead to a deadlock.

This commit changes the behavior slightly. Now we will never run jobs inside other jobs. Instead, we will queue them up and run them sequentially in the order they were enqueued. As a whole, they are still executed synchronously. Consider the example

```ruby
class Jobs::InnerJob < Jobs::Base
  def execute(args)
    puts "Running inner job"
  end
end

class Jobs::OuterJob < Jobs::Base
  def execute(args)
    puts "Starting outer job"
    Jobs.enqueue(:inner_job)
    puts "Finished outer job"
  end
end

Jobs.enqueue(:outer_job)
puts "All jobs complete"
```

The old behavior would result in:

```
Starting outer job
Running inner job
Finished outer job
All jobs complete
```

The new behavior will result in:
```
Starting outer job
Finished outer job
Running inner job
All jobs complete
```
2020-05-28 12:52:27 +01:00
David Taylor bde8862f0f FIX: Allow GitHub app client_id to be used for OAuth configuration 2020-05-21 10:44:25 +01:00
David Taylor e9f579bd3f DEV: Clean up thumbnail tests correctly 2020-05-19 11:04:02 +01:00