Commit Graph

7724 Commits

Author SHA1 Message Date
Krzysztof Kotlarek
e7c72cd1e4
FIX: deprecate whitelist constants (#10716)
Deprecation of:
WHITELISTED_REDIRECT_HOSTNAMES
CUSTOM_INTERPOLATION_KEYS_WHITELIST
WHITELISTED_SVG_ELEMENTS
2020-09-28 13:52:05 +10:00
Mark VanLandingham
b8015ab654
FIX: Dismiss unread respects tracked query param (#10714)
* WIP:  'dismiss...' respectes tracked query param

* Address review comments

* Dismiss new respects query params

* Remove comment

* Better variable name

* remove self
2020-09-25 12:39:37 -07:00
Penar Musaraj
b1692422b1
FEATURE: Support using system font in wizard font step (#10742)
Co-authored-by: Bianca Nenciu <nbianca@users.noreply.github.com>
2020-09-25 09:39:14 -04:00
Neil Lalonde
2a268bd89b
Version bump to v2.6.0.beta3 2020-09-24 16:05:52 -04:00
Arpit Jalan
cdf45f4fe6 Update regex for views search filter. 2020-09-24 17:05:55 +05:30
Arpit Jalan
0c5cd0d1ef FEATURE: advanced search filters for view count 2020-09-24 15:22:18 +05:30
Bianca Nenciu
214b4c3910
FIX: Remove category id from category class name (#10712)
The generated class name included ID (parent_slug-child_slug-child_id),
but the client side did not expect it (parent_slug-child_slug).
2020-09-23 19:22:07 +03:00
Mark VanLandingham
9f73e8779d
FIX: Only staff can edit topic details when first post is locked (#10729) 2020-09-23 11:13:18 -05:00
Bianca Nenciu
eb891778ff
DEV: Do not translate font names (#10723)
Use the names as provided by discourse-fonts and remove the
translated strings.

It also ensures that the selected font is present in case a font will
be removed in the future.
2020-09-23 12:00:07 +03:00
Bianca Nenciu
4abbe3d361
FEATURE: Make search filters case insensitive (#10715) 2020-09-23 11:59:42 +03:00
Vinoth Kannan
18f0cef72e DEV: skip topic deletion staff log when user skip new user tips.
Previously, every time when a user skipped the "new user tips" it will add a staff action log since the discobot's welcome post is destroyed.
2020-09-23 12:25:39 +05:30
Robin Ward
ce3fe2f4c4 REFACTOR: Support bundling our admin section as an ember addon 2020-09-22 15:14:29 -04:00
Krzysztof Kotlarek
0bb51dcbfa
FIX: TL2 promotion message and advance training (#10679)
This is a little bit of refactoring. Core Discourse should have default promotion message for TL2.

In addition, when the Discobot plugin is enabled, the user is invited to advanced training
2020-09-22 10:17:52 +10:00
jbrw
8dac8abdb4
DEV - write JS files to versioned subdirectories (#10707)
* DEV - write JS files to versioned subdirectories

* Cleaner JS in cacheBuster
2020-09-21 15:06:56 -04:00
Penar Musaraj
577293c438
FIX: respect moderator group permissions in guardian (#10713)
Since 9e4ed03, moderators can view groups with visibility level set to "Group owners, members and moderators".

This fixes an issue where moderators can see the group in /g but then get a 404 when clicking on individual groups.
2020-09-21 12:32:43 -04:00
Neil Lalonde
f1743ff69c
FIX: error "unknown attribute verified" in uploads rake tasks
The new column is verification_status.
verified: nil is now verification_status: unchecked
2020-09-21 10:47:52 -04:00
Arpit Jalan
376255ed84 UX: add width and height to GitHub onebox inline avatar in emails. 2020-09-21 11:47:09 +05:30
David Taylor
fe932eb644
FIX: Ensure multisite DB is set correctly when spawning backup process
Followup to 76477a1c8b
2020-09-18 20:06:49 +01:00
Penar Musaraj
c664ba8fa8
DEV: Fix flaky UploadSecurity spec 2020-09-18 14:36:41 -04:00
Anthony Dillon
01afcf30cc
FIX: Whitelist marker elements when uploading SVGs (#10703) 2020-09-18 14:13:07 -04:00
Roman Rizzi
f85f73be88
FEATURE: Review posts with media. (#10693)
To check if a post contains any embedded media, we look if the "image_sizes" attribute is present in the new post manager arguments.

We want to see one boxed links, but we only store the raw content of the post. To work around this, I extracted the onebox logic from the composer editor into a module.
2020-09-18 12:45:09 -03:00
Gerhard Schlager
76477a1c8b FIX: Forking prevented notifications from being sent after backup
This is a workaround for https://github.com/rubyjs/mini_racer/issues/175
2020-09-18 17:35:17 +02:00
Martin Brennan
14b324e5ed
FIX: Provide better API for registering custom upload public types (#10697)
With secure media and the UploadSecurity class, we need a nice way for plugins to register custom upload types that should be considered public and never secure.
2020-09-18 11:54:33 +10:00
Martin Brennan
80268357e7
DEV: Change upload verified column to be integer (#10643)
Per review https://review.discourse.org/t/dev-add-verified-to-uploads-and-fill-in-s3-inventory-10406/14180

Change the verified column for Upload to a verified_status integer column, to avoid having NULL as a weird implicit status.
2020-09-17 13:35:29 +10:00
Penar Musaraj
6610576deb
FIX: Clear stylesheet cache on base font change
Previously, we were not clearing the cache on themes using the default
color scheme.
2020-09-15 13:13:09 -04:00
Robin Ward
e7abba9bbe FIX: Missing files from previous commit 2020-09-15 10:59:41 -04:00
Penar Musaraj
273db57d6e
FEATURE: Allow admins to delete user SSO records in the UI (#10669)
Also displays the user's last payload in the admin UI to help with debugging SSO issues.
2020-09-15 10:00:10 -04:00
Joffrey JAFFEUX
bbddce4d3a
DEV: updates js transpiler to use babel 7 (#10627)
Updates our js transpiler code to use Babel 7.11.6

List of changes in this commit:

- Updates plugins, babel plugins all have a new version which doesn't contain -es2015- anymore
- Drops [transform-es2015-classes](https://babeljs.io/docs/en/babel-plugin-transform-classes) this plugin shouldn't be needed now that we don't support IE
- Drops check-es2015-constants, checking constants is now part of babel and the check-constants plugin is deprecated. As a result the behavior slightly changed, and is now wrapping every const call in a readOnlyError function which would throw if assigned a new value. This explains the modified spec.
- Adds [proposal-optional-chaining](https://babeljs.io/docs/en/babel-plugin-proposal-optional-chaining)

```javascript
const obj = {
  foo: {
    bar: {
      baz: 42,
    },
  },
};

const baz = obj?.foo?.bar?.baz; // 42
```

- Adds [proposal-json-strings](https://babeljs.io/docs/en/babel-plugin-proposal-json-strings)

```javascript
// IN
const ex = "before
after";
//                ^ There's a U+2028 char between 'before' and 'after'


// OUT
const ex = "before\u2028after";
//                ^ There's a U+2028 char between 'before' and 'after'
```

- Adds [proposal-nullish-coalescing-operator](https://babeljs.io/docs/en/babel-plugin-proposal-nullish-coalescing-operator)

```javascript
var object = {};
var foo = object.foo ?? "default"; // default
```

- Adds [proposal-logical-assignment-operators](https://babeljs.io/docs/en/babel-plugin-proposal-logical-assignment-operators)

```javascript
let a;
let b = 2;
a ||= b; // 2
```

- Adds [proposal-numeric-separator](https://babeljs.io/docs/en/babel-plugin-proposal-numeric-separator)

```javascript
let budget = 1_000_000_000_000;
console.log(budget === 10 ** 12); // true
```

- Adds proposal-object-rest-spread https://babeljs.io/docs/en/babel-plugin-proposal-object-rest-spread

```javascript
let { x, y, ...z } = { x: 1, y: 2, a: 3, b: 4 };
console.log(x); // 1
console.log(y); // 2
console.log(z); // { a: 3, b: 4 }
```

- Adds proposal-optional-catch-binding https://babeljs.io/docs/en/babel-plugin-proposal-optional-catch-binding

```javascript
try {

} catch {

} finally {
  // ensures finally is available in every browsers
}
```

- Adds improved regex support for firefox through (transform-dotall-regex](https://babeljs.io/docs/en/next/babel-plugin-transform-dotall-regex.html) and (proposal-unicode-property-regex](https://babeljs.io/docs/en/babel-plugin-proposal-unicode-property-regex)

- Drops async/generator stuff, the browser we target should allow to use this (excepts iterable async)
2020-09-15 09:26:33 +02:00
Guo Xiang Tan
2ff16b3650
FIX: TopicQuery.list_private_messages_unread ignore notification level 2020-09-15 13:33:11 +08:00
Penar Musaraj
06b4ca5dc7
FIX: Mark only uploads as verified/unverified in S3 inventory 2020-09-14 10:21:34 -04:00
David Taylor
66eda8c9df
DEV: Add include_pms option to TopicQuery (#10647)
This is intended for use by plugins which are building their own topic lists, and want to include PMs alongside regular topics (e.g. discourse-assign). It does not get used directly in core.
2020-09-14 12:07:35 +01:00
Krzysztof Kotlarek
cb58cbbc2c
FEATURE: allow to extend topic_eager_loads in Search (#10625)
This additional interface is required by encrypt plugin
2020-09-14 11:58:28 +10:00
Krzysztof Kotlarek
333ddd4011 SECURITY: return error on oversized images 2020-09-14 10:45:11 +10:00
Justin DiRose
450c731b54
DEV: Add discourse-algolia to official plugins (#10657) 2020-09-11 15:44:06 -05:00
jbrw
033cebf978
DEV - versions of JS files written to a JS file to be included by loa… (#10649)
* DEV - versions of JS files written to a JS file to be included by load-script and appended as params to URLs

* Formatting

* Incorporate feedback from PR

* Update filename of public-js-versions
2020-09-11 13:53:56 -04:00
Gerhard Schlager
81395be4c1 FIX: Ensure uploads are linked to post when revising a post
Editing a post didn't update the `post_uploads` right away. Instead it relied on the `CookedPostProcessor`. This can lead to an inconsistent state if uploads are added or removed during an edit and, for some reason, the `ProcessPost` job doesn't run (successfully). This inconsistency leads to missing uploads, because the newly added uploads appear to be unused and will be deleted by the `CleanUpUploads` job. In addition to that, uploads, which got removed during the edit, appear to be still in use and won't be deleted by the background job.

This commit ensures that the `post_uploads` are updated during the edit without relying on a background job.
2020-09-11 11:49:15 +02:00
Guo Xiang Tan
dbc630f45b
PERF: Fix N+1 queries on private messages route. 2020-09-11 15:20:27 +08:00
Sam Saffron
44fba9463b
FEATURE: Add support for not persistent sessions
In some cases Discourse admins may opt for sessions not to persist when a
browser is closed.

This is particularly useful in healthcare and education settings where
computers are shared among multiple workers.

By default `persistent_sessions` site setting is enabled, to opt out you
must disable the site setting.
2020-09-11 15:11:24 +10:00
Gerhard Schlager
ac70c48be4 FIX: Prevent "uploads are missing in S3" alerts after restoring a backup
After restoring a backup it takes up to 48 hours for uploads stored on S3 to appear in the S3 inventory. This change prevents alerts about missing uploads by preventing the EnsureS3UploadsExistence job from running in the first 48 hours after a restore. During the restore it  deletes the count of missing uploads from the PluginStore, so that an alert isn't triggered by an old number.
2020-09-10 21:37:48 +02:00
Joffrey JAFFEUX
a60ba75ac2
FIX: prevents eslint failure on unmatched patterns in plugins (#10648)
The recent upgrade of eslint fails if no files is found for the defined patterns.
2020-09-10 20:17:40 +02:00
Ahmed Gagan
ddab7cc239
DEV: Add plugin api to permit bulk_action parameters (#10638) 2020-09-10 17:18:45 +01:00
Roman Rizzi
efb9fd6ac0
FIX: Make sure rel attributes are correctly set. (#10645)
We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 12:59:51 -03:00
David Taylor
2f96474155
FIX: Ignore empty search terms in topic queries
Previously an empty search term would cause an invalid tsquery, and led to a 500 error. Now an empty string will be ignored.
2020-09-10 15:49:11 +01:00
Guo Xiang Tan
5732e4288e
DEV: Remove redundant variable. 2020-09-10 14:35:15 +08:00
Martin Brennan
7f2f87bf59
DEV: Review fixes (#10641)
See comments in https://review.discourse.org/t/dev-imap-log-to-database-10435/14337/6 for context.
2020-09-10 13:41:46 +10:00
Guo Xiang Tan
e6ca1b4326
FIX: Admin search for PMs should only search own PMs.
In c6ceda8c, a bug was introduced where an admin searching for his own
private messages will actually end up searching through all private
messages on the site.

Follow-up to c6ceda8c4e
2020-09-10 11:37:18 +08:00
Martin Brennan
dede942007
FEATURE: Allow email image embed with secure media (#10563)
This PR introduces a few important changes to secure media redaction in emails. First of all, two new site settings have been introduced:

* `secure_media_allow_embed_images_in_emails`: If enabled we will embed secure images in emails instead of redacting them.
* `secure_media_max_email_embed_image_size_kb`: The cap to the size of the secure image we will embed, defaulting to 1mb, so the email does not become too big. Max is 10mb. Works in tandem with `email_total_attachment_size_limit_kb`.

`Email::Sender` will now attach images to the email based on these settings. The sender will also call `inline_secure_images` in `Email::Styles` after secure media is redacted and attachments are added to replace redaction messages with attached images. I went with attachment and `cid` URLs because base64 image support is _still_ flaky in email clients.

All redaction of secure media is now handled in `Email::Styles` and calls out to `PrettyText.strip_secure_media` to do the actual stripping and replacing with placeholders. `app/mailers/group_smtp_mailer.rb` and `app/mailers/user_notifications.rb` no longer do any stripping because they are earlier in the pipeline than `Email::Styles`.

Finally the redaction notice has been restyled and includes a link to the media that the user can click, which will show it to them if they have the necessary permissions.

![image](https://user-images.githubusercontent.com/920448/92341012-b9a2c380-f0ff-11ea-860e-b376b4528357.png)
2020-09-10 09:50:16 +10:00
Krzysztof Kotlarek
d260e42c8a
FIX: topic_creator accepts participant_count in import mode (#10632)
The issue mentioned here: https://meta.discourse.org/t/imported-private-discussion-doesnt-appear-in-the-author-inbox/163252

`participant_count` is important to attribute for private messages. If they are imported, we should allow them to set that attribute.

A workaround would be evaluating `update_statistics` method on each Topic but that is less performant.
2020-09-10 08:16:57 +10:00
Roman Rizzi
cac64a95aa
FIX: We should check for watched words first even if the user is a fast typer. (#10630) 2020-09-09 14:36:22 -03:00
Robin Ward
add3d46bc0 FIX: approve post count did not take new topics into account
This means if a user created a new topic, and it was approved, it would not
count towards their approval count. Replies worked correctly.
2020-09-09 12:47:37 -04:00
Penar Musaraj
b409954398
DEV: Improve theme support for color definitions (#10634)
- Lets child components extend color definitions
- Includes default theme color definitions
- Fails gracefully on color stylesheet SCSS errors
- Includes theme variables when extending colors
2020-09-09 11:43:34 -04:00
Guo Xiang Tan
9b75d95fc6 PERF: Keep track of first unread PM and first unread group PM for user.
This optimization helps to filter away topics so that the joins on
related tables when querying for unread messages is not expensive.
2020-09-09 14:05:41 +08:00
Guo Xiang Tan
0398271f87
FIX: Admin should always be able to view group messages.
Follow-up to b0f22f2523.

Before this fix, `SiteSetting.enable_personal_messages = false` will
prevent admin from viewing group messages.
2020-09-09 08:58:21 +08:00
Penar Musaraj
d4a7058cb1
DEV: Strip unicode from color scheme stylesheet filenames (#10628) 2020-09-08 15:00:16 -04:00
Guo Xiang Tan
a1d135f12a
DEV: Correct use of sanitize_sql_array in TopicQuery. 2020-09-08 12:30:09 +08:00
Guo Xiang Tan
d3ebaa41ce
DEV: Address review comments for 5ed84d9885. 2020-09-08 11:17:35 +08:00
Guo Xiang Tan
5ed84d9885
SECURITY: Don't allow moderators to list PMs of all groups.
* Also return 404 when a user is trying to list PMs of a group that
cannot be accessed by the user.
2020-09-08 10:37:00 +08:00
Guo Xiang Tan
0d3239bf21
Revert "SECURITY: Don't allow moderators to view the admins inbox"
Superseeded by d9a5280f5665d12bf46efd8cdcc6200da2cdedd8

This reverts commit 18d35bf64a.
2020-09-08 10:36:49 +08:00
Daniel Waterworth
18d35bf64a SECURITY: Don't allow moderators to view the admins inbox 2020-09-07 18:02:41 +01:00
Martin Brennan
431bd84dec
FIX: Make deleted topic post bookmarks more resilient (#10619)
This PR ensures that new bookmarks cannot be created for deleted posts and topics, and also makes sure that if a bookmark was created and then the topic deleted that the show topic page does not error from trying to retrieve the bookmark reminder at.
2020-09-07 14:52:14 +10:00
Jordan Vidrine
e73ff68f75
FEATURE: Add css classes to stylesheet link elements (#10598)
Adding these classes to the stylesheet link elements in order to toggle dark/light schemes via this theme-component. Eventually this theme-component could possible be merged into core.
2020-09-04 15:40:40 -05:00
Roman Rizzi
726bae8f0c
DEV: Pass the for_export flag to the before_upload_creation event (#10599) 2020-09-04 17:24:40 -03:00
Bianca Nenciu
58b97ace23
DEV: Use a special import to declare font faces (#10583)
Update discourse-fonts to v0.0.3.

Follow-up to 7b7357147e.
2020-09-04 16:25:50 +03:00
Penar Musaraj
7a0d1cadc5
DEV: Add option to precompile default multisite stylesheets 2020-09-03 13:00:09 -04:00
Robin Ward
c4079780be REFACTOR: Remove _.merge 2020-09-03 10:03:33 -04:00
Daniel Waterworth
5bb30d8016 FIX: whoops, linting 2020-09-03 12:10:11 +01:00
Daniel Waterworth
6518cac5cf DEV: Add log rake task
Now, if you do `rake log db:migrate`, it'll run migrations, but with
logging enabled, so that you can see what is actually happening.
2020-09-03 11:43:30 +01:00
Joffrey JAFFEUX
02f0637892
FIX: updates highlightjs to latest version to avoid a freeze (#10578)
Example repro of the freeze:

```
```css
[
```
2020-09-02 15:14:48 +02:00
Dan Ungureanu
38c9c87128
FIX: Add to tags result set only visible tags (#10580) 2020-09-02 13:24:40 +03:00
Jeff Wong
d49e96c6a3
DEV: add plugin hooks for silence message parameters (#10538)
DEV: add plugin hooks for silence message parameters

Allows plugins to add, and update extra silence message params for custom
i18n vars

Allows plugins to override system messages via `message_title` and
`message_raw` parameters. We can later expose these params where necessary via event
hooks. Expose the parameter for the on user_silenced trigger.
2020-09-01 17:25:24 -07:00
Vinoth Kannan
3b55de90e5 FIX: skip pm view action log while generating webhook payload.
Currently, while generating webhook payloads for a topic it's accidentally adding a personal message view log in 'system' user's history.
2020-09-02 05:40:42 +05:30
Penar Musaraj
899b841554
DEV: Faster SCSS refresh locally for plugins
Only updates the targeted stylesheet when available
2020-09-01 13:56:40 -04:00
Jay Pfaffman
5e66d7d082
FIX: emails.rake use authentication (#10572)
It looks like support got added for DISCOURSE_SMTP_AUTHENTICATION in Discourse, but not in this rake task (or perhaps it's always been broken?).

N.B.: I have not tested this code, but it reportedly works and looks safe to me.

https://meta.discourse.org/t/smtp-unrecognized-authentication-type-with-office-365/145830/6?u=pfaffman
2020-09-01 11:08:18 -04:00
Daniel Waterworth
a3580fa292 DEV: Show plugin versions in CI 2020-09-01 16:05:07 +01:00
Blake Erickson
43ffd4d28f
DEV: Switch our fast_xor gem for xorcist (#10565)
* DEV: Switch our fast_xor gem for xorcist

We use the `xor` function as part of password hashing and we want to use
a faster version than the native ruby xor'ing feature so we use a gem
for this.

fast_xor has been abandoned, and xorcist fixed our initial holdup for
switching in https://github.com/fny/xorcist/issues/4

xorcist also has jruby support so we can remove our jruby fallback
logic.

* Move using statement inside of class
2020-08-31 13:20:44 -06:00
Bianca Nenciu
f2e14a3946
FEATURE: Add site setting and wizard step to set base font (#10250)
Co-authored-by: Neil Lalonde <neillalonde@gmail.com>
2020-08-31 13:14:09 +03:00
Jarek Radosz
7290ca1f4a DEV: Remove jquery.cookie
A follow up to c172f2068d

cc: @eviltrout
2020-08-31 11:53:01 +02:00
Faizaan Gagan
73c750a1db
FEATURE: allow specifying platform name for platform specific gems (#10559) 2020-08-31 12:36:04 +10:00
Krzysztof Kotlarek
94152e4640
FIX: dont error when bookmark topic is nil (#10555) 2020-08-31 09:15:36 +10:00
Gerhard Schlager
ce1620f2ad
FIX: Pluralized translation overrides didn't work for en_US
"en_US" doesn't contain most of the translations, so it falls back to "en". But that behavior stopped translation overrides to work for pluralized strings in "en_US", because it relies on existing translations. This fixes it by looking up the existing translation in all fallback locales.
2020-08-29 00:11:46 +02:00
jbrw
e1896f7e71
FIX - rails_route_from_url would fail if path contained unicode characters 2020-08-28 15:10:10 -04:00
Sam Saffron
80e17f5f9e
DEV: Remove staff bypass on fix missing
After thinking about it, I worry that this will potentially leave a site
setting set when people hit ctrl-c ... feels a tiny bit risky, so leaving
it out.
2020-08-28 12:35:35 +10:00
Sam Saffron
ed9323043f
DEV: Allow all uploads when fixing missing s3
If we do not do this we can not properly re-download files
2020-08-28 12:28:59 +10:00
Martin Brennan
2352f4bfc7
DEV: Replace SECURE_MEDIA_ROUTE const with other methods (#10545)
This is so if the route changes this const won't be around to bite us, use the Rails route methods instead.
2020-08-28 11:28:11 +10:00
David Taylor
a3577435f7
FEATURE: Additional control of iframes in oneboxes (#10523)
This commit adds a new site setting "allowed_onebox_iframes". By default, all onebox iframes are allowed. When the list of domains is restricted, Onebox will automatically skip engines which require those domains, and use a fallback engine.
2020-08-27 20:12:13 +01:00
Gerhard Schlager
c0cd69d280
DEV: Restoring backups didn't work on macOS
This is a follow-up for f51ccea028 because specs were failing on macOS. macOS uses bsdtar by default and the --transform option isn't supported by bsdtar. But the -s option is and it works the same.
2020-08-27 19:08:47 +02:00
Jarek Radosz
188d224303
DEV: Restore UploadRecovery specs (#10543)
Originally disabled in 0c0192e. Upload specs now use separate paths for each spec worker.

Fixes an issue in UploadRecovery#recover_from_local – it didn't take into account the testing infix (e.g. test_0) in the uploads/tombstone paths.
2020-08-27 15:57:10 +02:00
Guo Xiang Tan
40c6d90df3 PERF: Create a partial regular post_search_data index on large sites.
With the addition of `PostSearchData#private_message`, a partial
index consisting of only search data from regular posts can be created.
The partial index helps to speed up searches on large sites since PG
will not have to do an index scan on the entire search data index which
has shown to be a bottle neck.
2020-08-27 13:42:00 +08:00
Sam Saffron
f7314f8ab4
DEV: Improve s3 upload image analysis
- Introduces uploads:delete_missing_s3 which can be used to "give up" and
delete broken records from the database

- Fixes a bug in fix_missing_s3 - crashing on deleted posts

- Adds more info to analyze_missing_s3
2020-08-27 11:50:07 +10:00
Vinoth Kannan
618a7ecb35 FIX: default_tags_muted site setting won't have tag ids.
Instead it only have list of tag names separated by comma.
89fcb75af2
2020-08-26 23:05:29 +05:30
Sam Saffron
b60c39fdf0
DEV: search more carefully for missing uploads
rake uploads:analyze_missing_s3 was not looking at all places, amend it so
it looks in all the places where uploads could exist.
2020-08-26 17:48:42 +10:00
Robin Ward
2523a21fc3 FIX: Server side rendering was broken with new helperContext() 2020-08-25 09:16:32 -04:00
siriwatknp
1a2800ad07 fix: 🐛 category & tag search regex to support thai character 2020-08-25 16:12:26 +08:00
David Taylor
629ee5494d
FEATURE: Allow plugins to register parameter-based API routes (#10505)
Example usage:

```
add_api_parameter_route(
  method: :get,
  route: "users#bookmarks",
  format: :ics
)
```
2020-08-24 10:24:52 +01:00
Martin Brennan
e8a842ab8c
FIX: Stop category logo + background being marked secure (#10513)
Meta topic: https://meta.discourse.org/t/secure-media-uploads-breaks-category-logos/161693

Category backgrounds and logos are public uploads and should not be marked as secure.

I also discovered that a lot of the UploadSecurity specs for public types were returning false positives; this has been fixed.
2020-08-24 17:12:28 +10:00
Guo Xiang Tan
05174df5c0
FIX: Restrict personal_messages: advanced search filter to admin.
The filter noops if an incorrect username is passed. This filter is not
exposed as part of the UI but is only used when an admin transitions
from a search within a user's personal messages to the full page search.

Follow-up to 4b30799054.
2020-08-24 13:53:48 +08:00
Guo Xiang Tan
c6ceda8c4e
PERF: Avoid extra subquery when searching within PMs for normal user.
Note the following query being generated where the filter for a user's
private messages is executed twice.

```sql
SELECT "posts"."id", "posts"."user_id", "posts"."topic_id", "posts"."post_number", "posts"."raw", "posts"."cooked", "posts"."created_at", "posts"."updated_at", "posts"."reply_to_post_number", "posts"."reply_count", "posts"."quote_count", "posts"."deleted_at", "posts"."off_topic_count", "posts"."like_count", "posts"."incoming_link_count", "posts"."bookmark_count", "posts"."score", "posts"."reads", "posts"."post_type", "posts"."sort_order", "posts"."last_editor_id", "posts"."hidden", "posts"."hidden_reason_id", "posts"."notify_moderators_count", "posts"."spam_count", "posts"."illegal_count", "posts"."inappropriate_count", "posts"."last_version_at", "posts"."user_deleted", "posts"."reply_to_user_id", "posts"."percent_rank", "posts"."notify_user_count", "posts"."like_score", "posts"."deleted_by_id", "posts"."edit_reason", "posts"."word_count", "posts"."version", "posts"."cook_method", "posts"."wiki", "posts"."baked_at", "posts"."baked_version", "posts"."hidden_at", "posts"."self_edits", "posts"."reply_quoted", "posts"."via_email", "posts"."raw_email", "posts"."public_version", "posts"."action_code", "posts"."locked_by_id", "posts"."image_upload_id", (TS_RANK_CD(
  post_search_data.search_data,
  TO_TSQUERY('english', '''test'':*ABCD'),
  0|32
)
 * (
  CASE categories.search_priority
  WHEN 2
  THEN 0.6
  WHEN 3
  THEN 0.8
  WHEN 4
  THEN 1.2
  WHEN 5
  THEN 1.4
  ELSE
    CASE WHEN topics.closed
    THEN 0.9
    ELSE 1
    END
  END
)
) rank, topics.bumped_at topic_bumped_at FROM "posts" INNER JOIN "post_search_data" ON "post_search_data"."post_id" = "posts"."id" INNER JOIN "topics" ON "topics"."id" = "posts"."topic_id" AND ("topics"."deleted_at" IS NULL) LEFT JOIN categories ON categories.id = topics.category_id WHERE ("posts"."deleted_at" IS NULL) AND "posts"."post_type" IN (1, 2, 3) AND (topics.visible) AND (topics.archetype = 'private_message' AND post_search_data.private_message) AND (posts.topic_id IN (SELECT topic_id
FROM topic_allowed_users
WHERE user_id = 99999
UNION ALL
SELECT tg.topic_id
FROM topic_allowed_groups tg
JOIN group_users gu ON gu.user_id = 99999 AND gu.group_id = tg.group_id
)) AND (post_search_data.search_data @@ TO_TSQUERY('english', '''test'':*ABCD')) AND (posts.topic_id IN (SELECT topic_id
FROM topic_allowed_users
WHERE user_id = 99999
UNION ALL
SELECT tg.topic_id
FROM topic_allowed_groups tg
JOIN group_users gu ON gu.user_id = 99999 AND gu.group_id = tg.group_id
)) AND ((categories.id IS NULL) OR (NOT categories.read_restricted) OR (categories.id IN (999999))) ORDER BY rank DESC, topic_bumped_at DESC
```
2020-08-24 13:49:43 +08:00
Guo Xiang Tan
2f043dc89a
Fix lint. 2020-08-24 12:38:46 +08:00
Guo Xiang Tan
4b30799054
FIX: Correct personal_messages:<username> advanced search filter.
Renamed from `private_messages` to `personal_messages` without
deprecation because the `private_messages` advanced search filter never
worked in the first place when it was implemented.
2020-08-24 11:54:30 +08:00
Gerhard Schlager
f51ccea028
FIX: Backups should use relative paths for local uploads
This also ensures that restoring a backup works when it was created with the wrong upload paths in the time between ab4c0a4970 (shortly after v2.6.0.beta1) and this fix.
2020-08-21 15:22:28 +02:00
Guo Xiang Tan
106a2f58a2
DEV: Drop support for deprecated in:private search filter. 2020-08-21 17:18:39 +08:00
Guo Xiang Tan
0684118008
DEV: Remove array_agg from search orders that does not need it. 2020-08-21 14:39:07 +08:00
Neil Lalonde
713e6f1bb6
Version bump to v2.6.0.beta2 2020-08-20 16:36:25 -04:00
Gerhard Schlager
208d85aaff DEV: Ensure prettier uses the same patterns everywhere 2020-08-20 16:27:32 +02:00
Gerhard Schlager
11647b79f7 FIX: SiteSettings::LocalProcessProvider didn't work on multisite
It always used "test" as current site.
2020-08-20 11:15:20 +02:00
Vinoth Kannan
89fcb75af2 FIX: default_tags_muted setting should work for anonymous users too. 2020-08-20 10:40:03 +05:30
Blake Erickson
4395e4d165 FIX: Sending a PM through a flag on a deleted post
Because we allow all the other flag types on a deleted post we should be
able to send a pm to the user letting them know why we deleted their
post.

Bug report:

https://meta.discourse.org/t/-/161156
2020-08-19 17:36:52 -06:00
Vinoth Kannan
8348a41124
FEATURE: add regular_categories field in site setting & user option. (#10477)
Like "default watching" and "default tracking" categories option now the "regular" categories support is added. It will be useful for sites that are muted by default. The user option will be displayed only if `mute_all_categories_by_default` site setting is enabled.
2020-08-20 00:35:04 +05:30
Mark VanLandingham
a3c0d4a8b5
FIX: Error message when setting enforce 2fa with social logins (#10479) 2020-08-19 13:16:31 -05:00
Blake Erickson
367de2594d
FIX: Unlike own posts on ownership transfer (#10446)
* FIX: Unlike own posts on ownership transfer

If a user has liked a post that has passed the
`post_undo_action_window_mins` system setting window and you transfer ownership
of that post to that user you will be the owner of a post that you have
liked, but cannot unlike resulting in a weird UI behavior. This commit
fixes this issue.

The existing tests didn't check for the timeout window for unliking
posts so I added that in.

I couldn't find a good way to do this logic inside of the guardian class
so rather than duplicating behavior of the `PostActionDestroyer` class
inside of the `PostOwnerChanger` I decided to pass in a "bypass"
variable that could be used to check if the calling class is the
'post_owner_changer' and bypass the guardian instead. I went this route
because the guardian `can_delete_post_action` method has no way of
distinguishing how to allow a user to be able to unlike their own posts
after the timeout window but only on a post owner change.

* use an options hash instead
2020-08-19 09:21:02 -06:00
jbrw
aa1fc01307
FEATURE - Moderators can create and manage groups (#10432)
Enabling the moderators_manage_categories_and_groups site setting will allow moderator users to create/manage groups.

* show New Group form to moderators

* Allow moderators to update groups and read logs, where appropriate

* Rename site setting from create -> manage

* improved tests

* Migration should rename old log entries

* Log group changes, even if those changes mean you can no longer see the group

* Slight reshuffle

* RouteTo /g if they no longer have permissions to view group
2020-08-19 10:41:40 -04:00
Penar Musaraj
882b0aac19
DEV: Let themes extend color definitions (#10429)
Themes can now declare custom colors that get compiled in core's color definitions stylesheet, thus allowing themes to better support dark/light color schemes. 

For example, if you need your theme to use tertiary for an element in a light color scheme and quaternary in a dark scheme, you can add the following SCSS to your theme's `color_definitions.scss` file: 

```
:root {
  --mytheme-tertiary-or-quaternary: #{dark-light-choose($tertiary, $quaternary)};
}
```

And then use the `--mytheme-tertiary-or-quaternary` variable as the color property of that element. You can also use this file to add color variables that use SCSS color transformation functions (lighten, darken, saturate, etc.) without compromising your theme's compatibility with different color schemes.
2020-08-18 13:02:13 -04:00
Sam Saffron
2a7490149c
DEV: don't fail if in uploads:fix_missing_s3 when fix fails
Previously a single error on a file like invalid extension could fail the
entire rake task
2020-08-18 17:55:49 +10:00
Guo Xiang Tan
92b7fe4c62
PERF: Add partial index for non-pm search. 2020-08-18 15:55:08 +08:00
David Taylor
33c21c5921
DEV: Skip readonly tables when remapping content (#10461) 2020-08-18 08:53:12 +01:00
Sam Saffron
24fe08230f
FEATURE: ensure posts are rebaked when missing is fixed
This ensures any corrupt optimized images are removed and re-created
2020-08-18 15:37:24 +10:00
David Taylor
645b65adfa
FIX: Add playsinline to twitter GIFs 2020-08-18 00:26:51 +01:00
David Taylor
eb981b8fbb
FEATURE: Autoplay oneboxed twitter GIF media 2020-08-17 23:56:41 +01:00
David Taylor
a895e4a15b
FIX: Skip rendering twitter video if matching format not found
Followup to d5a4318a
2020-08-17 20:53:14 +01:00
David Taylor
d5a4318ac1
FIX: Replace iframe with <video> for twitter videos
This avoids the samesite cookie related error on chrome. It also adds support for twitter 'GIF' content, and allows videos to resize smoothly for narrow devices.
2020-08-17 20:37:36 +01:00
jahan-ggn
65649eaef0
User card settings (#10302)
* settings implemented

* prettier

* settings updated

* rubocop

* prettier

* Revert "rubocop"

This reverts commit 7805145a7d.

* Revert "prettier"

This reverts commit 2c53f4fa12.

* settings updated and changed

* rubocop

* changes applied

* final changes done

* Server side feature added

* spec changed

* changed user_updater and profile file

* Fix user card specs

* web hook serializer solved

* site-setting changed

Co-authored-by: Mark VanLandingham <markvanlan@gmail.com>
2020-08-17 12:37:45 -04:00
David Taylor
0a5376084d
FIX: Ensure auto close notice is posted with system locale
Previously it was created with the locale of the user who created the final post in the topic
2020-08-17 15:40:47 +01:00
Guo Xiang Tan
248bebb8cd
PERF: Remove extra subquery in search.
I also noticed that removing the subquery helps the planner to plan
better.
2020-08-17 13:52:12 +08:00
David Taylor
8ac85f54fb
DEV: Remove blob: workers from CSP (#10440)
Ace editor is reconfigured to load workers directly from their JS URL. Workers must be on the same origin as the site, so they will not use the CDN.
2020-08-14 18:15:30 +01:00
Vinoth Kannan
476d26159a
FEATURE: add new user option skip_new_user_tips. (#10437)
And add new site setting `default_other_skip_new_user_tips` in user preferences category.
2020-08-14 19:10:56 +05:30
David Taylor
3cf93e9a8f
Revert "DEV: Disable ACE editor worker blobs"
This reverts commit d5463d2a4d.

With S3 assets, CORS prevents loading worker assets directly.
2020-08-14 14:12:11 +01:00
David Taylor
a5608025aa
FIX: Add script asset locations to worker-src CSP directives
We no longer need :blob worker src since d5463d2a. But we do want to allow workers to be loaded from all our existing script-src options.
2020-08-14 12:31:50 +01:00
Martin Brennan
4670b62969
DEV: IMAP log to database (#10435)
Convert all IMAP logging to write to a database table for easier inspection. These logs are cleaned up daily if they are > 5 days old.

Logs can easily be watched in dev by setting DISCOURSE_DEV_LOG_LEVEL=\"debug\" and running tail -f development.log | grep IMAP
2020-08-14 12:01:31 +10:00
Penar Musaraj
eae8b0465c
DEV: Upgrade ACE Editor to 1.4.12, remove extra ACE files (#10431) 2020-08-13 21:53:27 -04:00
Martin Brennan
ffb31b8d2b
FIX: Do not require tagging to be enabled for IMAP archive and delete (#10426)
Previously we did an early return if either SiteSetting.tagging_enabled or SiteSetting.allow_staff_to_tag_pms was false when updating the email on the IMAP server -- however this also stopped us from archiving or deleting emails if either of these were disabled.
2020-08-13 14:04:40 +10:00
Sam Saffron
5be26c7d24
DEV: add error handling in case download fails 2020-08-13 13:48:23 +10:00
Jeff Wong
787ad7d84d FIX: allow plugin pinning to fetch missing commits
Add update for fetching git commits if they do not exist, eg with
clone --depth 1 - only can fetch via git fetch --depth 1 {remote} {ref}
the ref needs to be a full, non-ambiguous reference.
2020-08-12 20:03:37 -07:00
Sam Saffron
5011435ec7
DEV: do not correct sha when correctly uploads 2020-08-13 11:52:57 +10:00
Sam Saffron
f48fa30ecd
DEV: fix_missing_s3 attempts to re-download if missing
unverified uploads get re-downloaded and corrected if they exist in the task
2020-08-13 11:22:14 +10:00
David Taylor
bd0a7553c4
DEV: Detect when s3 inventory failure is caused by etag difference (#10427) 2020-08-13 09:30:28 +10:00
David Taylor
451b9b245f
DEV: Rename new upload rake tasks
These tasks are s3-specific, so update the names to make that very clear
2020-08-12 23:26:13 +01:00
Michael Brown
c25dc43f54 FIX: AWS S3 errors don't necessarily include a message
* If the error doesn't have a message, the class name will help
* example:
  before: "Failed to download #{filename} because "
  after: "Failed to download #{filename} because Aws::S3::Errors::NotFound"
2020-08-12 17:00:09 -04:00
Rafael dos Santos Silva
28669dfeb2
PERF: Faster TL3 promotion replies needed calculation (#10416)
Removing the LIMIT makes PostgreSQL use index_posts_on_user_id_and_created_at
which is much faster overall.

Before: 22 seconds
After: 100 ms
2020-08-12 11:28:34 -03:00
Penar Musaraj
6dd9f2eca2
FIX: color scheme selection with non-default theme
This fixes an issue where a non-default theme set to use the base color
scheme (i.e. the theme had an empty `color_scheme_id`) was loading the
default theme's color scheme instead.
2020-08-12 08:49:13 -04:00
Guo Xiang Tan
93f8396b4b
FIX: Limit PG headline based search blurb generation to 200 characters.
* Recovers omission characters '...' in blurb as well.
2020-08-12 15:34:27 +08:00
Sam Saffron
ec173a72d9
DEV: add more counts to analyze missing 2020-08-12 17:28:58 +10:00
Sam Saffron
97fe68e980
DEV: look for avatars when analyzing missing uploads 2020-08-12 14:04:21 +10:00
Sam Saffron
990cd8c2e4
FEATURE: introduce tasks for dealing with legacy broken uploads
`rake uploads:analyze_missing` can be used get rich information regarding
uploads missing from s3 (where verified is false)

`rake uploads:fix_missing` is a work in progress task for automatically
correcting certain historic issues. At the moment it simply rebakes all
posts with missing uploads, but it will improve over time
2020-08-12 13:33:04 +10:00
Martin Brennan
95b71b35d6
FEATURE: IMAP delete email sync for group inboxes (#10392)
Adds functionality to reflect topic delete in Discourse to IMAP inbox (Gmail only for now) and reflecting Gmail deletes in Discourse.

Adding lots of tests, various refactors and code improvements.

When Discourse topic is destroyed in PostDestroyer mark the topic incoming email as imap_sync: true, and do the opposite when post is recovered.
2020-08-12 10:16:26 +10:00
Penar Musaraj
c05aced094
FIX: Invalidate cache when updating color scheme colors (#10417) 2020-08-11 16:28:59 -04:00
Martin Brennan
b950b3fb3f
DEV: Add verified to uploads and fill in S3 inventory (#10406)
When we run the S3 inventory, mark uploads that exist as verified true, those that don't as verified false, and uploads not included in the check / not yet checked as verified nil.
2020-08-11 14:43:51 +10:00
Blake Erickson
2032c11f78
FIX: Return 422 when creating topics with tags w/out permission (#10400)
The UI prevents users from trying to create tags on topics when they
don't have permission, but if you are trying to add tags to a topic via
the API and you don't have permission before this change it would
silently succeed in creating the topic, but it wouldn't have any tags.

Now a 422 error will be returned with an error message when trying to
create a topic with tags when tagging is disabled or you don't have
enough trust level to add tags to a topic.

Bug report: https://meta.discourse.org/t/-/70525/14
2020-08-10 16:14:15 -06:00
jbrw
d67f7a7984
Allows mod posts to be created for category group moderators on closed/archived topics (#10399) 2020-08-10 15:21:01 -04:00
David Taylor
4476680d4b
UX: Set silence_reason using the system locale
Previously we would use the user's locale, which can be confusing for moderators.
2020-08-10 18:51:54 +01:00
Penar Musaraj
f179510a68
FIX: include both name and id in color scheme stylesheet filename slugs (#10397) 2020-08-07 13:43:45 -04:00