Commit Graph
100 Commits
Author SHA1 Message Date
Blake Erickson 53d861414d DEV: Fix group create response api response (#17711) 2022-07-28 16:49:33 +01:00
Blake Erickson 8b08b9a763 FIX: Rejected emails should not be cleaned up before their logs (#17648)
* FIX: Rejected emails should not be cleaned up before their logs

If we delete the rejected emails before we delete their associated logs
we will receive 404 errors trying to inspect an email message for that
log.

* don't add a blank line

* test for max value as well

* pr cleanup and add migration

* Fix failing test
2022-07-27 07:28:44 +01:00
Blake Erickson 8626c99a7d DEV: Tidy up create group api endpoint docs (#17586)
Switch to using json schema files for describing the requests and
responses.
2022-07-20 19:18:48 -06:00
Blake Erickson 59d514df34 DEV: Fix missing optional site.json fields (#17364)
The site.json endpoint was missing some optional fields on the
categories property that is returned. This commit documents the
`parent_category_id` which is present if there are subcategories and it
also documents the `custom_fields` property which the chat plugin is
using causing some flaky tests.
2022-07-08 09:04:50 +08:00
Blake Erickson 2861b9337d DEV: Fix flaky admin badges.json api docs spec (#17210)
* DEV: Fix flaky admin badges.json api docs spec

This commit is to fix this incredibly vague error message:

```
Failure/Error: expect(valid).to eq(true)

  expected: true
       got: false
```

From this test:

> Assertion: badges /admin/badges.json get success response behaves like
> a JSON endpoint response body matches the documented response schema

I was finally able to repro locally using parallel tests:

```
RAILS_ENV=test bundle exec ./bin/turbo_rspec
```

I *think* the parallel tests might be swallowing the `puts` output, but
when I also specified the individual spec file

```
RAILS_ENV=test bundle exec ./bin/turbo_rspec spec/requests/api/badges_spec.rb
```

It revealed the issue:

```
VALIDATION DETAILS: {"missing_keys"=>["i18n_name"]}
```

``` ruby
...
  def include_i18n_name?
    object.system?
  end
```

Looks like if the "system" user isn't being used the `i18n_name` won't
be returned in the json response so we shouldn't mark it as a required
attribute.

* Switch to using fab!

When using `let(:badge)` to fabricate a test badge it wouldn't be
returned from the controller, but switching to using `fab!` allows it to
be returned in the json data giving us a non-system badge to test
against.
2022-06-23 14:32:17 -06:00
Blake Erickson a5552321cf DEV: Fix flakey site.json rswag specs (#17161)
The category description fields as part of the rswag specs for the
site.json endpoint were flakey. Removing the `required` attribute allows
us to still document that these fields exists, but that depending on
certain site settings they may not be present in the response.
2022-06-20 12:33:33 -06:00
Blake Erickson f313276966 Version bump to v2.9.0.beta5 (#17077) 2022-06-13 15:27:06 -06:00
Blake Erickson ae6a907943 SECURITY: banner-info (#17071) 2022-06-13 11:10:21 -06:00
Blake Erickson 852a2f1727 DEV: Add spec for not_staged user scope (#17042)
Making sure to test this new scope.

Follow up to: 27d7b0c6de

as well as: 3941bad075
2022-06-08 10:43:21 -06:00
Blake Erickson 3941bad075 DEV: Remove this chained user scope (#17029)
This reverts one of the changes introduced just now in:

27d7b0c6de

I don't think we need this `activated_not_suspended_not_staged` scope
because we can just compose it ourselves via method chaining like
`User.activated.not_suspended.not_staged`.
2022-06-07 15:21:42 -06:00
Blake Erickson 27d7b0c6de DEV: Add new user scopes (#17026)
Adds two new user scopes:

- `not_staged`
- `activated_not_suspended_not_staged`

This will allow us to easily grab activated users that are not suspended
or staged.

See this PR feedback:

https://github.com/discourse/discourse-chat/pull/913#discussion_r890692266
2022-06-07 12:58:58 -06:00
Blake Erickson 35e17ce115 FIX: Email Send post has already been taken error (#16992)
* FIX: Email Send post has already been taken error

Adding a failing test first before coming up with a good solution.

Related: 357011eb3b

The above commit changed

```
PostReplyKey.find_or_create_by_safe!
```

to

```
PostReplyKey.create_or_find_by!
```

But I don't think it is working as a 1-1 replacement because of the
`Validation failed: Post has already been taken` error we are receiving
with this change. Also we need to make sure we don't re-introduce any
concurrency issues.

Reported: https://meta.discourse.org/t/224706/13

* Remove rails unique constraint and rely on db index

I believe this is what is causing `create_or_find_by!` to fail. Because
we have a unique constraint in the db I think we can remove this rails
unique constraint?

* clean up spec wording
2022-06-06 13:13:26 -06:00
Blake Erickson 4c0a9fd4c3 FIX: Email logs not finding attached incoming email (#16446) 2022-04-11 21:22:15 -06:00
Blake Erickson ec2930712d FIX: 500 error when creating a user with an integer username (#16370)
Via the API it is possible to create a user with an integer username. So
123 instead of "123". This causes the following 500 error:

```
NoMethodError (undefined method `unicode_normalize' for 1:Integer)
app/models/user.rb:276:in `normalize_username'
```

See: https://meta.discourse.org/t/222281
2022-04-04 15:15:32 -06:00
Blake Erickson 61248652cd DEV: Allow params to be passed on topic redirects (#16218)
* DEV: Allow params to be passed on topic redirects

There are several places where we redirect a url to a standard topic url
like `/t/:slug/:topic_id` but we weren't always passing query parameters
to the new url.

This change allows a few more query params to be included on the
redirect. The new params that are permitted are page, print, and
filter_top_level_replies. Any new params will need to be specified.

This also prevents the odd trailing empty page param that would
sometimes appear on a redirect. `/t/:slug/:id.json?page=`

* rubocop: fix missing space after comma

* fix another page= reference
2022-03-17 19:27:51 -06:00
Blake Erickson 02fa04e333 FIX: Update topic route id param (#16166)
This update topic route has never worked. Better late than never. I am
in favor of using non-slug urls when using the api so I do think we
should fix this route.

Just thought I would update the `:id` param to `:topic_id` here in the
routes file instead of updating the controller to handle both params.

Added a spec to test this route.

Also added the same constraint we have on other topic routes to ensure
we only pass in an ID that is a digit.
2022-03-11 11:01:08 -07:00
Blake Erickson d760fd4074 DEV: Add API scope for categories endpoint (#16105)
This change adds support for the categories endpoint to have an api
scope. Only adds GET scope for listing categories and for fetching a
single category.

See: https://meta.discourse.org/t/218080/4
2022-03-04 14:29:47 -07:00
Blake Erickson 07e80b52ef DEV: Add api scoping for updating topics. (#16101)
This change adds a new api scope for updating topics.

See: https://meta.discourse.org/t/219805
2022-03-04 10:44:56 -07:00
Blake Erickson df2441ee37 Top level replies (#16087)
* DEV: Show only top level replies

Adds a new query param to the topic view so that we can filter out posts
that aren't top level replies. If a post is a reply to another post
instead of the original topic post we should not include it in the
response if the `filter_top_level_replies` query param is present.

* add rspec test
2022-03-02 13:25:36 -07:00
Blake Erickson febc2361bd DEV: Document external topic id endpoints (#15897)
* DEV: Document external topic id endpoints

This commit documents the existing Create Topic endpoint with the
`external_id` param and documents the new get topic by external id
endpoint.

It also refactors the existing topic show endpoint to use the new format
where we load the expected json schema response from a file.

See: 71f7f7ed49

* clean up unused test variables
2022-02-10 19:01:19 -07:00
Blake Erickson e66241af7f DEV: Update api docs to update username and to get post replies (#15881)
Added api docs for:

 - update username
 - get post replies
2022-02-09 15:04:36 -07:00
Blake EricksonandAlan Guo Xiang Tan 71f7f7ed49 FEATURE: Add external_id to topics (#15825)
* FEATURE: Add external_id to topics

This commit allows for topics to be created and fetched by an
external_id. These changes are API only for now as there aren't any
front changes.

* add annotations

* add external_id to this spec

* Several PR feedback changes

- Add guardian to find topic
- 403 is returned for not found as well now
- add `include_external_id?`
- external_id is now case insensitive
- added test for posts_controller
- added test for topic creator
- created constant for max length
- check that it redirects to the correct path
- restrain external id in routes file

* remove puts

* fix tests

* only check for external_id in webhook if exists

* Update index to exclude external_id if null

* annotate

* Update app/controllers/topics_controller.rb

We need to check whether the topic is present first before passing it to the guardian.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>

* Apply suggestions from code review

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2022-02-08 20:55:32 -07:00
Blake Erickson 4bf6789bd7 DEV: Do not use hard-coded everyone group id (#15679)
Follow up to: 12f041de5d

Probably best to lookup the "everyone" group_id instead of hard-coding
it to `0`. Also now its more clear what this `0` means.
2022-01-21 15:56:45 -07:00
Blake Erickson 12f041de5d FIX: Tag watching for everyone tag groups (#15622)
* FIX: Tag watching for everyone tag groups

Tags in tag groups that have permissions set to everyone were not able
to be saved correctly. A user on their preferences page would mark the
tags that they wanted to save, but the watched_tags in the response
would be empty. This did not apply to admins, just regular users. Even
though the watched tags were being saved in the db, the user serializer
response was filtering them out. When a user refreshed their preferences
pages it would show zero watched tags.

This appears to be a regression introduced by:

0f598ca51e

The issue that needed to be fixed is that we don't track the "everyone"
group (which has an id of 0) in the group_users table. This is because
everyone has access to it, so why fill a row for every single user, that
would be a lot. The fix was to update the query to include tag groups
that had permissions set to the "everyone" group (group_id 0).

I also added another check to the existing spec for updating
watched tags for tags that aren't in a tag group so that it checks the
response body. I then added a new spec which updates watched tags for
tags in a tag group which has permissions set to everyone.

* Resolve failing tests

Improve SQL query syntax for including the "everyone" group with the id
of 0.

This commit also fixes a few failing tests that were introduced. It
turns out that the Fabrication of the Tag Group Permissions was faulty.
What happens when creating the tag groups without any permissions is
that it sets the permission to "everyone". If we then follow up with
fabricating a tag group permission on the tag group instead of having a
single permission it will have 2 (everyone + the group specified)! We
don't want this. To fix it I removed the fabrication of tag group
permissions and just set the permissions directly when creating the tag
group.

* Use response.parsed_body instead of JSON.parse
2022-01-18 15:02:29 -07:00
Blake Erickson a850568a52 DEV: Document delete post API endpoint (#15495)
Adding a spec for documenting the delete post API endpoint for our api
docs. As part of this added detailed info for the `force_destroy`
parameter for permanently deleting a post.
2022-01-07 16:09:32 -07:00
Blake Erickson b705971d42 DEV: Allow for taller images in posts and oneboxes (#15395)
The previous default aspect ratio for cropping tall images was a little
too strict and was cutting off images. This new setting should allow for
a larger range of image sizes before cropping them.
2021-12-22 16:01:34 -07:00
Blake Erickson b93b6c4299 FIX: Blurry onebox favicon images (#15258)
This is a fix to address blurry onebox favicon images if the site you
are linking to happens to have a favicon.ico file that contains multiple
images.

This fix detects of we are trying to create an upload for a favicon.ico
file. We then convert it to a png and not a jpeg like we were doing. We
want a png because it will preserve transparency, otherwise if we
convert it to a jpeg we lose that and it looks bad on dark themed sites.

This fix also addresses the fact that .ico files can include multiple
images. The blurry images we were producing was caused by the
ImageMagick `-flatten` option when the .ico file had multiple images
which then squishes them all together. So for .ico files we are no
longer flattening them and instead we are grabbing the last image in the
.ico bundle and converting that single image to a png.
2021-12-10 12:25:50 -07:00
Blake Erickson 643f82d8d6 DEV: Update email responses in api docs (#15178)
Documenting the `/u/:username:/emails.json` endpoint.

Also removing some email fields from user api responses because they
aren't actually included in the response unless you are querying
yourself.
2021-12-03 08:03:58 -07:00
Blake Erickson da9cd4f692 DEV: Swap out optipng with oxipng (#15013)
* DEV: Swap out optipng with oxipng

The oxipng binary has been added to our base docker image here:

https://github.com/discourse/discourse_docker/commit/244c9cb110df44eb9d846a24b5572471a2687071

oxipng is a rust replacement for optipng that provides increased
performance and multi-threading. Checkout
https://github.com/shssoichiro/oxipng for more info.

* add instructions for installing oxipng
2021-11-22 10:16:35 -07:00
Blake Erickson 892e33fd93 Add embed_url to the api docs (#14813)
When creating a topic via the api you can pass in the `embed_url` param,
so adding this to the api docs.

See: https://github.com/discourse/discourse_api_docs/pull/26
2021-11-03 19:22:55 -06:00
Blake Erickson 2fb9834821 DEV: Fix api docs for default calendar (#14539)
Change the type for default_calendar to a string.

The type specified for the default calendar in the api docs wasn't a
valid type. The linting in the api docs repo reports:

```
`type` can be one of the following only: "object", "array", "string", "number", "integer", "boolean", "null".
```

This linting currently is only in the `discourse_api_docs` repo.
2021-10-06 13:36:11 -06:00
Blake Erickson aaf7ac8936 DEV: Add include_subcategories param to api docs (#14534)
* DEV: Add include_subcategories param to api docs

Adding the `include_subcategories=true` query param to the
`/categories.json` api docs.

Follow up to: fe676f334a

* fix spec
2021-10-06 12:34:03 -06:00
Blake Erickson fe676f334a FEATURE: Return subcategories on categories endpoint (#14492)
* FEATURE: Return subcategories on categories endpoint

When using the API subcategories will now be returned nested inside of
each category response under the `subcategory_list` param. We already
return all the subcategory ids under the `subcategory_ids` param, but
you then would have to make multiple separate API calls to fetch each of
those subcategories. This way you can get **ALL** of the categories
along with their subcategories in a single API response.

The UI will not be affected by this change because you need to pass in
the `include_subcategories=true` param in order for subcategories to be
returned.

In a follow up PR I'll add the API scoping for fetching categories so
that a readonly API key can be used for the `/categories.json` endpoint. This
endpoint should be used instead of the `/site.json` endpoint for
fetching a sites categories and subcategories.

* Update PR based on feedback

- Have spec check for specific subcategory
- Move comparison check out of loop
- Only populate subcategory list if option present
- Remove empty array initialization
- Update api spec to allow null response

* More PR updates based on feedback

- Use a category serializer for the subcategory_list
- Don't include the subcategory_list param if empty
- For the spec check for the subcategory by id
- Fix spec to account for param not present when empty
2021-10-05 12:12:31 -06:00
Blake Erickson 4a4881613b DEV: Refactor the api docs for the user endpoint (#14377)
Due to the way that rswag expands shared components we were getting this
warning when linting our api docs:

```
Component: "user_response" is never used.
```

This change refactors the `api/users_spec.rb` file so that it uses the
new way of doing things with a separate `user_get_response.json` schema
file rather then the old way of loading a shared response inside of the
swagger_helper.rb file.
2021-09-20 10:04:57 -06:00
Blake Erickson 91453dd3fc DEV: Fix flaky site.json api test (#14351)
The color_scheme_id needs to be an integer not a string.

This is one of the failing tests that showed this error:

 https://github.com/discourse/discourse/runs/3598414971

It showed this error

`POSSIBLE ISSUE W/: /user_themes/0/color_scheme_id`

And this is part of the site.json response:

```
...
"user_themes"=>[{"theme_id"=>149, "name"=>"Cool theme 111", "default"=>false, "color_scheme_id"=>37}]
...
```
2021-09-15 18:03:08 -06:00
Blake Erickson c6bcf1f06c DEV: Add site.json to api docs (#14249)
Documenting the site.json api endpoint. This endpoint is often used as a
way to get all of the categories and subcategories in a single api call.
2021-09-07 10:36:05 -06:00
Blake Erickson d500c436c6 DEV: Add license field to api docs info section (#14248)
One of the fields that should be present for openapi docs is the "license"
field.

https://spec.openapis.org/oas/latest.html#infoObject

Our API docs already had a license, so this commit just specifies that
and provides a link to it.
2021-09-07 10:35:56 -06:00
Blake Erickson ee7809e8a8 DEV: Add missing operationIds to the api docs (#14235)
From the openapi spec:

 https://spec.openapis.org/oas/latest.html#fixed-fields-7

each endpoint needs to have an `operationId`:

> Unique string used to identify the operation. The id MUST be unique
> among all operations described in the API. The operationId value is
> case-sensitive. Tools and libraries MAY use the operationId to uniquely
> identify an operation, therefore, it is RECOMMENDED to follow common
> programming naming conventions.

Running the linter on our openapi.json file with this command:

`npx @redocly/openapi-cli lint openapi.json`

produced the following warning on all of our endpoints:

> Operation object should contain `operationId` field

This commit resolves these warnings by adding an operationId field to
each endpoint.
2021-09-03 07:39:29 -06:00
Blake Erickson 0e62602fbf FIX: Use named params correctly with dir-span (#14203) 2021-09-01 07:58:26 -06:00
Blake Erickson a2ccf0a9ff DEV: Use named parameters for dir-span helper (#14195)
* DEV: Use named parameters for dir-span helper

Follow up to: e50a5c0c73

In order to improve code clarity this change introduces named parameters
for the dir-span helper. This is specifically for the new `htmlSafe`
parameter which you can use instead of just passing in a boolean if the
strings you are passing in have already been escaped.

Before: `{{dir-span category.description false}}`
After: `{{dir-span category.description htmlSafe=true}}`

* Set default value for params arg
2021-08-31 13:51:08 +08:00
Blake Erickson 70eca1dc4e DEV: Update api docs for search endpoint (#14181) 2021-08-30 11:25:34 -06:00
Blake Erickson e50a5c0c73 DEV: Add default on encoding to dir-span (#14183) 2021-08-30 11:25:12 -06:00
Blake Erickson 75b0d6df93 SECURITY: escape cat name (#14154) 2021-08-25 17:11:58 -06:00
Blake Erickson b35695e411 DEV: Fix some openapi spec issues (#14037)
- Remove duplicate paths
- Remove query param listed in the path
2021-08-13 04:22:15 -06:00
Blake Erickson 65f6d46045 DEV: Fix several type issues with the api docs (#14016)
`nullable` is no longer a valid type, and types also can't be an empty
string, so just bringing a number of issues with types in compliance
with the openapi spec.
2021-08-12 12:25:17 -06:00
Blake Erickson 1799944a04 DEV: Specify the latest openapi spec version (#14012)
The latest openapi spec version is v3.1.0

 https://spec.openapis.org/oas/v3.1.0

Specifying the latest version will allow our openapi spec linter to use
this version and allow use to use the new type format that allows for
specifying a type as "null", which we need because sometimes our api
responses include null values instead of a "string", "integer", or
"object" type.

See: https://stackoverflow.com/a/48114322/588458
2021-08-11 12:38:02 -06:00
Blake Erickson ce015f5b75 DEV: Fix api docs tagging format (#14010)
When specifying multiple tags they should be separate strings, not a
single string.
2021-08-11 11:00:48 -06:00
Blake Erickson 413de9361f DEV: Disable PATCH requests (#13928)
We aren't actually calling any of the PATCH requests, so maybe we should
just stick with PUT unless we explicitly need PATCH for something.
2021-08-03 07:26:48 -06:00
Blake Erickson 5c6e7e3401 DEV: Document some of the badge api endpoints (#13919)
Adding some api documentation for the badge routes.
2021-08-03 06:25:12 -06:00
Blake Erickson 1c60be7658 DEV: Document anonymize user api endpoint (#13893)
Adding the anonymize user endpoint to the api docs.

See: https://meta.discourse.org/t/158704/7
2021-07-29 17:40:41 -06:00
Blake Erickson 6ac3f1f7b5 DEV: Return 400 instead of 500 for invalid top period (#13828)
* DEV: Return 400 instead of 500 for invalid top period

This change will prevent a fatal 500 error when passing in an invalid
period param value to the `/top` route.

* Check if the method exists first

I couldn't get `ListController.respond_to?` to work, but was still able
to check if the method exists with
`ListController.action_methods.include?`. This way we can avoid relying
on the `NoMethodError` exception which may be raised during the course
of executing the method.

* Just check if the period param value is valid

* Use the new TopTopic.validate_period method
2021-07-23 14:58:10 -06:00
Blake Erickson 448a564845 DEV: Remove super old code comment TODO (#13663)
This TODO comment has existed for 8 years. Sort must be working just
fine or we would have prioritized fixing it.

Removing this comment as a tiny step toward keeping our codebase nice
and tidy.
2021-07-07 14:34:05 -06:00
Blake Erickson fccf4d4375 DEV: Switch off of our image_optim fork (#13124)
The main image_optim gem now includes the timeout feature
that we had in our fork. So it is now safe to switch off of our fork and
back to the image_optim gem.

This is the link to the commit in the image_optim repo that adds the
timeout option:

https://github.com/toy/image_optim/commit/ec3767dde0d9f9f3d18bd688645583d847ba2e13

One difference with the new timeout implementation is that image_optim
now handles the timeout exceptions instead of bubbling them up:

https://github.com/toy/image_optim/blob/1ed03285873078185c129c66007231469461e429/lib/image_optim.rb#L128-L129

```
 rescue Errors::TimeoutExceeded
   handler.result
```

So a timeout will just return `nil`, which is the same response if it
couldn't optimize an image. I don't think we were really watching for
or doing anything about these timeout warnings in our logs so I think
this is an okay change to have and we will have less warnings in our
logs now too.
2021-05-24 19:27:20 -06:00
Blake Erickson 37812b8e35 Revert "DEV: Drop old IE11 intersection-observer references" (#13017)
This reverts commit 7360a0f70f.

iOS still wants this sometimes. Probably best to revert for now and we can
always remove this again later.

See: https://meta.discourse.org/t/189799/11?u=blake
2021-05-10 17:47:09 -06:00
Blake Erickson ee880c4c89 DEV: Extract out compound conditional (#13008) 2021-05-10 11:11:51 -06:00
Blake Erickson 23de5e959e DEV: Remove duplicate test (#12989)
I think this test was duplicated on accident. We test for the
exact thing a few lines up.
2021-05-07 15:34:14 -06:00
Blake Erickson 4b7b586861 DEV: Remove unused rswag script
We no longer use this script for helping generate rswag formatted docs because we can use json schema now.
2021-05-07 09:36:55 +08:00
Blake Erickson 1b02dce594 DEV: Drop old IE11 intersection-observer references (#12942)
It's been awhile since we have supported IE11 so it should be safe to remove
IntersectionObserver now.

From a TODO task in this repo:
> drop when we eventually drop IE11

Announcement of when we removed IE11 support:

https://meta.discourse.org/t/137984/40?u=blake
2021-05-04 17:54:54 -06:00
Blake Erickson 174c68509d DOCS: Update min versions (#12932)
Update the minimum required version for ruby, pg, and redis to match what is listed in the [INSTALL.md](https://github.com/discourse/discourse/blob/master/docs/INSTALL.md) file.
2021-05-03 12:58:34 -06:00
Blake Erickson 0f3b5387ea DEV: Add select-kit helper isDisabled (#12827)
In a qunit test for a plugin I need to be able to check if a select-kit
element is disabled or not. Adding this helper in core allows me to do that.
2021-04-26 14:39:41 -06:00
Blake Erickson ce251986bf DEV: Set min_search_term_length to 1 (#12610)
Japanese languages and the like should have a min_search_term_length of
1.

See: https://meta.discourse.org/t/97359/27

and

https://meta.discourse.org/t/185218/7
2021-04-05 17:37:58 -06:00
Blake Erickson 44153cde18 FIX: Be able to handle long file extensions (#12375)
* FIX: Be able to handle long file extensions

Some applications have really long file extensions, but if we truncate
them weird behavior ensues.

This commit changes the file extension size from 10 characters to 255
characters instead.

See:

https://meta.discourse.org/t/182824

* Keep truncation at 10, but allow uppercase and dashes
2021-03-17 12:01:29 -06:00
Blake Erickson 5c84f702b0 DEV: Update api docs to deprecate target_usernames field (#12394)
When creating a PM target_usernames has been deprecated for some time
now but the api docs have yet to reflect this.

The api docs now specify to use target_recipients.

See: eef21625c6
2021-03-12 16:48:39 -07:00
Blake Erickson 85870225f2 FIX: Allow additional valid theme .git url formats (#12385)
Some git repos have a different ssh url scheme than github and we should
support them.

This change updates our regex format to account for repos that don't
start with "git", but are still valid ssh urls.

Also I added some tests to account for the various formats and to ensure
we don't show the public key when using https urls.

See: https://meta.discourse.org/t/182668
2021-03-12 13:24:55 -07:00
Blake Erickson 2123561125 FIX: Mobile app notification urls w/ subfolder (#12282)
The urls that we generate for mobile post notifications don't take into
account the subfolder url if a site happens to have one configured. When
this happens when you tap on a new mobile notification it takes you to
a url that doesn't work because it is missing the subfolder portion.

I honestly think this should be handled in the Post model like we do
with the Topic model. `Post.url` should know how to handle subfolder
installs, but that seemed like a very risky change because there are
lots of other places in the codebase where we tack on the base_path and
I didn't want to risk duplicating it.

I also found a small typo in the topics controller spec.
2021-03-04 07:07:37 -07:00
Blake Erickson f53546c03d FIX: Exclude users with posts from purge_unactivated query (#12231)
Unactivated users that have posts cannot be deleted so we shouldn't
include them in the initial query to try and purge them. Otherwise we
are just loading up sidekiq with pointless work to be doing every day.

Without this change if there are 201 unactivated users to purge, but the
first 200 have posts, the 201st user will never be deleted even though
it is the only user that doesn't have a post and is actually the one
that should be deleted.
2021-03-01 16:46:28 +11:00
Blake Erickson 0b2b4bc245 DEV: Document the user suspend api endpoint (#12179)
Wrote an api docs rspec test for documenting the user suspend api
endpoint.
2021-02-23 05:58:22 -07:00
Blake Erickson dbcda617b3 DEV: Add a CSV importer for restoring deleted users (#12147)
This is an importer I wrote to restore some users that were
accidentally deleted for being purged as old staged users or old
unactivated users.

It reads from CSV files exported from a discourse sql backup.
2021-02-19 13:46:54 -07:00
Blake Erickson ed0e4582a1 DEV: If disabled do not change setting after import (#12142)
When running an import script there are many site settings that are
changed but we reset them back to where they were originally before the
import. However, there are two settings that we don't roll back:

```
purge_unactivated_users_grace_period_days
purge_deleted_uploads_grace_period_days
```

which could have some unintended consequences.

My first question is do we *really* have to change these settings? I'm
not a huge fan of changing someones settings without them really knowing
they were changed.

If we really do have to change these settings here is my proposed PR
where we don't alter the `purge_unactivated_users_grace_period_days` if
it has been disabled.

As I'm writing this another change we could make is that we don't change
either of these site settings if we detect that they aren't set to the
default values.

The drive behind this PR is that there is a discourse instance which
relies on staged users as part of their workflow and this setting was
changed by accident via the import script causing users to be deleted
that shouldn't have been.
2021-02-19 09:33:35 -07:00
Blake Erickson 395a903cf6 DEV: Show warning message when using ember css selectors (#12036)
* DEV: Show warning message when using ember css selectors

When editing the theme css via the admin UI a warning message
will be displayed if it detects that the `#emberXXX` or `.ember-view`
css selectors are being used. These are dynamic selectors that ember
generates, but they can change so they should not be used.

* Update error message text to be more helpful

* Display a warning instead of erroring out

This allows the theme to still be saved, but a warning is displayed.

Updated the tests to check for the error message.

Updated the pre tags css so that it wraps for long messages.
2021-02-11 13:48:57 -07:00
Blake Erickson 65a0d5bd76 DEV: Update category api docs (#12017)
Updated the category api docs to follow the new validation format.
2021-02-09 06:27:23 -07:00
Blake Erickson 8a001c2fe3 api docs example (#11997)
* DEV: Escape backslashes in curl example

We need to escape these backslashes otherwise they get filtered out when
generating the api docs.

* FIX: uniqItems should be uniqueItems
2021-02-08 10:09:44 -07:00
Blake Erickson 91fb298510 DEV: Add description and logo to api docs (#11984)
* DEV: Add description and logo to api docs

* Rename method to avoid name collision
2021-02-04 18:43:40 -07:00
Blake Erickson 151193bb11 document api endpoints (#11958)
* DEV: Documented several group endpoints

* documented some more endpoints

* document more api endpoints

* Document backup endpoints

* remove puts
2021-02-03 17:12:35 -07:00
Blake Erickson 67e185b33e document user endpoints (#11894)
* document user endpoints, allow for empty request/response bodies

* document more user endpoints, improve debugging output if no details are specified

* document some more user endpoints

* minor cleanup

* FIX: flakey tests due to bad regex
2021-01-29 11:27:11 -07:00
Blake Erickson 798b479e0d DEV: More API Doc improvements (#11849)
- Create helper wrapper method `load_spec_schema(name)`
- A minor change to tag_group_create response schema
- Document the uploads endpoint
2021-01-26 07:38:46 -07:00
Blake Erickson c889b676f8 DEV: Updates to api docs schema validation (#11801)
- Read in schemas from actual json files instead of a ruby hash. This is
helpful because we will be automatically generating .json schema files
from json responses and don't want to manually write ruby hash schema
files.

- Create a helper method for rspec schema validation tests to dry up code
2021-01-21 18:23:23 -07:00
Blake Erickson 7434116933 DEV: Add schema checking to api doc testing (#11721)
* DEV: Add schema checking to api doc testing

This commit improves upon rswag which lacks schema checking. rswag
really only checks that the https status matches, but this change adds
in the json-schema_builder gem which also has schema validation.

Now we can define schemas for each of our requests/responses in the
`spec/requests/api/schemas` directory which will make our documentation
specs a lot cleaner.

If we update a serializer by either adding or removing an attribute the
tests will now fail (this is a good thing!). Also if you change the type
of an attribute say from an array to a string the tests will now fail.
This will help significantly with keeping the docs in sync with actual
code changes! Now if you change how an endpoint will respond you will
have to update the docs too in order for the tests to pass. :D

This PR is inspired by:

 https://www.tealhq.com/post/how-teal-keeps-their-api-tests-and-documentation-in-sync

* Swap out json schema validator gem

Swapped out the outdated json-schema_builder gem with the json_schemer
gem.

* Add validation fields to schema

In order to have "strict" validation we need to add
`additionalProperties: false` to the schema, and we need to specify
which attributes are required.

Updated the debugging test output to print out the error details if
there are any.
2021-01-21 16:28:08 -07:00
Blake Erickson fa4af17580 DEV: Document get user by external_id api endpoint (#11717)
Added GET user by external_id to the api docs.
Fixed `/users/{username}` docs to be `/u/{username}`

Extracted out common user response into a shared helper.
2021-01-14 16:59:58 -07:00
Blake Erickson 06084fb8be FIX: IndexCategoryForSearch category_id job exception (#11239)
On category create an exception will be thrown on this job because the
save transaction hasn't completed yet and the job cannot find the
category id. To prevent this we can use the rails 6 `after_save_commit`
hook that will fire after the category save transaction has finished for
both update and create actions.
2020-11-16 10:18:25 -07:00
Blake Erickson c9515ede2e DEV: Add api docs for invite and group update endpoint (#11019)
Documenting a few more endpoints so that our api docs can be
automatically generated. Made a couple other minor changes, like
including the "OK" example for our default success response.
2020-10-26 08:55:22 -06:00
Blake Erickson cce41f32b8 DEV: Switch over to a fork of ember-rails (#10603)
We are switching over to a fork because we are currently on a pinned
version of ember-rails 0.18.5 which is pretty old. Upgrading to the
latest version causes many things to break which isn't really worth the
time to debug while we plan to completely switch over to ember-cli
somewhat soonish. Our fork contains a single cherry-pick commit

https://github.com/emberjs/ember-rails/pull/534

which will fix an issue when running the `rails g migration` command and
it spits out a bunch of deprecation warnings.
2020-09-08 13:23:40 -06:00
Blake Erickson 4dfdfeaafd DEV: Resolve this child category pending request spec (#10604)
There is a request spec that was ignored with the `xit` flag almost a
year ago and every time you generate the api docs with

```
rake rswag:specs:swaggerize
```

it shows the output of this pending test and I guess I finally got sick
of looking at it, so here is a fix for it.

Original Commit: d84c34ad75
2020-09-08 13:23:11 -06:00
Blake Erickson a14c9078d3 DEV: Document /u/{username}.json API endpoint
Added some more specs that will be used to auto generate the api docs.
2020-09-04 16:24:43 -06:00
Blake Erickson b16915d166 DEV: Switch over to a fork of ember-rails
We are switching over to a fork because we are currently on a pinned
version of ember-rails 0.18.5 which is pretty old. Upgrading to the
latest version causes many things to break which isn't really worth the
time to debug while we plan to completely switch over to ember-cli
somewhat soonish. Our fork contains a single cherry-pick commit

https://github.com/emberjs/ember-rails/pull/534

which will fix an issue when running the `rails g migration` command and
it spits out a bunch of deprecation warnings.
2020-09-04 14:47:56 -06:00
Blake Erickson 67dec38f31 FIX: Gravatar download attempt if user is missing their email
It is possible that a user could exist without an email, if so we should
not enqueue a job to download their gravatar.

This commit resolves this error that can occur:

```
Job exception: undefined method `email' for nil:NilClass
/var/www/discourse/app/models/user.rb:1204:in `email'
/var/www/discourse/app/jobs/regular/update_gravatar.rb:12:in `execute'
```

This commit also fixes the original spec which actually was wrong. The
job never enqueued in the original spec and so the gravatar was never
actually updated and the test was checking if the two values were the
same, but they were both null and never updated, so of course they were
the same!

A new test has also been added to make sure the gravatar job isn't
enqueued when a user's email is missing.
2020-09-02 20:19:46 -06:00
Blake Erickson d79f50a3c6 DEV: Remove rb-inotify gem from Gemfile
We are no longer directly referencing the rb-inotify gem directly in
code. This was just a spec level dependency anyways.

Using `git log -S "Inotify"` resulted in these two commits as usages of
`Inotify`:

- b56b11d96a
- 9cf03b352c

both from 2013, but we no longer are using inotify in

https://github.com/discourse/discourse/blob/master/lib/tasks/autospec.rake

which appears to be the only file that was using it.

Based on this info we can safely remove rb-inotify from the Gemfile.

Just as a side note we still do have a couple of gems that do have
rb-inotify as a dependency: listen, and lru_redux.
2020-09-02 11:21:40 +08:00
Blake Erickson 43ffd4d28f DEV: Switch our fast_xor gem for xorcist (#10565)
* DEV: Switch our fast_xor gem for xorcist

We use the `xor` function as part of password hashing and we want to use
a faster version than the native ruby xor'ing feature so we use a gem
for this.

fast_xor has been abandoned, and xorcist fixed our initial holdup for
switching in https://github.com/fny/xorcist/issues/4

xorcist also has jruby support so we can remove our jruby fallback
logic.

* Move using statement inside of class
2020-08-31 13:20:44 -06:00
Blake Erickson 02833e133c FIX: Suspend API to require suspend_until and reason params
These fields are required when using the UI and if `suspend_until`
params isn't used the user never is actually suspended so we should
require these fields when suspending a user.
2020-08-26 19:05:48 -06:00
Blake Erickson 7cfd5f87ff FIX: Topic Timer auto opening closed topics (#10524)
This commit is addressing an issue where it is possible that there could
be multiple topic timer jobs running to close a topic or a weird race
condition state causing a topic that was just closed to be re-opened.

By removing the logic from the Topic Timer model into the Topic Timer
controller endpoint we isolate the code that is used for setting an
auto-open or an auto-close timer to just that functionality making the
topic timer background jobs safer if multiple are running.

Possibly in the future if we would like this logic back in the model a
refactor will be needed where we actually pass in the auto-close and
auto-open action instead of mixing it with the close and open
action that is currently being passed to the controller.
2020-08-26 11:17:12 +10:00
Blake Erickson 4395e4d165 FIX: Sending a PM through a flag on a deleted post
Because we allow all the other flag types on a deleted post we should be
able to send a pm to the user letting them know why we deleted their
post.

Bug report:

https://meta.discourse.org/t/-/161156
2020-08-19 17:36:52 -06:00
Blake Erickson 367de2594d FIX: Unlike own posts on ownership transfer (#10446)
* FIX: Unlike own posts on ownership transfer

If a user has liked a post that has passed the
`post_undo_action_window_mins` system setting window and you transfer ownership
of that post to that user you will be the owner of a post that you have
liked, but cannot unlike resulting in a weird UI behavior. This commit
fixes this issue.

The existing tests didn't check for the timeout window for unliking
posts so I added that in.

I couldn't find a good way to do this logic inside of the guardian class
so rather than duplicating behavior of the `PostActionDestroyer` class
inside of the `PostOwnerChanger` I decided to pass in a "bypass"
variable that could be used to check if the calling class is the
'post_owner_changer' and bypass the guardian instead. I went this route
because the guardian `can_delete_post_action` method has no way of
distinguishing how to allow a user to be able to unlike their own posts
after the timeout window but only on a post owner change.

* use an options hash instead
2020-08-19 09:21:02 -06:00
Blake Erickson ea2e58e622 DEV: Bump rotp gem to latest version (#10472)
The rotp gem is currently pinned to version 5.1.0 and this will bump it
up to version 6.0.1.

Follow up to: 85d4370f79

because this issue we were waiting on is now closed:

https://github.com/mdp/rotp/issues/98

Because version 6 is now encoding the params I needed to update the
tests as well.
2020-08-19 09:16:33 -06:00
Blake Erickson c767fb1ed3 DEV: Upgrade highline gem to latest (#10471)
Currently we have pinned highline to version 1.7.0. This is the gem that
we use to have an interactive command line for tasks like `rake
admin:create`.

Upgrading to the latest version 2.0.3 will remove ruby 2.7 deprecation
warnings.

I'm not sure why *this* gem was pinned. I manually executed a couple of
our rake tasks that use this and everything seems fine.
2020-08-19 09:15:23 -06:00
Blake Erickson ba9fbb3443 FIX: Composer upload icon regression because of HEIC
Now that HEIC conversion is enabled by default we need to update a few
file extension checks to include `heic` and heif` formats.

https://meta.discourse.org/t/-/144326/23?u=blake

bug reported here:

https://meta.discourse.org/t/-/161197?u=blake
2020-08-17 19:39:26 -06:00
Blake Erickson c2f3c0dc44 FIX: generate_topic_thumbnails job infinitely running for corrupted images
It's possible that the original topic image is broken in some form, so
we shouldn't try and generate a topic thumbnail for it. The fix will
prevent the generate_topic_thumbnails job being enqueued every time the
topic is viewed.
2020-08-13 17:08:32 -06:00
Blake Erickson c68563a281 DEV: Improve API usage when creating * updating categories
The category model already has a default value for `color` and
`text_color` so they don't need to be required via the API. The ember UI
already requires that colors be selected.

The name of the category also doesn't need to be required when updating
the category either because we are already passing in the id for the
category we want to change.

These changes improve the api experience because you no longer have to
lookup the category name, color, or text color before updating a single
category attribute. When creating a category the name is still required.

https://meta.discourse.org/t/-/132424/2
2020-08-12 12:28:29 -06:00
Blake Erickson ee366f7ac7 FIX: Ensure load more directory items has a .json url
The `/directory_items` route needs to have a .json url, but the rails
url helper `_path` doesn't return the format of the route.

I tried passing in a format options to `directory_items_path`. Which
works in the rails console

```
[8] pry(main)> directory_items_path(params.merge(:format => :json))
=> "/directory_items.json?page=1"
```

but when I added that some logic to the controller it comes out as

```
/directory_items?format=json&page=1
```

(which is actually how I expect it to work based on how you pass in the
format param). Anyways, because I couldn't figure out how to pass a
format to the `_path` helper I just used URI.parse to append `.json`
manually.
2020-08-11 12:43:55 -06:00
Blake Erickson 2032c11f78 FIX: Return 422 when creating topics with tags w/out permission (#10400)
The UI prevents users from trying to create tags on topics when they
don't have permission, but if you are trying to add tags to a topic via
the API and you don't have permission before this change it would
silently succeed in creating the topic, but it wouldn't have any tags.

Now a 422 error will be returned with an error message when trying to
create a topic with tags when tagging is disabled or you don't have
enough trust level to add tags to a topic.

Bug report: https://meta.discourse.org/t/-/70525/14
2020-08-10 16:14:15 -06:00
Blake Erickson 33982c0c9e DEV: Document notifications and tags api endpoints
Added some more specs that will be used to auto generate the api docs.
2020-08-07 14:30:54 -06:00