Change the type for default_calendar to a string.
The type specified for the default calendar in the api docs wasn't a
valid type. The linting in the api docs repo reports:
```
`type` can be one of the following only: "object", "array", "string", "number", "integer", "boolean", "null".
```
This linting currently is only in the `discourse_api_docs` repo.
* DEV: Add include_subcategories param to api docs
Adding the `include_subcategories=true` query param to the
`/categories.json` api docs.
Follow up to: fe676f334a
* fix spec
* FEATURE: Return subcategories on categories endpoint
When using the API subcategories will now be returned nested inside of
each category response under the `subcategory_list` param. We already
return all the subcategory ids under the `subcategory_ids` param, but
you then would have to make multiple separate API calls to fetch each of
those subcategories. This way you can get **ALL** of the categories
along with their subcategories in a single API response.
The UI will not be affected by this change because you need to pass in
the `include_subcategories=true` param in order for subcategories to be
returned.
In a follow up PR I'll add the API scoping for fetching categories so
that a readonly API key can be used for the `/categories.json` endpoint. This
endpoint should be used instead of the `/site.json` endpoint for
fetching a sites categories and subcategories.
* Update PR based on feedback
- Have spec check for specific subcategory
- Move comparison check out of loop
- Only populate subcategory list if option present
- Remove empty array initialization
- Update api spec to allow null response
* More PR updates based on feedback
- Use a category serializer for the subcategory_list
- Don't include the subcategory_list param if empty
- For the spec check for the subcategory by id
- Fix spec to account for param not present when empty
Due to the way that rswag expands shared components we were getting this
warning when linting our api docs:
```
Component: "user_response" is never used.
```
This change refactors the `api/users_spec.rb` file so that it uses the
new way of doing things with a separate `user_get_response.json` schema
file rather then the old way of loading a shared response inside of the
swagger_helper.rb file.
The color_scheme_id needs to be an integer not a string.
This is one of the failing tests that showed this error:
https://github.com/discourse/discourse/runs/3598414971
It showed this error
`POSSIBLE ISSUE W/: /user_themes/0/color_scheme_id`
And this is part of the site.json response:
```
...
"user_themes"=>[{"theme_id"=>149, "name"=>"Cool theme 111", "default"=>false, "color_scheme_id"=>37}]
...
```
One of the fields that should be present for openapi docs is the "license"
field.
https://spec.openapis.org/oas/latest.html#infoObject
Our API docs already had a license, so this commit just specifies that
and provides a link to it.
From the openapi spec:
https://spec.openapis.org/oas/latest.html#fixed-fields-7
each endpoint needs to have an `operationId`:
> Unique string used to identify the operation. The id MUST be unique
> among all operations described in the API. The operationId value is
> case-sensitive. Tools and libraries MAY use the operationId to uniquely
> identify an operation, therefore, it is RECOMMENDED to follow common
> programming naming conventions.
Running the linter on our openapi.json file with this command:
`npx @redocly/openapi-cli lint openapi.json`
produced the following warning on all of our endpoints:
> Operation object should contain `operationId` field
This commit resolves these warnings by adding an operationId field to
each endpoint.
* DEV: Use named parameters for dir-span helper
Follow up to: e50a5c0c73
In order to improve code clarity this change introduces named parameters
for the dir-span helper. This is specifically for the new `htmlSafe`
parameter which you can use instead of just passing in a boolean if the
strings you are passing in have already been escaped.
Before: `{{dir-span category.description false}}`
After: `{{dir-span category.description htmlSafe=true}}`
* Set default value for params arg
`nullable` is no longer a valid type, and types also can't be an empty
string, so just bringing a number of issues with types in compliance
with the openapi spec.
The latest openapi spec version is v3.1.0
https://spec.openapis.org/oas/v3.1.0
Specifying the latest version will allow our openapi spec linter to use
this version and allow use to use the new type format that allows for
specifying a type as "null", which we need because sometimes our api
responses include null values instead of a "string", "integer", or
"object" type.
See: https://stackoverflow.com/a/48114322/588458
* DEV: Return 400 instead of 500 for invalid top period
This change will prevent a fatal 500 error when passing in an invalid
period param value to the `/top` route.
* Check if the method exists first
I couldn't get `ListController.respond_to?` to work, but was still able
to check if the method exists with
`ListController.action_methods.include?`. This way we can avoid relying
on the `NoMethodError` exception which may be raised during the course
of executing the method.
* Just check if the period param value is valid
* Use the new TopTopic.validate_period method
This TODO comment has existed for 8 years. Sort must be working just
fine or we would have prioritized fixing it.
Removing this comment as a tiny step toward keeping our codebase nice
and tidy.
The main image_optim gem now includes the timeout feature
that we had in our fork. So it is now safe to switch off of our fork and
back to the image_optim gem.
This is the link to the commit in the image_optim repo that adds the
timeout option:
https://github.com/toy/image_optim/commit/ec3767dde0d9f9f3d18bd688645583d847ba2e13
One difference with the new timeout implementation is that image_optim
now handles the timeout exceptions instead of bubbling them up:
https://github.com/toy/image_optim/blob/1ed03285873078185c129c66007231469461e429/lib/image_optim.rb#L128-L129
```
rescue Errors::TimeoutExceeded
handler.result
```
So a timeout will just return `nil`, which is the same response if it
couldn't optimize an image. I don't think we were really watching for
or doing anything about these timeout warnings in our logs so I think
this is an okay change to have and we will have less warnings in our
logs now too.
It's been awhile since we have supported IE11 so it should be safe to remove
IntersectionObserver now.
From a TODO task in this repo:
> drop when we eventually drop IE11
Announcement of when we removed IE11 support:
https://meta.discourse.org/t/137984/40?u=blake
* FIX: Be able to handle long file extensions
Some applications have really long file extensions, but if we truncate
them weird behavior ensues.
This commit changes the file extension size from 10 characters to 255
characters instead.
See:
https://meta.discourse.org/t/182824
* Keep truncation at 10, but allow uppercase and dashes
When creating a PM target_usernames has been deprecated for some time
now but the api docs have yet to reflect this.
The api docs now specify to use target_recipients.
See: eef21625c6
Some git repos have a different ssh url scheme than github and we should
support them.
This change updates our regex format to account for repos that don't
start with "git", but are still valid ssh urls.
Also I added some tests to account for the various formats and to ensure
we don't show the public key when using https urls.
See: https://meta.discourse.org/t/182668
The urls that we generate for mobile post notifications don't take into
account the subfolder url if a site happens to have one configured. When
this happens when you tap on a new mobile notification it takes you to
a url that doesn't work because it is missing the subfolder portion.
I honestly think this should be handled in the Post model like we do
with the Topic model. `Post.url` should know how to handle subfolder
installs, but that seemed like a very risky change because there are
lots of other places in the codebase where we tack on the base_path and
I didn't want to risk duplicating it.
I also found a small typo in the topics controller spec.
Unactivated users that have posts cannot be deleted so we shouldn't
include them in the initial query to try and purge them. Otherwise we
are just loading up sidekiq with pointless work to be doing every day.
Without this change if there are 201 unactivated users to purge, but the
first 200 have posts, the 201st user will never be deleted even though
it is the only user that doesn't have a post and is actually the one
that should be deleted.
This is an importer I wrote to restore some users that were
accidentally deleted for being purged as old staged users or old
unactivated users.
It reads from CSV files exported from a discourse sql backup.
When running an import script there are many site settings that are
changed but we reset them back to where they were originally before the
import. However, there are two settings that we don't roll back:
```
purge_unactivated_users_grace_period_days
purge_deleted_uploads_grace_period_days
```
which could have some unintended consequences.
My first question is do we *really* have to change these settings? I'm
not a huge fan of changing someones settings without them really knowing
they were changed.
If we really do have to change these settings here is my proposed PR
where we don't alter the `purge_unactivated_users_grace_period_days` if
it has been disabled.
As I'm writing this another change we could make is that we don't change
either of these site settings if we detect that they aren't set to the
default values.
The drive behind this PR is that there is a discourse instance which
relies on staged users as part of their workflow and this setting was
changed by accident via the import script causing users to be deleted
that shouldn't have been.
* DEV: Show warning message when using ember css selectors
When editing the theme css via the admin UI a warning message
will be displayed if it detects that the `#emberXXX` or `.ember-view`
css selectors are being used. These are dynamic selectors that ember
generates, but they can change so they should not be used.
* Update error message text to be more helpful
* Display a warning instead of erroring out
This allows the theme to still be saved, but a warning is displayed.
Updated the tests to check for the error message.
Updated the pre tags css so that it wraps for long messages.
* DEV: Escape backslashes in curl example
We need to escape these backslashes otherwise they get filtered out when
generating the api docs.
* FIX: uniqItems should be uniqueItems
* document user endpoints, allow for empty request/response bodies
* document more user endpoints, improve debugging output if no details are specified
* document some more user endpoints
* minor cleanup
* FIX: flakey tests due to bad regex
- Read in schemas from actual json files instead of a ruby hash. This is
helpful because we will be automatically generating .json schema files
from json responses and don't want to manually write ruby hash schema
files.
- Create a helper method for rspec schema validation tests to dry up code
* DEV: Add schema checking to api doc testing
This commit improves upon rswag which lacks schema checking. rswag
really only checks that the https status matches, but this change adds
in the json-schema_builder gem which also has schema validation.
Now we can define schemas for each of our requests/responses in the
`spec/requests/api/schemas` directory which will make our documentation
specs a lot cleaner.
If we update a serializer by either adding or removing an attribute the
tests will now fail (this is a good thing!). Also if you change the type
of an attribute say from an array to a string the tests will now fail.
This will help significantly with keeping the docs in sync with actual
code changes! Now if you change how an endpoint will respond you will
have to update the docs too in order for the tests to pass. :D
This PR is inspired by:
https://www.tealhq.com/post/how-teal-keeps-their-api-tests-and-documentation-in-sync
* Swap out json schema validator gem
Swapped out the outdated json-schema_builder gem with the json_schemer
gem.
* Add validation fields to schema
In order to have "strict" validation we need to add
`additionalProperties: false` to the schema, and we need to specify
which attributes are required.
Updated the debugging test output to print out the error details if
there are any.
Added GET user by external_id to the api docs.
Fixed `/users/{username}` docs to be `/u/{username}`
Extracted out common user response into a shared helper.
On category create an exception will be thrown on this job because the
save transaction hasn't completed yet and the job cannot find the
category id. To prevent this we can use the rails 6 `after_save_commit`
hook that will fire after the category save transaction has finished for
both update and create actions.
Documenting a few more endpoints so that our api docs can be
automatically generated. Made a couple other minor changes, like
including the "OK" example for our default success response.
We are switching over to a fork because we are currently on a pinned
version of ember-rails 0.18.5 which is pretty old. Upgrading to the
latest version causes many things to break which isn't really worth the
time to debug while we plan to completely switch over to ember-cli
somewhat soonish. Our fork contains a single cherry-pick commit
https://github.com/emberjs/ember-rails/pull/534
which will fix an issue when running the `rails g migration` command and
it spits out a bunch of deprecation warnings.
There is a request spec that was ignored with the `xit` flag almost a
year ago and every time you generate the api docs with
```
rake rswag:specs:swaggerize
```
it shows the output of this pending test and I guess I finally got sick
of looking at it, so here is a fix for it.
Original Commit: d84c34ad75
We are switching over to a fork because we are currently on a pinned
version of ember-rails 0.18.5 which is pretty old. Upgrading to the
latest version causes many things to break which isn't really worth the
time to debug while we plan to completely switch over to ember-cli
somewhat soonish. Our fork contains a single cherry-pick commit
https://github.com/emberjs/ember-rails/pull/534
which will fix an issue when running the `rails g migration` command and
it spits out a bunch of deprecation warnings.
It is possible that a user could exist without an email, if so we should
not enqueue a job to download their gravatar.
This commit resolves this error that can occur:
```
Job exception: undefined method `email' for nil:NilClass
/var/www/discourse/app/models/user.rb:1204:in `email'
/var/www/discourse/app/jobs/regular/update_gravatar.rb:12:in `execute'
```
This commit also fixes the original spec which actually was wrong. The
job never enqueued in the original spec and so the gravatar was never
actually updated and the test was checking if the two values were the
same, but they were both null and never updated, so of course they were
the same!
A new test has also been added to make sure the gravatar job isn't
enqueued when a user's email is missing.
We are no longer directly referencing the rb-inotify gem directly in
code. This was just a spec level dependency anyways.
Using `git log -S "Inotify"` resulted in these two commits as usages of
`Inotify`:
- b56b11d96a
- 9cf03b352c
both from 2013, but we no longer are using inotify in
https://github.com/discourse/discourse/blob/master/lib/tasks/autospec.rake
which appears to be the only file that was using it.
Based on this info we can safely remove rb-inotify from the Gemfile.
Just as a side note we still do have a couple of gems that do have
rb-inotify as a dependency: listen, and lru_redux.
* DEV: Switch our fast_xor gem for xorcist
We use the `xor` function as part of password hashing and we want to use
a faster version than the native ruby xor'ing feature so we use a gem
for this.
fast_xor has been abandoned, and xorcist fixed our initial holdup for
switching in https://github.com/fny/xorcist/issues/4
xorcist also has jruby support so we can remove our jruby fallback
logic.
* Move using statement inside of class
These fields are required when using the UI and if `suspend_until`
params isn't used the user never is actually suspended so we should
require these fields when suspending a user.
This commit is addressing an issue where it is possible that there could
be multiple topic timer jobs running to close a topic or a weird race
condition state causing a topic that was just closed to be re-opened.
By removing the logic from the Topic Timer model into the Topic Timer
controller endpoint we isolate the code that is used for setting an
auto-open or an auto-close timer to just that functionality making the
topic timer background jobs safer if multiple are running.
Possibly in the future if we would like this logic back in the model a
refactor will be needed where we actually pass in the auto-close and
auto-open action instead of mixing it with the close and open
action that is currently being passed to the controller.
Because we allow all the other flag types on a deleted post we should be
able to send a pm to the user letting them know why we deleted their
post.
Bug report:
https://meta.discourse.org/t/-/161156
* FIX: Unlike own posts on ownership transfer
If a user has liked a post that has passed the
`post_undo_action_window_mins` system setting window and you transfer ownership
of that post to that user you will be the owner of a post that you have
liked, but cannot unlike resulting in a weird UI behavior. This commit
fixes this issue.
The existing tests didn't check for the timeout window for unliking
posts so I added that in.
I couldn't find a good way to do this logic inside of the guardian class
so rather than duplicating behavior of the `PostActionDestroyer` class
inside of the `PostOwnerChanger` I decided to pass in a "bypass"
variable that could be used to check if the calling class is the
'post_owner_changer' and bypass the guardian instead. I went this route
because the guardian `can_delete_post_action` method has no way of
distinguishing how to allow a user to be able to unlike their own posts
after the timeout window but only on a post owner change.
* use an options hash instead
The rotp gem is currently pinned to version 5.1.0 and this will bump it
up to version 6.0.1.
Follow up to: 85d4370f79
because this issue we were waiting on is now closed:
https://github.com/mdp/rotp/issues/98
Because version 6 is now encoding the params I needed to update the
tests as well.
Currently we have pinned highline to version 1.7.0. This is the gem that
we use to have an interactive command line for tasks like `rake
admin:create`.
Upgrading to the latest version 2.0.3 will remove ruby 2.7 deprecation
warnings.
I'm not sure why *this* gem was pinned. I manually executed a couple of
our rake tasks that use this and everything seems fine.
It's possible that the original topic image is broken in some form, so
we shouldn't try and generate a topic thumbnail for it. The fix will
prevent the generate_topic_thumbnails job being enqueued every time the
topic is viewed.
The category model already has a default value for `color` and
`text_color` so they don't need to be required via the API. The ember UI
already requires that colors be selected.
The name of the category also doesn't need to be required when updating
the category either because we are already passing in the id for the
category we want to change.
These changes improve the api experience because you no longer have to
lookup the category name, color, or text color before updating a single
category attribute. When creating a category the name is still required.
https://meta.discourse.org/t/-/132424/2
The `/directory_items` route needs to have a .json url, but the rails
url helper `_path` doesn't return the format of the route.
I tried passing in a format options to `directory_items_path`. Which
works in the rails console
```
[8] pry(main)> directory_items_path(params.merge(:format => :json))
=> "/directory_items.json?page=1"
```
but when I added that some logic to the controller it comes out as
```
/directory_items?format=json&page=1
```
(which is actually how I expect it to work based on how you pass in the
format param). Anyways, because I couldn't figure out how to pass a
format to the `_path` helper I just used URI.parse to append `.json`
manually.
The UI prevents users from trying to create tags on topics when they
don't have permission, but if you are trying to add tags to a topic via
the API and you don't have permission before this change it would
silently succeed in creating the topic, but it wouldn't have any tags.
Now a 422 error will be returned with an error message when trying to
create a topic with tags when tagging is disabled or you don't have
enough trust level to add tags to a topic.
Bug report: https://meta.discourse.org/t/-/70525/14
When inviting users to a PM a small post is created showing that you
invited the user, but the actual list of participants never updated
until you refreshed the page or interacted with the private-message-map
widget triggering it to refresh. This change will trigger a refresh on
the private-message-map widget ensuring that the UI is updated with the
current list of participants.
Ensure that the inappropriate flag is used and not some other flag
type. If some other flag is used a message will be posted letting the
user know they used the wrong flag and the original flag will be removed
so that they can try again.
Bug reported on meta: https://meta.discourse.org/t/-/157075
Apparently latest.json and latest.rss are not routed to the same
controller methods. This change allows for any passed in query
parameters to actually be applied to the rss route.
This came in as a request on meta:
https://meta.discourse.org/t/-/155812/6
* DEV: Show message when cannot invite user to PM
When inviting a user to a PM return a message that says, "Sorry, this
user can't be invited." if they have been muted or are not in a users
allowed pm users list.
* Minor refactor & improved some text
Before this commit if you were bulk removing group members and passed in
a user who wasn't currently a member of that group the whole request
would fail. This change will return a 200 response now listing the users
that were removed and those that were skipped.
* FEATURE: Allow List for PMs
This feature adds a new user setting that is disabled by default that
allows them to specify a list of users that are allowed to send them
private messages. This way they don't have to maintain a large list of
users they don't want to here from and instead just list the people they
know they do want. Staff will still always be able to send messages to
the user.
* Update PR based on feedback
It's possible through an import or other means to have images larger
than the current max allowed image size in the db.
If this happens the thumbnail generation job will keep running
indefinitely trying to download a new copy of the original but
discarding it because it is larger than the max_file_size eventually
causing this error
`Job exception: undefined method `path' for nil:NilClass`
because the newly downloaded image is now nil.
This fix stops the enqueuing of the `GenerateTopicThumbnails` job for
all images that happen to be larger than the max image size.
If any value, including nil, is passed in as an argument the default
won't be set, so we need to handle when a non-Array value is passed in
to the `generate_thumbnails!` method.
Added a small helper class to for seed data because we need to add the
same filter to multisite:migrate as we have in db:migrate. Having this
filter in both places means we can get rid of the SKIP_SEED flag.
This came in as a request on meta to include the raw field in the post
webhook serializer.
https://meta.discourse.org/t/-/49045/55?u=blake
Including this field can prevent needing to make a 2nd API request to
get the raw field of a post.
It would be handy down the road if we updated the webhook ui to specify
fields or arguments that you wanted to be included in the serialized
data, but most requests I've seen to update the serializers have been
valid requests that are good to add anyways, so I don't think we have
reached that point yet.
Looks like some html elements like `aside` and `section` will throw an error
when checking if they are inline or not. The commit simply handles
```
Job exception: undefined method `inline?' for nil:NilClass
```
and adds a test for it.
Fixes a regression in
e8fb9d4066
which caused a bug where you couldn't send a message to a group that
contained an Uppercase letter. Added a test case for this.
Bug report: https://meta.discourse.org/t/-/152999
Blake Erickson