836 Commits
Author SHA1 Message Date
Martin Brennan 542b65070a DEV: Decrease RSpec startup time (#43151)
Reduces RSpec startup time by avoiding work that the selected spec does
not need.

The main changes are:

- Mark development/test dependencies as require: false where they can be
loaded safely at their point of use.
- Avoid loading Capybara, the Playwright driver, system helpers, and
page objects unless the selected specs need them.
- Autoload several large spec helper modules and load Rails tasks only
for task specs.
- Load OAuth providers, schema libraries, API documentation support, and
QR-code support at their actual call sites.
- Preload production dependencies before workers fork to preserve
copy-on-write sharing.
- Register parallel_tests Rake tasks only in local environments so
production bundles do not require test-only gems.
- Make rbtrace opt-in via RBTRACE=1.
- Avoid creating an unrelated top-level fixture in user_spec.rb.
- Explicitly load Capybara in the nginx integration spec, which sits
outside the system spec directory.

The new lazy support files work as follows:

- lazy_fabricators.rb indexes core fabricator definitions without
executing every file. When an unregistered fabricator is first
requested, only the file defining it is loaded. Plugin suites retain
eager fabricator loading for compatibility.
- lazy_faker.rb intercepts a missing Faker constant, loads the full
Faker gem on first use, and retries the lookup.
- lazy_pry.rb provides a temporary pry method that loads pry and
pry-rails on first use, then delegates to the real implementation.

On my machine, startup for bin/rspec spec/models/user_spec.rb:20
decreased from approximately 2.1 seconds to 1.1 seconds—a reduction of
roughly 47%.
2026-09-03 11:41:29 +10:00
Martin Brennan d2bbcd5f6d Revert "DEV: Decrease rspec startup time (#42946)" (#43147)
This reverts commit 8b8dbf5bae.
2026-09-03 10:59:58 +10:00
Martin Brennan 8b8dbf5bae DEV: Decrease rspec startup time (#42946)
Reduces RSpec startup time by avoiding work that the selected spec does
not need.

The main changes are:

- Mark development/test dependencies as require: false where they can be
loaded safely at their point of use.
- Avoid loading Capybara, the Playwright driver, system helpers, and
page objects unless system or request specs need them.
- Autoload several large spec helper modules and load Rails tasks only
for task specs.
- Load OAuth providers, schema libraries, API documentation support, and
QR-code support at their actual call sites.
- Make rbtrace opt-in via RBTRACE=1.
- Avoid creating an unrelated top-level fixture in user_spec.rb.
 
The new lazy support files work as follows:

- lazy_fabricators.rb indexes core fabricator definitions without
executing every file. When an unregistered fabricator is first
requested, only the file defining it is loaded. Plugin suites retain
eager fabricator loading for compatibility.
- lazy_faker.rb intercepts a missing Faker constant, loads the full
Faker gem on first use, and retries the lookup.
- lazy_pry.rb provides a temporary pry method that loads pry and
pry-rails on first use, then delegates to the real implementation.

On my machine, startup for bin/rspec spec/models/user_spec.rb:20
decreased from approximately 2.1 seconds to 1.1 seconds—a reduction of
roughly 47%.
2026-09-03 10:11:50 +10:00
89bcc0f498 FEATURE: Add voice as a core plugin (#43044)
Adds **voice** (formerly the standalone `resenha` plugin) as a core
plugin: Discord-style voice and video rooms powered by WebRTC —
sidebar-first rooms, direct calls, optional LiveKit SFU routing,
chat-thread integration, live subtitles, and recordings.

## Rename

The plugin was renamed `resenha` → `voice` as part of the move (`module
::Voice`, `plugins/voice`, `voice_*` settings and tables, mounted at
`/voice`), consistent with `plugins/chat`.

Sites that ran the plugin under its old name are converted by a
post-deploy migration: table/index/trigger renames, `resenha_*` →
`voice_*` site settings, reviewable types, chat thread custom fields,
the badge grouping, and rebake marks for posts/chat messages whose
cooked HTML embeds `/resenha/` links. Fresh installs create the
`voice_*` schema directly. The `resenha_invitation` notification type
keeps its id (1000) under the new `voice_invitation` name.

## Size

Large media assets (wasm noise-suppression engines, ML models, vendored
SDK bundles, ~75MB) are not in this tree — they ship in the
[discourse_voice_assets](https://github.com/discourse/discourse_voice_assets)
gem and are served from a gem-version-stamped path, so the plugin adds
~2.5MB of source.

## Also included

- `DEV: Fix bin/lint crash on unbundled plugin paths` —
`bundled_plugins` is a `Set`, which has no `#exclude?` without
ActiveSupport, so linting any unbundled `plugins/` path raised
`NoMethodError`.

---------

Co-authored-by: Gabriel Grubba <70247653+Grubba27@users.noreply.github.com>
Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
Co-authored-by: Keegan George <kgeorge13@gmail.com>
Co-authored-by: David Taylor <david@taylorhq.com>
Co-authored-by: Renato Atilio <renatoat@gmail.com>
Co-authored-by: dupless54 <126967564+dupless54@users.noreply.github.com>
Co-authored-by: Arpit Jalan <arpit@arpitjalan.com>
2026-09-01 13:08:26 -03:00
Rafael dos Santos Silva 87775447c0 FIX: Silent truncation and corruption of AI structured output streams (#41716)
Reported at
https://meta.discourse.org/t/translation-silently-truncated-when-json-stream-parsing-breaks-no-error-raised/407251

## The bug

Some providers stream structured output whose string values were
unescaped by an outer JSON parse, so real newlines appear inside string
values. When that happened, `JsonStreamingTracker` had two failure
modes:

- It marked the stream broken and `StructuredOutput` fell back to
`BestEffortJsonParser`, whose extraction regex (`[^"]+`) cut the value
at the first escaped quote and left `\n` sequences as literal text. A
2,000-char translation could come back as ~50 chars, cut right before
the first quoted word — exactly what the report shows.
- Its escape-and-resume hack (`String#dump` + buffer-growth offset)
miscomputed the resume index whenever the chunk contained non-ASCII,
quotes, or backslashes, silently duplicating or corrupting content
**without ever marking the stream broken**.

Testing a realistic corrupted payload across chunk sizes 1–60: 31
produced the truncated fallback, 27 produced silent corruption, 1
raised, and only 1 came out correct. Either way the result was persisted
as a successful translation with nothing in the logs.

## The fix

Replace the hand-rolled parsing with two gems and keep only glue:

- **json_completer** (pure Ruby): `JsonStreamingTracker` now feeds the
cumulative buffer — with control characters re-escaped — to an
incremental, truncation-tolerant parser and notifies consumers of
changed keys. The corrupted payloads above stream correctly at every
chunk size, so the broken-stream path is only reached for responses that
aren't JSON at all.
- **smarter_json**: `BestEffortJsonParser` becomes a three-attempt chain
(strict-with-completion → control-chars re-escaped → lenient) covering
the quirk shapes the old regexes handled: single quotes, unquoted keys,
markdown fences, prose-wrapped JSON.

This deletes the vendored 668-line SAX parser, the resume hack, and all
manual regex extraction (net −694 lines), and adds a log warning
whenever a response falls back to best-effort parsing.

## Behavior changes

- Scalars now stream progressively: mid-stream
`read_buffered_property(:number)` returns the digits buffered so far
instead of `nil`. Consumers act on final values, so this only affects
mid-stream reads.
- Arrays of objects stream partial objects mid-stream instead of
returning `nil` until finish.
- A trailing comma in an array reads as a `nil` placeholder slot until
the next element arrives.
- Partial tool calls surface a few more progressive updates (the openai
endpoint spec count moved 128 → 134); values still only ever grow.

## Tests

- Regression specs for the report: unescaped control characters with
escaped quotes/emoji streamed across chunk boundaries, fenced +
unescaped responses, truncated JSON, numeric casting.
- 916 examples green across `completions/`, `translation/`,
`modules/ai_helper/`, and `utils/`.
2026-07-15 12:00:24 -03:00
Régis Hanol 36a8a51ef0 FEATURE: Route all GitHub API requests through one rate-limited client (#40637)
GitHub oneboxes and the discourse-github plugin talked to GitHub's REST
and
GraphQL API with no rate-limit awareness. On busy instances this
exhausted
GitHub's limits (60 requests/hour unauthenticated, 5000 authenticated),
and
because there was no backoff every render kept hitting GitHub and
re-failing
-- which GitHub's docs warn can get an integration banned. The recently
added PR-status onebox multiplied the number of calls and made it far
worse.

GitHub access was also fragmented: the core onebox engines used OpenURI,
the
discourse-github plugin used Octokit, and the discourse-ai bot tools
used
FinalDestination::HTTP -- three HTTP stacks, three tokens, and
inconsistent
(or entirely missing) error and rate-limit handling.

This introduces a single client, Discourse::GithubApi, that every GitHub
data-API request now flows through. It is built on Faraday with the
SSRF-safe
FinalDestination adapter and:

- authenticates per token (Bearer) and returns plain string-keyed Hashes
(get/post) or raw bodies (raw_get) -- one response shape, no
Octokit/Sawyer
- only ever sends the access token to api.github.com and
  raw.githubusercontent.com, rejecting any other absolute URL, so a
  user-derived path can never leak a token to an arbitrary host
- backs off on rate limits both reactively (403/429) and proactively
(when
X-RateLimit-Remaining hits 0), honouring Retry-After /
X-RateLimit-Reset,
via a shared Redis flag (GithubRateLimit) keyed per token so each
token's
  budget and the shared unauthenticated/IP budget back off independently
- short-circuits while backing off without ever sleeping, so onebox
rendering
  and post baking degrade to a plain link instead of blocking a request
- caches ETags and sends If-None-Match, so unchanged resources return
304s
  that do not count against the rate limit

Every caller was moved onto it:

- the 6 core GitHub onebox engines, via a slimmed
Onebox::Mixins::GithubApi
adapter that keeps their public methods and translates client errors
back
to the OpenURI::HTTPError vocabulary they already rescue (engines
unchanged)
- the github_blob raw.githubusercontent.com fetch
- the discourse-github plugin (badges, linkback, permalinks, token
validator),
which no longer uses the octokit and sawyer gems (they stay in the
Gemfile for
the discourse-code-review official plugin, which still depends on them)
- the discourse-ai bot's GitHub tools (search code, diff, file content,
  search files)

Also adds a GithubOneboxBackoff admin problem check that surfaces while
one of
the onebox token identities is backing off -- scoped to the tokens
resolved by
Onebox::GithubAccess (each configured github_onebox_access_tokens entry
plus the
unauthenticated client) so a backoff on the AI bot or linkback token is
not
misattributed to onebox. Its message points admins at the relevant
setting with
the {{setting:...}} link marker, which problem-check messages now expand
too.
Onebox token resolution is centralised in Onebox::GithubAccess, and the
onebox
cache TTL for transient GitHub failures is shortened so they recover
quickly.

GitHub OAuth login, theme git-clone, the inbound webhook, and the
Oneboxer
FinalDestination URL-resolution special-cases for github.com are
intentionally
out of scope -- they are different concerns, not the rate-limited data
API.
2026-06-15 10:59:10 +02:00
Joffrey JAFFEUX 237ed2ae3e FEATURE: Use Liquid for workflow template rendering (#40602)
Previously, the workflow Template node rendered Mustache template. This
change uses Liquid instead as we think it's a better UX. Shopify uses
(and created) liquid making it a very well know system.
2026-06-05 13:04:45 +02:00
Gerhard Schlager 06b32204c0 MT: Split the migrations tooling into separate gems (#40492)
Previously, the migrations tooling was a single flat `migrations/` tree,
autoloaded by one global Zeitwerk loader and driven by a Thor CLI, so each
planned next step had nowhere clean to land.

This change splits it into four `path:`-referenced gems — `migrations-core`,
`migrations-tooling`, `migrations-converters`, and `migrations-importer` —
served by a single Samovar-based `disco` binary, without rewriting any domain
logic.

### Why now

The DSL refactor that replaced the IntermediateDB YAML config just landed,
which is the cheapest moment to do this. Everything queued behind it — column
coverage verification, the `discourse-migrations` validation plugin, the
transformer framework, and private converter isolation — either has nowhere
clean to land in the flat tree or would have to be retrofitted into a gem
layout later. Doing the split now, while it's still a pure move (suite green,
no domain logic touched), is far cheaper than after another round of features
has built on the flat layout.

### What changes

- **Four gems under `migrations/`**, all `path:`-referenced from the root
  `Gemfile` (nothing is published to RubyGems): `core` (CLI framework, UI, DB
  infrastructure, IntermediateDB, and the conversion framework), `tooling`
  (schema DSL and `schema` commands), `converters` (implementations and source
  adapters), and `importer` (row and uploads import).
- **A single CLI binary:** `migrations/bin/cli` (Thor) becomes `disco`
  (Samovar), with each gem registering its own commands. Same surface —
  `convert`, `import`, `upload`, `schema generate|validate|…` — and Rails is
  still booted lazily.
- **Isolated test suites:** each gem runs its own no-Rails specs in a new CI
  job, while the existing job keeps running the Rails-integration specs.
2026-06-02 22:20:03 +02:00
Jarek Radosz 6ad878c973 DEV: Update bundler from 2.6.4 to 4.0.11 (#40441) 2026-06-01 21:35:53 +02:00
SamandRafael Silva fa54f62348 FEATURE: extract text from document uploads for LLM prompts (#39634)
Document attachments (doc, docx, xls, xlsx, rtf, csv, md, txt) are now
converted to text before being included in LLM prompts, instead of
being forwarded as raw base64 payloads. PDFs remain the only format
sent as a raw upload, capped at 10MB.

New converters under lib/completions:

- DocToText shells out to antiword
- DocxToText parses OOXML directly with size and depth limits
- XlsToText shells out to xls2csv
- XlsxToText parses OOXML and shared strings into CSV-style text
- RtfToText is a custom RTF tokenizer with destination/group handling

Plain text formats (csv, md, txt) are read with a 1MB byte cap and
UTF-8 normalization. Extracted text is truncated to 100k characters,
with a preamble noting the original filename and size.

Dialect trimming now uses token-aware truncation against a per-message
budget so large extracted documents collapse cleanly under the prompt
limit, rather than the previous step-based slicing of raw content.

Other changes:

- LlmModel.normalize_attachment_types is shared with UploadEncoder and
  collapses "markdown" to "md" so the canonical extension is consistent
  across model config, UI defaults, and encoder output
- ai-llm-attachment-types adds csv, xls, xlsx to the default choices
- Locale strings clarify that vision controls images and
  allowed_attachment_types controls documents

---------

Co-authored-by: Rafael Silva <xfalcox@gmail.com>
2026-05-05 08:16:23 +10:00
Alan Guo Xiang Tanandducks fae7cd138d FEATURE: Add opt-in IAM role assumption for S3 operations (#39174)
This commit adds support for assuming an IAM role when performing S3
operations. When `s3_role_arn` is configured alongside static access
keys, Discourse will use AWS STS AssumeRole to obtain temporary, scoped
credentials instead of using the static keys directly.

Two new settings are introduced:

1. `s3_role_arn`: The ARN of the IAM role to assume.
2. `s3_role_session_name`: An optional session name which falls back to
the server hostname when blank.

Both are available as GlobalSettings (env vars) and SiteSettings (admin
UI). The feature is entirely opt-in, when `s3_role_arn` is blank
everything behaves exactly as before.

Co-authored-by: ducks <868959+ducks@users.noreply.github.com>
2026-04-15 04:29:50 +08:00
Jarek Radosz d4246c4bba DEV: Update rubyzip to 3.x (attempt 2) (#39151)
This reverts commit 6b4e048545.

The issues have been resolved.
2026-04-08 13:38:26 +02:00
Jarek Radosz 6b4e048545 Revert "DEV: Update rubyzip to 3.x (#38932)" (#39078)
This reverts commit b9bf992d61.
2026-04-02 18:36:59 +02:00
Loïc Guitaut 6b243ffdfd DEV: Remove Unicorn web server in favor of Pitchfork (#39032)
Pitchfork has been the default web server for some time now. This
removes Unicorn entirely to simplify the codebase and unblock future
improvements (like Rack 3).

Notable changes beyond the straightforward removal:

- `Discourse.after_unicorn_worker_fork` →
`Discourse.apply_worker_db_variables_overrides`: renamed and wired into
pitchfork.conf.rb's `after_worker_fork`. This actually *fixes*
per-worker DB variable overrides (`unicorn_worker_db_variables_*`) which
were never called under Pitchfork.
- `bin/ember-cli`: `--unicorn` flag renamed to `--server` (`-u` kept).
- `lib/demon/sidekiq.rb`: removed Unicorn-specific USR1/USR2 signal
handlers and `reopen_logs` (called `Unicorn::Util.reopen_logs`), which
were already dead code under Pitchfork.

Intentionally kept unchanged:
- `config/unicorn_launcher` (used by Docker images, separate effort)
- `docker_manager` plugin (separate repo)
- `UNICORN_*` env vars (renaming deferred)
- Rack < 3 constraint (separate PR)
2026-04-01 15:04:59 +02:00
Rafael dos Santos Silva 18a0a8daeb FEATURE: Add AWS Bedrock Converse API provider (#38903)
## Summary

Adds a new `aws_bedrock_converse` inference provider that uses the
official AWS SDK (`aws-sdk-bedrockruntime`) and the Converse API. This
runs alongside the existing `aws_bedrock` provider — fully additive,
zero risk to existing configurations.

### Why a new provider?

The existing `aws_bedrock` provider manually handles SigV4 signing, URL
construction, binary event stream decoding, and maintains a hardcoded
model ID mapping table. It only supports Claude and Nova models.

The new provider delegates all of this to the official AWS SDK, which
means:

- **Model-agnostic** — works with any model available on Bedrock
(Claude, Nova, Kimi, MiniMax, Mistral, Llama, DeepSeek, NVIDIA, Qwen,
GLM, etc.) without any model-specific code
- **Application Inference Profiles** — users can set cross-region
profiles (`us.anthropic.claude-sonnet-4-20250514-v1:0`) or application
inference profile ARNs directly as the model name
- **Bedrock API Key auth** — supports the new AWS Bedrock API keys
(Bearer token auth) in addition to IAM access keys, STS role assumption,
and automatic credential resolution from environment/instance profiles
- **No maintenance burden** — no model ID mapping table to update when
AWS adds new models, no manual SigV4 signing, no binary event stream
decoding
- **Native tools only** — no XML tool fallback; uses the Converse API's
built-in tool support

### Authentication options (priority order)

| Config | Auth method |
|---|---|
| `role_arn` set | STS AssumeRole (SigV4) |
| `access_key_id` set | Static IAM credentials (SigV4) |
| API key set (no access_key_id/role_arn) | Bearer token (Bedrock API
key) |
| Nothing set | SDK auto-resolves (env vars, instance profile, ECS task
role) |

### Features supported

- Streaming and non-streaming completions
- Native tool use with tool_choice (auto/any/specific tool)
- Structured output via Converse API's `output_config` (models that
support it)
- Extended thinking / adaptive thinking with signature preservation for
multi-turn
- Interleaved thinking with tool calls (thinking blocks preserved per
tool_call message)
- Prompt caching via `cache_point` blocks
- Effort parameter (low/medium/high/max)
- `extra_model_fields` provider param for arbitrary
`additionalModelRequestFields` (beta features like `anthropic_beta`, 1M
context, interleaved thinking)

### New files

- `lib/completions/endpoints/aws_bedrock_converse.rb` — endpoint using
`Aws::BedrockRuntime::Client`
- `lib/completions/dialects/converse.rb` — unified Converse API dialect
- `lib/completions/dialects/converse_tools.rb` — tool formatting
- `lib/completions/converse_message_processor.rb` — response processing
for SDK typed objects

## Tested against real Bedrock API

All tests performed using Bedrock API Key auth (Bearer token) against
live endpoints with 9 different models from 8 providers:

| Test | Claude Sonnet 4 | Claude Haiku 4.5 | Kimi K2.5 | MiniMax M2 |
DeepSeek 3.2 | NVIDIA Nemotron 3 120B | Qwen3 Next 80B | GLM 5 | Mistral
Small |
|---|---|---|---|---|---|---|---|---|---|
| Non-streaming text |  |  |
 |  |  |
 |  |  |
 |
| Streaming text |  |  |
 |  |  |
 |  |  |
 |
| Multi-turn conversation |  |  |
 |  |  |
 |  |  |
 |
| Tool use (non-streaming) |  |  |
 |  |  |
 |  |  |
 |
| Tool use (streaming) |  |  |
 |  |  |
 |  |  |  model
unsupported |
| Structured output (non-streaming) | — |  |  model
unsupported |  |  |
 |  |  |  model
unsupported |
| Structured output (streaming) | — |  |  model
unsupported |  |  |
 |  |  |  model
unsupported |
| Bearer token auth |  |  |
 |  |  |
 |  |  |
 |
| Cross-region inference profile |  |
 | — | — | — | — | — | — | — |
| Audit logging + token tracking |  |
 |  |  |
 |  |  |
 |  |

> **Notes:**
> - Claude Sonnet 4 structured output not tested — requires 4.5+ for
this feature and those cross-region profiles were not available in the
test region.
> - Kimi K2.5 and Mistral Small do not support Bedrock's native
structured output.
> - Mistral Small does not support streaming tool use.
> - All  results are model-level limitations, not code issues — the
Converse API correctly surfaces the error.

## Test plan

- [ ] Existing `aws_bedrock` provider tests pass (`bin/rspec
spec/lib/completions/endpoints/aws_bedrock_spec.rb`)
- [ ] New provider tests pass (`bin/rspec
spec/lib/completions/endpoints/aws_bedrock_converse_spec.rb`)
- [ ] Create an LLM model with provider "AWS Bedrock (Converse API)" in
admin UI
- [ ] Verify basic completion works with a Bedrock API key (just region
+ API key, no IAM keys needed)
- [ ] Verify tool use works in AI bot conversations
- [ ] Verify structured output works with a supported model (Claude
Haiku 4.5+)
2026-03-30 12:37:30 -03:00
Alan Guo Xiang Tan 0303bf599e DEV: Move puma to test-only dependency (#38652)
Puma is no longer used as an application server in production or
development — Pitchfork is the default, invoked automatically via
`bin/unicorn`. The only remaining use of Puma is as Capybara's embedded
test server, where it runs in a single-process threaded mode to share
database transactions with the test suite.

This commit moves the `puma` gem from a top-level dependency into the
test group to reflect its actual usage. The production Puma
configuration file is removed since it is unused, and stale
`defined?(Puma)` checks and comments are cleaned up.
2026-03-18 05:23:12 +08:00
David Taylor 502aa3687c DEV: Bump sidekiq to v7.3.10 (#38601)
This tightens the connection_pool dependency, so that we don't upgrade
to an incompatible version
2026-03-16 09:21:52 +00:00
Jarek Radosz e54f0753ca DEV: Drop pry-stack_explorer gem (#38293)
the gem hasn't been updated in 5 years. also, the `debug` gem is now
recommended for debugging

`binding_of_caller` gem has not been directly used
2026-03-06 14:19:46 +01:00
Loïc Guitaut 7cf341eea7 DEV: Bump required Ruby version to 3.4 (#37819)
Now that our base images ship Ruby 3.4, our Gemfile should require it as
the minimum allowed version.
2026-02-13 17:34:47 +01:00
David Taylor 0f1c9574f2 DEV: Update stage_security_fixes rake task (#37346)
- Moves it from `version_bump.rake` to the new `release.rake`. This is
the last thing which was pending upgrade, so we can now delete the whole
`version_bump.rake` file & spec

- Adds support for release/* branches

- Adds an interactive prompt to choose which security PRs to include

- Creates & merges the PR using `gh` CLI. Less manual work.
2026-01-28 15:12:29 +00:00
Michael Brown e935afbb62 DEV: always load the excon gem
We ran into trouble with MethodProfiler referencing excon without having loaded
it as it was relying on an initialiser to load it and depending on the side
effect.

If the excon gem is going to be loaded anyways, it doesn't make sense to have
it not loaded by default; this will be more robust.
2026-01-15 17:13:11 -05:00
Martin BrennanandRégis Hanol d2252b5cd3 DEV: Remove IMAP support in Discourse (#37002)
per
https://meta.discourse.org/t/imap-support-for-group-inboxes/160588/39?u=martin
we have been planning to remove IMAP support for a while,
because of its low usage and adoption, high complexity, and maintenance
burden.
This commit removes all IMAP-related code, including models,
jobs, services, and frontend components.

---------

Co-authored-by: Régis Hanol <regis@hanol.fr>
2026-01-12 10:07:26 +10:00
SamandMark McClure d31ca17c57 FEATURE: update mathjax to version 4.1 (#36814)
This upgrades to Mathjax 4.1 and latest katex
Implement rich text composer support for math
Adds support for /( )/ and /[ /] which was missing and very common now 
Removes math javascript from our repo

---------

Co-authored-by: Mark McClure <mcmcclur@unca.edu>
2026-01-09 08:24:31 +11:00
Loïc Guitaut a3fd7eb462 DEV: Add ruby-lsp-rspec (#35735)
We already have `ruby-lsp` and `ruby-lsp-rails`, so let’s add
`ruby-lsp-rspec` too, as it helps navigate specs quite a lot.
2025-10-31 12:20:20 +01:00
Loïc Guitaut 154224f109 DEV: Add Pitchfork alongside Unicorn (#35370)
This PR adds Pitchfork, as we want to move away from Unicorn ultimately.

Unicorn still boots by default, so there should be no disruption for
anyone.

To use Pitchfork instead of Unicorn, the `RUN_PITCHFORK` environment
variable must be set.
This will make `bin/rails s` and `config/unicorn_launcher` boot
Pitchfork. `unicorn_launcher` was patched because that way we can easily
switch between Unicorn and Pitchfork without having to change too many
things on the infra side.

The upgrader from the `docker_manager` plugin doesn’t work yet with
Pitchfork. This will be addressed in a future PR.
2025-10-24 11:08:23 +02:00
David Taylor e5ca38d2be DEV: Add ruby-lsp to development gems (#35170)
This significantly simplifies and speeds up using the ruby LSP in
development, since a separate 'composed bundle' is no longer needed.
2025-10-03 13:17:01 +02:00
Loïc Guitaut 2e47ee8a9e DEV: Fix Unicorn reloading (#34980)
Recently we replaced the `pry-byebug` gem by the `debug` one. It broke
the auto-reload mechanism we have in the development environment for
Unicorn.

Indeed, the `debug` gem by default will add an `at_exit` hook that will
wait for all its children to exit. It clashes with our own mechanism.

The workaround is to require `debug/prelude` instead of `debug`: the
`debugger` command and breakpoints will still work but without the hook
being set.
2025-09-25 15:51:02 +02:00
Rafael dos Santos Silva f608e0cd7e DEV: Move debugging gem from byebug to debug (#34827)
The official `debug` gem provides more modern debugging capabilities,
better syntax highlighting, the native ability to use vscode / chrome
debuggers and is maintained alongside ruby itself.
2025-09-17 10:08:52 -03:00
Loïc Guitaut eedda1f809 DEV: Enable Goldiloader by default
As things are going well with Goldiloader enabled, we can now enable it
by default.
2025-08-26 09:48:56 +02:00
Loïc Guitaut 1dc7295bba DEV: Add Goldiloader behind a setting (#34006)
This will allow us to try out Goldiloader and see how it performs.
2025-08-04 12:05:02 +02:00
Loïc Guitaut 3aa6842a5c DEV: Pin Rack to < 3 (#33891)
Upgrading Rack to version 3 will break Unicorn. When we don’t use Unicorn anymore, we’ll be able to upgrade to Rack 3.
2025-07-28 11:43:52 +02:00
aa2fb29fa6 DEV: Use rollup for theme JS compilation (#33103)
This commit is a complete reimplementation of our theme JS compilation
system.

Previously, we compiled theme JS into AMD `define` statements on a
per-source-file basis, and then concatenated them together for the
client. These AMD modules would integrate with those in Discourse core,
allowing two way access between core/theme modules. Going forward, we'll
be moving away from AMD, and towards native ES modules in core. Before
we can do that, we need to stop relying on AMD as the 'glue' between
core and themes/plugins.

This change introduces Rollup (running in mini-racer) as a compiler for
theme JS. This is configured to generate a single ES Module which
exports a list of 'compat modules'. Core `import()`s the modules for
each active theme, and adds them all to AMD. In future, this consumption
can be updated to avoid AMD entirely.

All module resolution within a theme is handled by Rollup, and does not
use AMD.

Import of core/plugin modules from themes are automatically transformed
into calls to a new `window.moduleBroker` interface. For now, this is a
direct interface to AMD. In future, this can be updated to point to real
ES Modules in core.

Despite the complete overhaul of the internals, this is not a breaking
change, and should have no impact on existing themes. If any
incompatibilities are found, please report them on
https://meta.discourse.org.

---------

Co-authored-by: Jarek Radosz <jarek@cvx.dev>
Co-authored-by: Chris Manson <chris@manson.ie>
2025-07-25 12:02:29 +01:00
Blake Erickson af3abb54e3 FEATURE: Add support for aws MediaConvert (#33092)
When enabled this will convert uploaded videos to a standard format that should
be playable on all devices and browsers.

The goal of this feature is to prevent codec playback issues that
sometimes can occur with video uploads.

It uses an adapter pattern, so that other services for video conversion
could be easily added in the future.
2025-07-23 11:58:33 -06:00
Sam fcd022b5b3 DEV: revert hiredis upgrade (#33785)
Revert hiredis upgrade due to .info failing on some environments

https://meta.discourse.org/t/rebuild-error/375387
2025-07-23 18:25:05 +10:00
Sam 471f0d0cd4 PERF: switch redis driver to hiredis (#33773)
Adds high performance redis driver, see:
https://github.com/redis-rb/redis-client

It is automatically configured.
2025-07-23 12:27:39 +10:00
Jarek Radosz 7d4974b2bb DEV: Unlock gem versions (#33753)
let's see what shakes out in the next dependabot run
2025-07-22 19:49:20 +02:00
Jarek Radosz 1821e2b900 DEV: Move discourse-ai to core (#33749)
https://meta.discourse.org/t/373574

Internal `/t/-/156778`
2025-07-22 15:07:59 +02:00
Jarek Radosz 97b1f04f8d DEV: Move discourse-github to core (#33749)
https://meta.discourse.org/t/373574

Internal `/t/-/156778`
2025-07-22 15:07:59 +02:00
Loïc Guitaut 0eab7daea4 DEV: Upgrade Rails to version 8.0.2
- Migrated from annotate to annotaterb as the former is not maintained
  anymore.
- Dropped our `fast_pluck` patch as the default `pluck` implementation
  seems now faster.
2025-07-22 09:59:44 +02:00
Jarek Radosz ff7a0f2391 DEV: Move discourse-subscriptions to core (#33570)
https://meta.discourse.org/t/373574

Internal `/t/-/156778`
2025-07-15 16:38:05 +02:00
Jarek Radosz 5e86f2c21f DEV: Move discourse-zendesk-plugin to core (#33570)
https://meta.discourse.org/t/373574

Internal `/t/-/156778`
2025-07-15 16:38:05 +02:00
David Taylor 60b3a216e9 DEV: Drop fakeweb dependency (#33069)
This was only be `require`'d by one official plugin
(discourse-perspective), but was causing failures in specs of unrelated
plugins. See
https://github.com/discourse/discourse-perspective-api/pull/110
2025-06-04 14:29:25 +01:00
Ted Johansson 2eb459116c DEV: Update Bullet to latest version (#32904)
A few minor versions of Bullet were incompatible with Discourse because we use our own content security policy middleware.

This has now been fixed upstream and released in 8.0.7.
2025-06-04 09:40:06 +08:00
Ted Johansson 29eb2eb096 FIX: Lock Bullet gem to a compatible version (#32724)
Because we use a custom ContentSecurityPolicy middleware, the latest versions (> 8.0.3) of Bullet error out on load.

This PR locks the gem to the latest compatible version.
2025-05-15 10:03:24 +08:00
Joffrey JAFFEUX b6aad28ccf DEV: replace selenium driver with playwright (#31977)
This commit is replacing the system specs driver (selenium) by
Playwright: https://playwright.dev/

We are still using Capybara to write the specs but they will now be run
by Playwright. To achieve this we are using the non official ruby
driver: https://github.com/YusukeIwaki/capybara-playwright-driver

### Notable changes

- `CHROME_DEV_TOOLS` has been removed, it's not working well with
playwright use `pause_test` and inspect browser for now.

- `fill_in` is not generating key events in playwright, use `send_keys`
if you need this.

### New spec options

#### trace

Allows to capture a trace in a zip file which you can load at
https://trace.playwright.dev or locally through `npx playwright
show-trace /path/to/trace.zip`

_Example usage:_

```ruby
it "shows bar", trace: true do
  visit("/")

  find(".foo").click

  expect(page).to have_css(".bar")
end
```

#### video

Allows to capture a video of your spec.

_Example usage:_

```ruby
it "shows bar", video: true do
  visit("/")

  find(".foo").click

  expect(page).to have_css(".bar")
end
```

### New env variable

#### PLAYWRIGHT_SLOW_MO_MS

Allow to force playwright to wait DURATION (in ms) at each action.

_Example usage:_

```
PLAYWRIGHT_SLOW_MO_MS=1000 rspec foo_spec.rb
```

#### PLAYWRIGHT_HEADLESS

Allow to be in headless mode or not. Default will be headless.

_Example usage:_

```
PLAYWRIGHT_HEADLESS=0 rspec foo_spec.rb # will show the browser
```

### New helpers

#### with_logs

Allows to access the browser logs and check if something specific has
been logged.

_Example usage:_

```ruby
with_logs do |logger|
  # do something

  expect(logger.logs.map { |log| log[:message] }).to include("foo")
end
```

#### add_cookie

Allows to add a cookie on the browser session.

_Example usage:_

```ruby
add_cookie(name: "destination_url", value: "/new")
```

#### get_style

Get the property style value of an element.

_Example usage:_

```ruby
expect(get_style(find(".foo"), "height")).to eq("200px")
```

#### get_rgb_color

Get the rgb color of an element.

_Example usage:_

```ruby
expect(get_rgb_color(find("html"), "backgroundColor")).to eq("rgb(170, 51, 159)")
```
2025-05-06 10:44:14 +02:00
David Taylor e7450cc6da DEV: Migrate from sprockets to propshaft for assets (#32475)
We are no longer using any of the transpilation/bundling features of
Sprockets. We only use it to serve assets in development, and then
collect & fingerprint them in production. This commit switches us to use
the more modern "Propshaft" gem for that functionality.

Propshaft is much simpler than Sprockets. Instead of taking a
combination of paths + "precompile" list, Propshaft simply assumes all
files in the configured directory are required in production. Previously
we had some base paths configured quite high in the directory structure,
and then only precompiled selected assets within the directory. That's
no longer possible, so this commit refactors those places (mostly
plugin-related) to use dedicated directories under
`app/assets/generated/`.

Another difference is that Propshaft applies asset digests in
development as well as production. This is great for caching & dev/prod
consistency, but does mean some small changes were required in tests.

We previously had some freedom-patches applied to Sprockets. Some of
those had to be ported across to Propshaft. We now have three patches:

1. Skip adding digest hashes to webpack-generated chunks (which are
already digested, and referred to from other js files)

2. Avoid raising errors for missing assets in test mode. We don't always
compile assets before running basic RSpec tests.

3. Maintain relative paths for sourcemap URLs, so that files don't need
to be recompiled depending on their CDN path

Significant refactors are made to the `assets.rake` and `s3.rake` tasks,
which rely on implementation details of Sprockets/Propshaft.
2025-04-30 08:59:32 +01:00
David Taylor c62a4a4759 PERF: Compile main locale bundles just-in-time (#32335)
Previously all locale bundles would be built & compressed during
assets:precompile. For most sites, only one of these languages was
actually used, so this is fairly wasteful.

This commit moves the main locale bundle into the
ExtraLocalesController, which has recently undergone many improvements
to make it more efficient. This allows locale files to be bundled "just
in time" when they're first accessed.

Now that brotli level=6 is enabled for these assets in our nginx config,
this change should have no impact on the locale bundle size.
2025-04-28 10:31:27 +01:00
Loïc Guitaut 2ed31fea64 DEV: Upgrade the Redis gem to v5.4 2025-03-19 14:34:00 +01:00
Ted Johansson e87bfad23b Revert "DEV: Replace Rinku native gem with PrettyText" (#31692)
Reverts discourse/discourse#31557

This is causing excessive spacing due to the addition of empty `<p>`
tags. Revert first while we fix that.
2025-03-07 12:01:22 +10:00
Sam 503645dc34 DEV: bump to major release of mini_racer (#31640)
Only notable changes is that we added
internal support for TimeWithZone which
was absent from previous release

We also improved error messages for T_OBJECT
2025-03-05 12:06:36 +11:00