Previously, an initgroups() failure was only logged, and the worker
process went on with the supplementary groups inherited from the master
process, since setgid() does not change them. Anything relying on the
configured user's groups then failed at run time, far from the log
message, and the outcome depended on which worker hit the error.
Now such a failure is fatal, similarly to setgid() and setuid(). The
exception is EPERM, which is logged and ignored: it is returned when
supplementary groups cannot be changed at all, for example, in a user
namespace with setgroups denied, see user_namespaces(7), or under
seccomp.
This dates back to 42b12b34f (0.1.11).
Previously, r->headers_in.content_length_n was used, which is not 0 till
the request body is fully discarded, and the request might hang.
Adapted for nginx: the same change is applied to the HTTP/2 proxy
module (proxy_http_version 2), which is not present in freenginx and
used r->headers_in.content_length_n in the same way. With an HTTP/2
upstream, the stale length resulted in the request being rejected
by the upstream, since the stream was closed without the body.
Co-authored-by: Elijah Zupancic <e.zupancic@f5.com>
Signed-off-by: Elijah Zupancic <e.zupancic@f5.com>
Origin: <https://freenginx.org/hg/nginx/rev/51e0dc7137843feda75b497773f598d3836c60a1>
Previously, if ngx_pool_cleanup_add() failed, already incremented
connection counter was left as is, eventually resulting in unexpected
connection limiting or shared memory exhaustion. Fix is to decrement
the counter by calling the cleanup handler with data on stack.
Signed-off-by: Elijah Zupancic <e.zupancic@f5.com>
Origin: <https://freenginx.org/hg/nginx/rev/722e252fcabe6413eb63555890273d7b0e7fc132>
Previously, an invalid working_directory was only detected on worker
process startup: if chdir() failed, worker processes exited and were
not respawned, while "nginx -t" reported success.
Now the directory is checked during configuration parsing. Note that
this doesn't cover insufficient permissions for worker processes.
Previously, a relative path was resolved against the current working
directory of the master process, which depends on how nginx is started.
Now it is resolved relative to prefix, similar to pid and error_log.
The encoder wrote accumulated bits by casting the destination to
uint64_t *, or uint32_t * on 32-bit platforms, and assigning through
it, which requires the destination to be suitably aligned.
In HTTP/2 the destination is a temporary buffer from ngx_palloc() and
is always aligned. In HTTP/3 it points past a variable number of
prefix integer bytes, so its alignment is arbitrary and the store is
undefined behaviour.
Byte order conversion is now separated from the store, which is done
with ngx_memcpy().
The parser now emits NGX_JSON_KEY_ESCAPED and
NGX_JSON_VALUE_STRING_ESCAPED for strings holding escape sequences, so
consumers no longer rescan tokens.
On 32-bit platforms, ngx_atoi() fails on syntactically valid Integers
above 2147483647, and the whole Priority field was rejected. Now such
values are ignored, similar to Decimals.
The documented argument type for this operation is an integer boolean.
Passing 1 as u_long makes the call silently fail on 64-bit big-endian
platforms, as the kernel receives only high 4 bytes set to 0.
While both FIOASYNC and O_ASYNC are non-standard, the former is more
widespread and is already used in ngx_process.c.
Fixes build on Solaris and Illumos.
i2d_SSL_SESSION() may unexpectedly return a negative value on
memory allocation failure, even though the NULL buffer was passed,
notably in LibreSSL.
Reported by szzzy.
After SSL shutdown, lingering close reads the socket with recv(). With
kernel TLS receive offload, which OpenSSL enables for TLS 1.3 since
3.2.0, recv() fails with EIO on a record other than application data,
such as the "close notify" alert from the client, and this was logged
at the "alert" level.
Now ngx_ssl_shutdown() marks connections with kernel TLS receive
offload, and EIO on such connections is ignored, so the read ends as
at EOF.
Reported by soukichi on GitHub.
Closes: https://github.com/nginx/nginx/issues/651
The COPY and MOVE handler temporarily replaces r->uri with the
destination to map it to a path, restoring it only on success. When
ngx_http_map_uri_to_path() failed for the destination, the handler
returned without restoring r->uri, leaving it set to the client-supplied
destination. Later processing that derives a target from the request
URI, such as an error_page with a variable, then resolved against the
destination instead of the request target.
Accepted control connections discarded errno when setting close-on-exec,
nonblocking I/O, or signal ownership. The error log identified the failed
fcntl() operation but omitted the operating system reason.
Pass ngx_socket_errno to the three error logs, consistently with listener
setup and inherited socket handling.
The I/O completion ports event method has never been completed: the
module is compiled into Windows builds, but is not documented or
supported.
Prodded by Marek Ludha.
A zero length field value in encoder stream instructions left the value
data pointer NULL, passed then to ngx_memcpy() in ngx_http_v3_insert().
For the reasons outlined in b0a4d0fb8, it is now set to an empty string.
Previously, after a "rewrite" had changed the URI, and before a new
location was chosen, the new URI might not match the current location
prefix. This becomes a problem when the location uses "alias", which
is why normally such inconsistency is detected by checking the
r->valid_location flag. Meanwhile this flag only handles the
inconsistency when using the "break" directive or parameter. However
in absence of those, there's still an inconsistency window from URI
change up until the end of the current set of rewrite directives.
URI to path mapping within this window might result in a buffer
overflow if location's prefix was longer than the rewritten URI.
The following example resulted in a buffer overflow with the request
"/foo/1":
location /foo/ {
alias /bar/;
rewrite ^/foo/.*$ /x;
return 200 $request_filename;
}
Alternatively, $request_filename might be used in access log format,
"set", "if", "add_header", etc.
The fix is to reject URI mapping in ngx_http_map_uri_to_path() if the
r->uri_changed flag is set.
In b76754049 buffer overrun protection in script was introduced.
However in ngx_http_script_run() data could still spill over to the
reserved area. This had no visible effects except the return value
which could extend beyond the maximum formatted string size. This
value is however currently unused in the nginx code.
Later in a8289aa69 string length recalculation was implemented to
protect against buffer underwrite. Considering a possible spillover to
the reserved area, the final string size could become larger than the
allocated buffer size. This could later result in writing beyond
the buffer boundaries. Maximum buffer overflow is the size of the
reserved area.
This could affect calculating document root from variables in HTTP
or using access logging in HTTP or Stream. In the case of access
logging a possible impact is limited by an off-by-one write of a null
byte.
Reported by Matt Suiche of Tolmo Inc.
The per-worker ticket key mirror is allocated with three slots
(ngx_array_push_n(keys, 3)), matching the shared memory
ticket_keys[3] array and the decrypt callback, which scans all
keys->nelts slots by key name. Key rotation, however, synced
only two slots from shared memory, leaving the third slot's
name, hmac_key, and aes_key uninitialized.
A ticket whose key name matched the uninitialized slot (which
holds unspecified pool memory, all zero in the best case) could
therefore be HMAC-verified and decrypted with uninitialized key
material, allowing a forged session.
Sync all keys->nelts slots, as ngx_ssl_ticket_keys_cleanup()
already assumes.
Reported by PhiNode Labs.
ngx_http_v3_parse_data() stored the remaining length of a skipped
(unknown) frame in st->length -- the same field the request body
filter uses for the number of DATA payload bytes to pass on as
request body. When an unknown frame was split across parser
invocations, the skip was interrupted with st->length holding the
skip remainder; on the next buffer the body filter, which calls
the parser only while st->length == 0, treated that remainder as
request body.
When proxying, this could result in sending unframed bytes, potentially
allowing for bypass of client_max_body_size, which is enforced only on
the DATA path, or an injection.
The fix is to keep unknown frame payload in a separate st->skip counter.
Reported by Hansol Lee of GOTROOT.
Each pending asynchronous operation holds a reference in
r->main->blocked, an 8-bit field, and the increments were not checked.
With 256 operations pending on a single request the counter wrapped to
zero, after which the request could be freed while those operations
still referenced its pool. This is the same overflow that 4ec67cf86
addressed for r->main->count, and left unaddressed for r->main->blocked.
Operations beyond the limit are now performed synchronously where a
synchronous path exists, and refused otherwise, so that the counter
cannot overflow.
Reported by cyberlancer.
The X-Accel-Redirect header might appear in cache files if its handling
is ignored with the "proxy_ignore_headers" directive. If the cache file
is later served with different settings, ngx_http_upstream_process_headers()
used to call ngx_http_upstream_finalize_request(NGX_DECLINED), which
is not expected to happen before the cleanup handler is installed and
resulted in ngx_http_finalize_request(NGX_DONE) (after 2b032e8df,
nginx 1.7.11), leading to unexpected request counter decrement, "request
count is zero" alerts, and segmentation faults.
Similarly, errors in ngx_http_upstream_process_headers() resulted in
ngx_http_upstream_finalize_request(NGX_HTTP_INTERNAL_SERVER_ERROR) being
called. This is also not expected to happen before the cleanup handler is
installed, and resulted in ngx_http_finalize_request(NGX_DONE) without
proper request finalization.
Fix is to avoid calling ngx_http_upstream_finalize_request() from
ngx_http_upstream_process_headers(), notably when the cleanup handler
is not yet installed. Errors are now simply return NGX_ERROR, so the
caller is responsible for proper finalization by calling either
ngx_http_finalize_request() or ngx_http_upstream_finalize_request().
And X-Accel-Redirect handling now does not call
ngx_http_upstream_finalize_request(NGX_DECLINED) if no cleanup handler
is installed.
Reported by Jiří Setnička
(https://mailman.nginx.org/pipermail/nginx-devel/2024-February/HWLYHOO3DDB3XTFT6X3GRMXIEJ3SJRUA.html).
The MaxMind DB format is now recognized, with the fallback to the
legacy format (reached End-of-Life in May 2022). Migration requires
switching to MMDB database files:
geoip_country /usr/share/GeoIP/GeoLite2-Country.mmdb;
geoip_city /usr/share/GeoIP/GeoLite2-City.mmdb;
geoip_org /usr/share/GeoIP/GeoLite2-ASN.mmdb;
Either libmaxminddb or libGeoIP is sufficient to build the modules;
configure fails if neither is found.
Notes on the MMDB lookups:
*) $geoip_country_code3 and $geoip_city_country_code3 are converted from
the alpha-2 codes with a compiled-in ISO 3166-1 table, as the MMDB
databases provide alpha-2 codes only.
*) $geoip_org looks up "autonomous_system_organization", "isp",
"organization", and "domain" in turn, to match the legacy ASNum, ISP,
Org, and Domain database editions.
*) $geoip_area_code has no MMDB counterpart and is never found.
*) $geoip_region and $geoip_region_name come from subdivisions[0], and
thus may differ in value from the legacy ones.
*) $geoip_dma_code comes from location.metro_code, which is deprecated
by MaxMind and may be absent.
Co-authored-by: Nitin Swami <n.swami@f5.com>
Before 63af971a1 (1.31.7), the NGINX_BPF_MAPS environment variable
was passed to the new binary on binary upgrade whenever the quic_bpf
module was built, even if "quic_bpf" was not enabled, with an empty
value if there were no QUIC BPF socket groups.
Since 63af971a1, such an empty value was treated as inherited maps,
and the new binary refused to start: with the "cannot be disabled
during binary upgrade" error if "quic_bpf" was not enabled, or with
a parsing error otherwise.
The fix is to ignore an empty value, which means there are no maps
to inherit.
The SO_COOKIE dependency is removed. QUIC BPF dispatching now uses
separate sockets for new and established connections: listener sockets
handle new connections, while per-worker sockets handle established
ones. Server connection IDs encode a random worker key, which BPF
uses to route established packets directly to the correct worker
socket. New connections are routed through listen keys using a
dual slot scheme that enables both reload and binary upgrade.
On reload, new workers register new sockets and keys while old workers
continue receiving established connections through their encoded keys.
On binary upgrade, the new master uses the opposite slot, and exiting
masters clear their entries so traffic shifts to the surviving
instance.
Co-authored-by: Roman Arutyunyan <arut@nginx.com>
Use bpf_skb_load_bytes() in the reuseport BPF helper when the required
QUIC header bytes are not available through direct packet access. This
keeps the direct access fast path while allowing the helper to parse the
flags byte, DCID length byte, and DCID key from non linear skb data.
A similar approach was implemented in [1].
[1] https://git.angie.software/web-server/angie/commit/4b527f5
Advertise SETTINGS_NO_RFC7540_PRIORITIES (0x9) in the initial SETTINGS
frame and order the output by the RFC 9218 effective priority instead of
the RFC 7540 dependency tree, which is removed in the same step.
Placing the switch last keeps it self-contained and easy to revert.
RFC 9218, Section 2.1: a server can send SETTINGS_NO_RFC7540_PRIORITIES
with a value of 1 to indicate that it will ignore RFC 7540 priority
signals. A received value other than 0 or 1 is a connection error.
Frames are ordered by urgency first, then non-incremental ahead of
incremental, then by stream id, which keeps streams of equal priority
in the order their requests arrived (RFC 8941, Section 4.2). The same
order is applied to ngx_http_v2_waiting_queue().
The RFC 7540 priority scheme is removed: PRIORITY frames are validated
for framing errors and then ignored, the priority fields of a HEADERS
frame are skipped, and the dependency tree (node parent/children/weight,
the dependencies queue, ngx_http_v2_set_dependency(),
ngx_http_v2_node_children_update() and the per-connection
priority_limit) is gone. RFC 9113, Sections 5.3.1 and 5.3.2, deprecate
stream prioritization: it proved complex, was implemented unevenly and
widely ignored, and its protocol elements are retained for wire
compatibility only.
With the tree gone, a buffered PRIORITY_UPDATE node no longer needs to
be attached to it, so the set_dependency() call from the buffering path
falls away, leaving the node parked in h2c->closed on its own.
Assisted-by: opencode:claude-opus-4.8
When nginx proxies a request over HTTP/2 and the upstream returns a
Priority response header, record it as the stream's server priority and
recompute the effective priority by merging it with the client's value
(RFC 9218, Section 8): a parameter present in the response overrides the
client's, a parameter absent leaves it unchanged. The client and server
inputs are kept separate so a later client PRIORITY_UPDATE does not
discard the server hint.
The merge affects only how this hop schedules the stream; the Priority
header forwarded to the client is left unchanged, so
$upstream_http_priority continues to report what the origin sent.
Assisted-by: opencode:claude-opus-4.8
Parse the RFC 9218 Priority request header, an RFC 8941 Structured
Fields Dictionary such as "u=3, i".
The header lines are collected during header block processing and, once
the block is complete, combined into a single value and parsed once in
ngx_http_v2_run_request(), as RFC 8941, Section 4.2, requires all field
lines of the same field to be combined before parsing. This mirrors how
cookies are handled. The header is still passed through to the request
so it stays visible to variables and proxying.
A malformed field is ignored, leaving the defaults. A PRIORITY_UPDATE
frame overrides the header (RFC 9218, Section 7): if a frame has already
been applied to the stream the header is not consulted.
Assisted-by: opencode:claude-opus-4.8
Implement the RFC 9218 PRIORITY_UPDATE frame (type 0x10). The frame is
sent on stream 0 and carries a prioritized stream id and a Priority
Field Value; it is validated for framing errors and dispatched from
ngx_http_v2_state_head().
A PRIORITY_UPDATE conveys a complete set of parameters (RFC 9218,
Section 7): omitted parameters reset to their defaults, so the value
replaces any earlier signal. A frame for a stream that already exists
is applied at once. A frame for a not-yet-created stream is buffered on
its node until the HEADERS frame arrives; the node is parked in
h2c->closed and, while the RFC 7540 tree is still present, attached to
the root as is done for a PRIORITY frame with a zero dependency. The
number of buffered signals is bounded by the node reuse budget.
A malformed value is logged and ignored, leaving the current priority
untouched. A value that does not fit the state buffer while the frame
is fragmented is skipped.
The signals are only collected here; the scheduler still uses the
RFC 7540 dependency tree until the switch to RFC 9218.
Assisted-by: opencode:claude-opus-4.8
Implement a strict RFC 8941 Structured Fields Dictionary parser for the
RFC 9218 Extensible Prioritization Scheme, shared by HTTP/2 (and, later,
HTTP/3).
New files:
- ngx_http_priority.h: the ngx_http_priority_t and
ngx_http_priority_state_t types and the parse/state API.
- ngx_http_priority.c: the parser plus ngx_http_priority_state_init()
and ngx_http_priority_state_update() (client/server merge).
The field is parsed strictly: a syntax error fails the whole field
(RFC 8941, Section 4.2, treats it as absent), and only on a successful
parse are the RFC 9218, Section 4, rules applied, ignoring unknown
members, out-of-range urgency and members of an unexpected type. On
error the caller keeps any previously signalled priority. The parser
bounds the number of members, inner-list items and parameters, and the
length of individual strings, tokens and byte sequences, that it will
accept.
The sources are built only with HTTP/2 (for now), alongside the Huffman
sources, rather than unconditionally into the core HTTP module.
Assisted-by: opencode:claude-opus-4.8
Since 2fb32ff (1.27.5), the connection frame threshold degraded to a
few or zero frames when using a small stream buffer. The previously
hardcoded minimum of 10000 frames is now used.
When set to zero, this resulted in a buffer with no room to read into.
As there is no meaningful interpretation of a zero-sized buffer, such
configuration is explicitly rejected.
Reported by kh-lee
OpenSSL 1.0.2+ validates IP addresses only against iPAddress
subjectAltName entries via X509_check_ip(). For older OpenSSL
versions, equivalent validation is now performed.
Note that dNSName entries are still logged when checking an IP
address, they are not used for matching.
Co-authored-by: Sergey Kandaurov <pluknet@nginx.com>
Previously, ASN1_STRING_length() was used as a length for "%*s" format
specifier, which is wrong, since string length is expected to be "size_t",
and ASN1_STRING_length() returns "int".
Origin: <https://freenginx.org/hg/nginx/rev/a540fac7e04a>
Signed-off-by: Eugene Grebenschikov <e.grebenshchikov@f5.com>
It is possible to receive application data together with socket close.
In this case, SSL_read() may return SSL_ERROR_SYSCALL with ECONNRESET,
specifically after OpenSSL sent TLSv1.3 tickets on a closed connection.
Before OpenSSL 4.0, a subsequent SSL_write() returned SSL_ERROR_SYSCALL
with EPIPE in this case. The behaviour was changed to remember the
error state and handle such a call as a protocol error, see for details
https://github.com/openssl/openssl/issues/30631. This makes SSL_write()
return SSL_ERROR_SSL with the empty error queue, which results in
"[crit] ... SSL_write() failed" errors, as caught by ssl_stapling.t.
The fix is to check the result of the last read stored in c->ssl->last,
to avoid calling SSL_write() after a failed read. For consistency, the
check is added to ngx_ssl_sendfile(): SSL_sendfile() resets the error
state the same way SSL_write() does, though it does not seem to be
reachable in http.
Based on a patch by Maxim Dounin.
Signed-off-by: Sergey Kandaurov <pluknet@nginx.com>
Origin: <https://freenginx.org/hg/nginx/rev/ff5becaf5444>
OpenSSL up to 3.0, when a TCP connection is closed by the peer, returns
SSL_ERROR_SYSCALL without an error queue and with errno set to 0. Since
OpenSSL 3.0 with SSL_OP_IGNORE_UNEXPECTED_EOF, which nginx sets, this is
reported as SSL_ERROR_ZERO_RETURN instead.
Closing a connection without the close_notify alert is incorrect, yet
common enough in the real world to be handled as a non-error condition.
Truncation attacks are expected to be prevented at the protocol level;
notably, they are not possible for HTTP/1.x requests at all, and only
possible for HTTP/1.x responses if neither Content-Length nor chunked
transfer encoding is used.
Previously, "ERR_peek_error() == 0" was used to detect this, which is
too broad: it also matches TCP-level errors, such as ECONNRESET, which
are better reported as errors, the way the plain TCP code does it in
ngx_unix_recv().
Now only a clean TCP close is treated as a clean connection close. Most
notably, connection resets now result in "SSL_read() failed" errors.
This change is a modified version of a patch by Maxim Dounin. The
modifications include resolved conflicts and the refined commit log.
Signed-off-by: Sergey Kandaurov <pluknet@nginx.com>
Origin: <https://freenginx.org/hg/nginx/rev/4c0b66b5046f>
Vadim Zhestikov
Maxim DouninandElijah Zupancic
Roman Arutyunyan
Sergey Kandaurov
최대우 (Daewoo Choi)
Konstantin Pavlov
Sai Krishna Kumar Reddy YADAMAKANTI
Sergey A. Osokin
Evgeny Shirykalov
nishat shabbir
Aleksei Bavshin
Oskar Eichler
dgprivate
Sergey KandaurovandNitin Swami
Ivan Ovchinnikov
Andrew Clayton
Sourav Bhowmik
Eugene GrebenschikovandSergey Kandaurov