On 32-bit platforms, ngx_atoi() fails on syntactically valid Integers
above 2147483647, and the whole Priority field was rejected. Now such
values are ignored, similar to Decimals.
The documented argument type for this operation is an integer boolean.
Passing 1 as u_long makes the call silently fail on 64-bit big-endian
platforms, as the kernel receives only high 4 bytes set to 0.
While both FIOASYNC and O_ASYNC are non-standard, the former is more
widespread and is already used in ngx_process.c.
Fixes build on Solaris and Illumos.
i2d_SSL_SESSION() may unexpectedly return a negative value on
memory allocation failure, even though the NULL buffer was passed,
notably in LibreSSL.
Reported by szzzy.
After SSL shutdown, lingering close reads the socket with recv(). With
kernel TLS receive offload, which OpenSSL enables for TLS 1.3 since
3.2.0, recv() fails with EIO on a record other than application data,
such as the "close notify" alert from the client, and this was logged
at the "alert" level.
Now ngx_ssl_shutdown() marks connections with kernel TLS receive
offload, and EIO on such connections is ignored, so the read ends as
at EOF.
Reported by soukichi on GitHub.
Closes: https://github.com/nginx/nginx/issues/651
The COPY and MOVE handler temporarily replaces r->uri with the
destination to map it to a path, restoring it only on success. When
ngx_http_map_uri_to_path() failed for the destination, the handler
returned without restoring r->uri, leaving it set to the client-supplied
destination. Later processing that derives a target from the request
URI, such as an error_page with a variable, then resolved against the
destination instead of the request target.
Accepted control connections discarded errno when setting close-on-exec,
nonblocking I/O, or signal ownership. The error log identified the failed
fcntl() operation but omitted the operating system reason.
Pass ngx_socket_errno to the three error logs, consistently with listener
setup and inherited socket handling.
The I/O completion ports event method has never been completed: the
module is compiled into Windows builds, but is not documented or
supported.
Prodded by Marek Ludha.
A zero length field value in encoder stream instructions left the value
data pointer NULL, passed then to ngx_memcpy() in ngx_http_v3_insert().
For the reasons outlined in b0a4d0fb8, it is now set to an empty string.
Previously, after a "rewrite" had changed the URI, and before a new
location was chosen, the new URI might not match the current location
prefix. This becomes a problem when the location uses "alias", which
is why normally such inconsistency is detected by checking the
r->valid_location flag. Meanwhile this flag only handles the
inconsistency when using the "break" directive or parameter. However
in absence of those, there's still an inconsistency window from URI
change up until the end of the current set of rewrite directives.
URI to path mapping within this window might result in a buffer
overflow if location's prefix was longer than the rewritten URI.
The following example resulted in a buffer overflow with the request
"/foo/1":
location /foo/ {
alias /bar/;
rewrite ^/foo/.*$ /x;
return 200 $request_filename;
}
Alternatively, $request_filename might be used in access log format,
"set", "if", "add_header", etc.
The fix is to reject URI mapping in ngx_http_map_uri_to_path() if the
r->uri_changed flag is set.
In b76754049 buffer overrun protection in script was introduced.
However in ngx_http_script_run() data could still spill over to the
reserved area. This had no visible effects except the return value
which could extend beyond the maximum formatted string size. This
value is however currently unused in the nginx code.
Later in a8289aa69 string length recalculation was implemented to
protect against buffer underwrite. Considering a possible spillover to
the reserved area, the final string size could become larger than the
allocated buffer size. This could later result in writing beyond
the buffer boundaries. Maximum buffer overflow is the size of the
reserved area.
This could affect calculating document root from variables in HTTP
or using access logging in HTTP or Stream. In the case of access
logging a possible impact is limited by an off-by-one write of a null
byte.
Reported by Matt Suiche of Tolmo Inc.
The per-worker ticket key mirror is allocated with three slots
(ngx_array_push_n(keys, 3)), matching the shared memory
ticket_keys[3] array and the decrypt callback, which scans all
keys->nelts slots by key name. Key rotation, however, synced
only two slots from shared memory, leaving the third slot's
name, hmac_key, and aes_key uninitialized.
A ticket whose key name matched the uninitialized slot (which
holds unspecified pool memory, all zero in the best case) could
therefore be HMAC-verified and decrypted with uninitialized key
material, allowing a forged session.
Sync all keys->nelts slots, as ngx_ssl_ticket_keys_cleanup()
already assumes.
Reported by PhiNode Labs.
ngx_http_v3_parse_data() stored the remaining length of a skipped
(unknown) frame in st->length -- the same field the request body
filter uses for the number of DATA payload bytes to pass on as
request body. When an unknown frame was split across parser
invocations, the skip was interrupted with st->length holding the
skip remainder; on the next buffer the body filter, which calls
the parser only while st->length == 0, treated that remainder as
request body.
When proxying, this could result in sending unframed bytes, potentially
allowing for bypass of client_max_body_size, which is enforced only on
the DATA path, or an injection.
The fix is to keep unknown frame payload in a separate st->skip counter.
Reported by Hansol Lee of GOTROOT.
Each pending asynchronous operation holds a reference in
r->main->blocked, an 8-bit field, and the increments were not checked.
With 256 operations pending on a single request the counter wrapped to
zero, after which the request could be freed while those operations
still referenced its pool. This is the same overflow that 4ec67cf86
addressed for r->main->count, and left unaddressed for r->main->blocked.
Operations beyond the limit are now performed synchronously where a
synchronous path exists, and refused otherwise, so that the counter
cannot overflow.
Reported by cyberlancer.
The X-Accel-Redirect header might appear in cache files if its handling
is ignored with the "proxy_ignore_headers" directive. If the cache file
is later served with different settings, ngx_http_upstream_process_headers()
used to call ngx_http_upstream_finalize_request(NGX_DECLINED), which
is not expected to happen before the cleanup handler is installed and
resulted in ngx_http_finalize_request(NGX_DONE) (after 2b032e8df,
nginx 1.7.11), leading to unexpected request counter decrement, "request
count is zero" alerts, and segmentation faults.
Similarly, errors in ngx_http_upstream_process_headers() resulted in
ngx_http_upstream_finalize_request(NGX_HTTP_INTERNAL_SERVER_ERROR) being
called. This is also not expected to happen before the cleanup handler is
installed, and resulted in ngx_http_finalize_request(NGX_DONE) without
proper request finalization.
Fix is to avoid calling ngx_http_upstream_finalize_request() from
ngx_http_upstream_process_headers(), notably when the cleanup handler
is not yet installed. Errors are now simply return NGX_ERROR, so the
caller is responsible for proper finalization by calling either
ngx_http_finalize_request() or ngx_http_upstream_finalize_request().
And X-Accel-Redirect handling now does not call
ngx_http_upstream_finalize_request(NGX_DECLINED) if no cleanup handler
is installed.
Reported by Jiří Setnička
(https://mailman.nginx.org/pipermail/nginx-devel/2024-February/HWLYHOO3DDB3XTFT6X3GRMXIEJ3SJRUA.html).
The MaxMind DB format is now recognized, with the fallback to the
legacy format (reached End-of-Life in May 2022). Migration requires
switching to MMDB database files:
geoip_country /usr/share/GeoIP/GeoLite2-Country.mmdb;
geoip_city /usr/share/GeoIP/GeoLite2-City.mmdb;
geoip_org /usr/share/GeoIP/GeoLite2-ASN.mmdb;
Either libmaxminddb or libGeoIP is sufficient to build the modules;
configure fails if neither is found.
Notes on the MMDB lookups:
*) $geoip_country_code3 and $geoip_city_country_code3 are converted from
the alpha-2 codes with a compiled-in ISO 3166-1 table, as the MMDB
databases provide alpha-2 codes only.
*) $geoip_org looks up "autonomous_system_organization", "isp",
"organization", and "domain" in turn, to match the legacy ASNum, ISP,
Org, and Domain database editions.
*) $geoip_area_code has no MMDB counterpart and is never found.
*) $geoip_region and $geoip_region_name come from subdivisions[0], and
thus may differ in value from the legacy ones.
*) $geoip_dma_code comes from location.metro_code, which is deprecated
by MaxMind and may be absent.
Co-authored-by: Nitin Swami <n.swami@f5.com>
Before 63af971a1 (1.31.7), the NGINX_BPF_MAPS environment variable
was passed to the new binary on binary upgrade whenever the quic_bpf
module was built, even if "quic_bpf" was not enabled, with an empty
value if there were no QUIC BPF socket groups.
Since 63af971a1, such an empty value was treated as inherited maps,
and the new binary refused to start: with the "cannot be disabled
during binary upgrade" error if "quic_bpf" was not enabled, or with
a parsing error otherwise.
The fix is to ignore an empty value, which means there are no maps
to inherit.
The SO_COOKIE dependency is removed. QUIC BPF dispatching now uses
separate sockets for new and established connections: listener sockets
handle new connections, while per-worker sockets handle established
ones. Server connection IDs encode a random worker key, which BPF
uses to route established packets directly to the correct worker
socket. New connections are routed through listen keys using a
dual slot scheme that enables both reload and binary upgrade.
On reload, new workers register new sockets and keys while old workers
continue receiving established connections through their encoded keys.
On binary upgrade, the new master uses the opposite slot, and exiting
masters clear their entries so traffic shifts to the surviving
instance.
Co-authored-by: Roman Arutyunyan <arut@nginx.com>
Use bpf_skb_load_bytes() in the reuseport BPF helper when the required
QUIC header bytes are not available through direct packet access. This
keeps the direct access fast path while allowing the helper to parse the
flags byte, DCID length byte, and DCID key from non linear skb data.
A similar approach was implemented in [1].
[1] https://git.angie.software/web-server/angie/commit/4b527f5
Advertise SETTINGS_NO_RFC7540_PRIORITIES (0x9) in the initial SETTINGS
frame and order the output by the RFC 9218 effective priority instead of
the RFC 7540 dependency tree, which is removed in the same step.
Placing the switch last keeps it self-contained and easy to revert.
RFC 9218, Section 2.1: a server can send SETTINGS_NO_RFC7540_PRIORITIES
with a value of 1 to indicate that it will ignore RFC 7540 priority
signals. A received value other than 0 or 1 is a connection error.
Frames are ordered by urgency first, then non-incremental ahead of
incremental, then by stream id, which keeps streams of equal priority
in the order their requests arrived (RFC 8941, Section 4.2). The same
order is applied to ngx_http_v2_waiting_queue().
The RFC 7540 priority scheme is removed: PRIORITY frames are validated
for framing errors and then ignored, the priority fields of a HEADERS
frame are skipped, and the dependency tree (node parent/children/weight,
the dependencies queue, ngx_http_v2_set_dependency(),
ngx_http_v2_node_children_update() and the per-connection
priority_limit) is gone. RFC 9113, Sections 5.3.1 and 5.3.2, deprecate
stream prioritization: it proved complex, was implemented unevenly and
widely ignored, and its protocol elements are retained for wire
compatibility only.
With the tree gone, a buffered PRIORITY_UPDATE node no longer needs to
be attached to it, so the set_dependency() call from the buffering path
falls away, leaving the node parked in h2c->closed on its own.
Assisted-by: opencode:claude-opus-4.8
When nginx proxies a request over HTTP/2 and the upstream returns a
Priority response header, record it as the stream's server priority and
recompute the effective priority by merging it with the client's value
(RFC 9218, Section 8): a parameter present in the response overrides the
client's, a parameter absent leaves it unchanged. The client and server
inputs are kept separate so a later client PRIORITY_UPDATE does not
discard the server hint.
The merge affects only how this hop schedules the stream; the Priority
header forwarded to the client is left unchanged, so
$upstream_http_priority continues to report what the origin sent.
Assisted-by: opencode:claude-opus-4.8
Parse the RFC 9218 Priority request header, an RFC 8941 Structured
Fields Dictionary such as "u=3, i".
The header lines are collected during header block processing and, once
the block is complete, combined into a single value and parsed once in
ngx_http_v2_run_request(), as RFC 8941, Section 4.2, requires all field
lines of the same field to be combined before parsing. This mirrors how
cookies are handled. The header is still passed through to the request
so it stays visible to variables and proxying.
A malformed field is ignored, leaving the defaults. A PRIORITY_UPDATE
frame overrides the header (RFC 9218, Section 7): if a frame has already
been applied to the stream the header is not consulted.
Assisted-by: opencode:claude-opus-4.8
Implement the RFC 9218 PRIORITY_UPDATE frame (type 0x10). The frame is
sent on stream 0 and carries a prioritized stream id and a Priority
Field Value; it is validated for framing errors and dispatched from
ngx_http_v2_state_head().
A PRIORITY_UPDATE conveys a complete set of parameters (RFC 9218,
Section 7): omitted parameters reset to their defaults, so the value
replaces any earlier signal. A frame for a stream that already exists
is applied at once. A frame for a not-yet-created stream is buffered on
its node until the HEADERS frame arrives; the node is parked in
h2c->closed and, while the RFC 7540 tree is still present, attached to
the root as is done for a PRIORITY frame with a zero dependency. The
number of buffered signals is bounded by the node reuse budget.
A malformed value is logged and ignored, leaving the current priority
untouched. A value that does not fit the state buffer while the frame
is fragmented is skipped.
The signals are only collected here; the scheduler still uses the
RFC 7540 dependency tree until the switch to RFC 9218.
Assisted-by: opencode:claude-opus-4.8
Implement a strict RFC 8941 Structured Fields Dictionary parser for the
RFC 9218 Extensible Prioritization Scheme, shared by HTTP/2 (and, later,
HTTP/3).
New files:
- ngx_http_priority.h: the ngx_http_priority_t and
ngx_http_priority_state_t types and the parse/state API.
- ngx_http_priority.c: the parser plus ngx_http_priority_state_init()
and ngx_http_priority_state_update() (client/server merge).
The field is parsed strictly: a syntax error fails the whole field
(RFC 8941, Section 4.2, treats it as absent), and only on a successful
parse are the RFC 9218, Section 4, rules applied, ignoring unknown
members, out-of-range urgency and members of an unexpected type. On
error the caller keeps any previously signalled priority. The parser
bounds the number of members, inner-list items and parameters, and the
length of individual strings, tokens and byte sequences, that it will
accept.
The sources are built only with HTTP/2 (for now), alongside the Huffman
sources, rather than unconditionally into the core HTTP module.
Assisted-by: opencode:claude-opus-4.8
Since 2fb32ff (1.27.5), the connection frame threshold degraded to a
few or zero frames when using a small stream buffer. The previously
hardcoded minimum of 10000 frames is now used.
When set to zero, this resulted in a buffer with no room to read into.
As there is no meaningful interpretation of a zero-sized buffer, such
configuration is explicitly rejected.
Reported by kh-lee
OpenSSL 1.0.2+ validates IP addresses only against iPAddress
subjectAltName entries via X509_check_ip(). For older OpenSSL
versions, equivalent validation is now performed.
Note that dNSName entries are still logged when checking an IP
address, they are not used for matching.
Co-authored-by: Sergey Kandaurov <pluknet@nginx.com>
Previously, ASN1_STRING_length() was used as a length for "%*s" format
specifier, which is wrong, since string length is expected to be "size_t",
and ASN1_STRING_length() returns "int".
Origin: <https://freenginx.org/hg/nginx/rev/a540fac7e04a>
Signed-off-by: Eugene Grebenschikov <e.grebenshchikov@f5.com>
It is possible to receive application data together with socket close.
In this case, SSL_read() may return SSL_ERROR_SYSCALL with ECONNRESET,
specifically after OpenSSL sent TLSv1.3 tickets on a closed connection.
Before OpenSSL 4.0, a subsequent SSL_write() returned SSL_ERROR_SYSCALL
with EPIPE in this case. The behaviour was changed to remember the
error state and handle such a call as a protocol error, see for details
https://github.com/openssl/openssl/issues/30631. This makes SSL_write()
return SSL_ERROR_SSL with the empty error queue, which results in
"[crit] ... SSL_write() failed" errors, as caught by ssl_stapling.t.
The fix is to check the result of the last read stored in c->ssl->last,
to avoid calling SSL_write() after a failed read. For consistency, the
check is added to ngx_ssl_sendfile(): SSL_sendfile() resets the error
state the same way SSL_write() does, though it does not seem to be
reachable in http.
Based on a patch by Maxim Dounin.
Signed-off-by: Sergey Kandaurov <pluknet@nginx.com>
Origin: <https://freenginx.org/hg/nginx/rev/ff5becaf5444>
OpenSSL up to 3.0, when a TCP connection is closed by the peer, returns
SSL_ERROR_SYSCALL without an error queue and with errno set to 0. Since
OpenSSL 3.0 with SSL_OP_IGNORE_UNEXPECTED_EOF, which nginx sets, this is
reported as SSL_ERROR_ZERO_RETURN instead.
Closing a connection without the close_notify alert is incorrect, yet
common enough in the real world to be handled as a non-error condition.
Truncation attacks are expected to be prevented at the protocol level;
notably, they are not possible for HTTP/1.x requests at all, and only
possible for HTTP/1.x responses if neither Content-Length nor chunked
transfer encoding is used.
Previously, "ERR_peek_error() == 0" was used to detect this, which is
too broad: it also matches TCP-level errors, such as ECONNRESET, which
are better reported as errors, the way the plain TCP code does it in
ngx_unix_recv().
Now only a clean TCP close is treated as a clean connection close. Most
notably, connection resets now result in "SSL_read() failed" errors.
This change is a modified version of a patch by Maxim Dounin. The
modifications include resolved conflicts and the refined commit log.
Signed-off-by: Sergey Kandaurov <pluknet@nginx.com>
Origin: <https://freenginx.org/hg/nginx/rev/4c0b66b5046f>
If COPY or MOVE methods are enabled, the DAV module assumes that it is
configured for the whole server, and does not try validate that the
destination URI is within the particular location. This, however, does
not work well if it is in fact configured in a non-root location, and
this location uses the "alias" directive, such as in the following
configuration:
location /prefix/ {
dav_methods COPY;
alias /foo/;
}
In particular, if the destination URI is shorter than the aliased
location prefix, calling ngx_http_map_uri_to_path() with such
destination URI used to result in segmentation faults (9739e755b). And
if the destination URI is longer than the aliased location prefix, but
does not match it, calling ngx_http_map_uri_to_path() resulted in
unexpected path values, including ones above the "alias" directory
specified.
The fix is to check destination URI prefix if "alias" is used, similarly
to what we do with "try_files". Additionally, the
ngx_http_map_uri_to_path() function was updated with additional sanity
checking to prevent similar issues.
This change is a modified version of a patch by Maxim Dounin. The
modifications include skipping the prefix check if the URI was
rewritten, which preserves the "alias" diagnostics of
ngx_http_map_uri_to_path().
Signed-off-by: Vadim Zhestikov <v.zhestikov@f5.com>
Origin: <https://freenginx.org/hg/nginx/rev/aae28b083279>
Instead of failing the handshake, it is now accepted and ignored like
any other unknown extension. Unlike stated in RFC 9001, Section 8.2,
this represents less strict behaviour following RFC 8446, Section 4.2.
With the OpenSSL compatibility layer, the transport parameters custom
TLS extension used to be registered only in SSL contexts of virtual
servers reachable over QUIC.
The ClientHello callback introduced in 0373fe5d9 (1.29.2) is called
by OpenSSL between collecting and parsing extensions, so the custom
extensions may not match the expected number if the SSL context was
replaced with SSL_set_SSL_CTX().
The fix is to register the extension in all SSL contexts, including
those that cannot be reached over QUIC, so that the array bounds are
preserved; the compatibility layer as a whole is initialized only
if the configuration has QUIC listeners. The keylog callback,
meaningful for QUIC connections, is kept on QUIC addresses.
Reported by Banny Liao.
The environment may be replaced while parsing configuration, as it
happens with the perl module, which calls ngx_set_environment() when
creating an interpreter. As a result, the control socket was not
inherited on binary upgrade.
The variable is now read before ngx_init_cycle(), similarly to
inherited listening sockets.
Previously, when looking up a nested location after a prefix location
match, the nested regex or predicate match error could be ignored if
more regex or predicate locations were configured at the current level.
Now such error leads to request finalization.
nishat shabbir
Aleksei Bavshin
Vadim Zhestikov
Oskar Eichler
dgprivate
Sergey Kandaurov
Roman Arutyunyan
Maxim Dounin
Sergey KandaurovandNitin Swami
Ivan Ovchinnikov
Sai Krishna Kumar Reddy YADAMAKANTIandRoman Arutyunyan
Andrew Clayton
Sourav Bhowmik
Eugene GrebenschikovandSergey Kandaurov
Ava HahnandPavel Pautov