If the script result turned out to be shorter than its predicted
length, the result string contained uninitialized bytes at the end.
The fix is to cut the result string by its actual size.
The following locations returned trailing garbage to the client with
URI "/1234abcd".
map $uri $foo {
~^/(?<bar>[0-9]).*$ $bar;
}
location ~(?<bar>[0-9]*)[a-z]*$ {
return 200 $1:$foo;
}
location ~(?<bar>[0-9]*)[a-z]*$ {
set $qux $1:$foo;
return 200 $qux;
}
With this change, all script copy operations now check if there is
enough room in the buffer. To do so, the script engine now provides the
e->end pointer, which specifies expected buffer end, and each copy
operation is checked against it with the ngx_http_script_check_length()
function.
The e->end pointer is optional and only checked when set, thus
introducing no incompatible API changes. All standard functions were
updated to use it, notably ngx_http_complex_value(), ngx_http_script_run(),
ngx_http_script_regex_start_code(), ngx_http_script_complex_value_code().
Direct script evaluation in the proxy, fastcgi, scgi, uwsgi, grpc proxy,
index, and try_files modules will be updated by a separate patch.
In particular, this catches issues as observed when evaluating variables
with side effects, such as in the following configuration:
map $uri $map {
~(?<capture>.*) $capture;
}
set $capture "";
set $temp "$capture $map";
As well as when evaluating non-cacheable variables, where length of a
variable might change between length and copy codes, such as in the
following configuration:
map prefix:$capture $map_volatile {
volatile;
~(?<capture>.*) $capture;
}
set $capture "";
set $temp "$map_volatile";
Similar changes were made in the stream module.
Signed-off-by: Roman Arutyunyan <arut@nginx.com>
Origin: <https://freenginx.org/hg/nginx/rev/d172f506ab5f>
Similarly to "listen ... sndbuf/rcvbuf", proxy_socket_sndbuf and
proxy_socket_rcvbuf can be used to set the SO_SNDBUF and SO_RCVBUF
socket options, respectively, on upstream connections. By default,
the values are left unchanged.
Closes: https://github.com/nginx/nginx/issues/1297
Complex value compilation scans strings for $1..$9 capture references.
Check that a byte after '$' is present before testing it, matching
ngx_str_t length semantics and avoiding reliance on NUL termination.
Apply the same check to both HTTP and stream implementations.
If the last variant was given an explicit percentage to reach 100%, it
did not cover hash values on the range boundary or due to accumulating
rounding error. Now this corresponds to the configuration with "*".
The new $ssl_sigalgs variable lists all signature algorithms
advertised by the client in its ClientHello message,
colon-separated, in analogy to $ssl_ciphers and $ssl_curves.
On OpenSSL 4.0+, SSL_get0_sigalg() is used, which returns proper
TLS scheme names (e.g. "rsa_pkcs1_sha256", "ecdsa_secp256r1_sha256")
and falls back to "0xHHHH" hex for unknown schemes.
On OpenSSL 1.0.2+, SSL_get_sigalgs() is used. Since OBJ_nid2sn()
on the combined psignhash NID is not injective across TLS
SignatureScheme codes (e.g. 0x0403 and 0x081a both yield
"ecdsa-with-SHA256"), raw SignatureScheme codes are reported to
avoid ambiguity and to match the hex fallback format of
SSL_get0_sigalg().
With older versions, as well as BoringSSL, LibreSSL, and AWS-LC,
the variable is empty.
Example configuration:
upstream u {
least_time connect | first_byte | last_byte [inflight];
server a;
server b;
}
Co-authored-by: Roman Arutyunyan <arut@nginx.com>
Added the proxy_ssl_alpn directive, which sets the list of protocols
to advertise via ALPN during upstream TLS handshakes. Each argument
is a complex value, so variables are accepted. In particular,
proxy_ssl_alpn $ssl_alpn_protocol;
inherits the protocol negotiated in the downstream TLS handshake.
When all evaluated values are empty or absent, no ALPN extension is
sent, equivalent to the directive not being set at all.
Closes#406 on GitHub.
When configured, it enables Multipath TCP support on a listen socket.
As of now it works on Linux starting with Linux 5.6 and glibc 2.32,
where it is enabled with an IPPROTO_MPTCP socket(2) protocol.
To avoid EADDRINUSE errors in bind() and listen() when transitioning
between sockets with different protocols, SO_REUSEPORT is set on both
sockets. See f7f1607bf for potential implications.
Based on previous work by Maxime Dourov and Anthony Doeraene.
Validation is rewritten to follow RFC 3986 host syntax, based on
ngx_http_parse_request_line(). The following is now rejected:
- the rest of gen-delims "#", "?", "@", "[", "]"
- other unwise delims <">, "<", ">", "\", "^", "`', "{", "|", "}"
- IP literals with a trailing dot, missing closing bracket, or pct-encoded
- a port subcomponent with invalid values
- characters in upper half
The function interface is changed to follow a common approach
to other functions used to setup SSL_CTX, with an exception of
"ngx_conf_t *cf" since it is not bound to nginx configuration.
This is required to report and propagate SSL_CTX_set_ex_data()
errors, as reminded by Coverity (CID 1668589).
Variables contain the IANA name of the signature scheme[1] used to sign
the TLS handshake.
Variables are only meaningful when using OpenSSL 3.5 and above, with older
versions they are empty. Moreover, since this data isn't stored in a
serialized session, variables are only available for new sessions.
[1] https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml
Requested by willmafh.
After f10bc5a763 the address was set to NULL only when local address was
not specified at all. In case complex value evaluated to an empty or
invalid string, local address remained unchanged. Currenrly this is not
a problem since the value is only set once. This change is a preparation
for being able to change the local address after initial setting.
The change introduces an SNI based virtual server selection during
early ClientHello processing. The callback is available since
OpenSSL 1.1.1; for older OpenSSL versions, the previous behaviour
is kept.
Using the ClientHello callback sets a reasonable processing order
for the "server_name" TLS extension. Notably, session resumption
decision now happens after applying server configuration chosen by
SNI, useful with enabled verification of client certificates, which
brings consistency with BoringSSL behaviour. The change supersedes
and reverts a fix made in 46b9f5d38 for TLSv1.3 resumed sessions.
In addition, since the callback is invoked prior to the protocol
version negotiation, this makes it possible to set "ssl_protocols"
on a per-virtual server basis.
To keep the $ssl_server_name variable working with TLSv1.2 resumed
sessions, as previously fixed in fd97b2a80, a limited server name
callback is preserved in order to acknowledge the extension.
Note that to allow third-party modules to properly chain the call to
ngx_ssl_client_hello_callback(), the servername callback function is
passed through exdata.
The ssl_certificate_compression directive allows to send compressed
server certificates. In OpenSSL, they are pre-compressed on startup.
To simplify configuration, the SSL_OP_NO_TX_CERTIFICATE_COMPRESSION
option is automatically cleared if certificates were pre-compressed.
SSL_CTX_compress_certs() may return an error in legitimate cases,
e.g., when none of compression algorithms is available or if the
resulting compressed size is larger than the original one, thus it
is silently ignored.
Certificate compression is supported in Chrome with brotli only,
in Safari with zlib only, and in Firefox with all listed algorithms.
It is supported since Ubuntu 24.10, which has OpenSSL with enabled
zlib and zstd support.
The actual list of algorithms supported in OpenSSL depends on how
the library was configured; it can be brotli, zlib, zstd as listed
in RFC 8879.
Passwords were not preserved in optimized SSL contexts, the bug had
appeared in d791b4aab (1.23.1), as in the following configuration:
server {
proxy_ssl_password_file password;
proxy_ssl_certificate $ssl_server_name.crt;
proxy_ssl_certificate_key $ssl_server_name.key;
location /original/ {
proxy_pass https://u1/;
}
location /optimized/ {
proxy_pass https://u2/;
}
}
The fix is to always preserve passwords, by copying to the configuration
pool, if dynamic certificates are used. This is done as part of merging
"ssl_passwords" configuration.
To minimize the number of copies, a preserved version is then used for
inheritance. A notable exception is inheritance of preserved empty
passwords to the context with statically configured certificates:
server {
proxy_ssl_certificate $ssl_server_name.crt;
proxy_ssl_certificate_key $ssl_server_name.key;
location / {
proxy_pass ...;
proxy_ssl_certificate example.com.crt;
proxy_ssl_certificate_key example.com.key;
}
}
In this case, an unmodified version (NULL) of empty passwords is set,
to allow reading them from the password prompt on nginx startup.
As an additional optimization, a preserved instance of inherited
configured passwords is set to the previous level, to inherit it
to other contexts:
server {
proxy_ssl_password_file password;
location /1/ {
proxy_pass https://u1/;
proxy_ssl_certificate $ssl_server_name.crt;
proxy_ssl_certificate_key $ssl_server_name.key;
}
location /2/ {
proxy_pass https://u2/;
proxy_ssl_certificate $ssl_server_name.crt;
proxy_ssl_certificate_key $ssl_server_name.key;
}
}
In OpenSSL, session resumption always happens in the default SSL context,
prior to invoking the SNI callback. Further, unlike in TLSv1.2 and older
protocols, SSL_get_servername() returns values received in the resumption
handshake, which may be different from the value in the initial handshake.
Notably, this makes the restriction added in b720f650b insufficient for
sessions resumed with different SNI server name.
Considering the example from b720f650b, previously, a client was able to
request example.org by presenting a certificate for example.org, then to
resume and request example.com.
The fix is to reject handshakes resumed with a different server name, if
verification of client certificates is enabled in a corresponding server
configuration.
A new directive "ssl_certificate_cache max=N [valid=time] [inactive=time]"
enables caching of SSL certificate chain and secret key objects specified
by "ssl_certificate" and "ssl_certificate_key" directives with variables.
Co-authored-by: Aleksei Bavshin <a.bavshin@nginx.com>
This simplifies merging protocol values after ea15896 and ebd18ec.
Further, as outlined in ebd18ec18, for libraries preceeding TLSv1.2+
support, only meaningful versions TLSv1 and TLSv1.1 are set by default.
While here, fixed indentation.
Previously, all upstream DNS entries would be immediately re-resolved
on config reload. With a large number of upstreams, this creates
a spike of DNS resolution requests. These spikes can overwhelm the
DNS server or cause drops on the network.
This patch retains the TTL of previous resolutions across reloads
by copying each upstream's name's expiry time across configuration
cycles. As a result, no additional resolutions are needed.
After configuration is reloaded, it may take some time for the
re-resolvable upstream servers to resolve and become available
as peers. During this time, client requests might get dropped.
Such servers are now pre-resolved using the "cache" of already
resolved peers from the old shared memory zone.
Specifying the upstream server by a hostname together with the
"resolve" parameter will make the hostname to be periodically
resolved, and upstream servers added/removed as necessary.
This requires a "resolver" at the "http" configuration block.
The "resolver_timeout" parameter also affects when the failed
DNS requests will be attempted again. Responses with NXDOMAIN
will be attempted again in 10 seconds.
Upstream has a configuration generation number that is incremented each
time servers are added/removed to the primary/backup list. This number
is remembered by the peer.init method, and if peer.get detects a change
in configuration, it returns NGX_BUSY.
Each server has a reference counter. It is incremented by peer.get and
decremented by peer.free. When a server is removed, it is removed from
the list of servers and is marked as "zombie". The memory allocated by
a zombie peer is freed only when its reference count becomes zero.
Co-authored-by: Roman Arutyunyan <arut@nginx.com>
Co-authored-by: Sergey Kandaurov <pluknet@nginx.com>
Co-authored-by: Vladimir Homutov <vl@nginx.com>
TLSv1 and TLSv1.1 are formally deprecated and forbidden to negotiate due
to insufficient security reasons outlined in RFC 8996.
TLSv1 and TLSv1.1 are disabled in BoringSSL e95b0cad9 and LibreSSL 3.8.1
in the way they cannot be enabled in nginx configuration. In OpenSSL 3.0,
they are only permitted at security level 0 (disabled by default).
The support is dropped in Chrome 84, Firefox 78, and deprecated in Safari.
This change disables TLSv1 and TLSv1.1 by default for OpenSSL 1.0.1 and
newer, where TLSv1.2 support is available. For older library versions,
which do not have alternatives, these protocol versions remain enabled.
Starting from TLSv1.1 (as seen since draft-ietf-tls-rfc2246-bis-00),
the "certificate_authorities" field grammar of the CertificateRequest
message was redone to allow no distinguished names. In TLSv1.3, with
the restructured CertificateRequest message, this can be similarly
done by optionally including the "certificate_authorities" extension.
This allows to avoid sending DNs at all.
In practice, aside from published TLS specifications, all supported
SSL/TLS libraries allow to request client certificates with an empty
DN list for any protocol version. For instance, when operating in
TLSv1, this results in sending the "certificate_authorities" list as
a zero-length vector, which corresponds to the TLSv1.1 specification.
Such behaviour goes back to SSLeay.
The change relaxes the requirement to specify at least one trusted CA
certificate in the ssl_client_certificate directive, which resulted in
sending DNs of these certificates (closes#142). Instead, all trusted
CA certificates can be specified now using the ssl_trusted_certificate
directive if needed. A notable difference that certificates specified
in ssl_trusted_certificate are always loaded remains (see 3648ba7db).
Co-authored-by: Praveen Chaudhary <praveenc@nvidia.com>
Previously handlers were mandatory. However they are not always needed.
For example, a server configured with ssl_reject_handshake does not need a
handler. Such servers required a fake handler to pass the check. Now handler
absence check is moved to runtime. If handler is missing, the connection is
closed with 500 code.
Passing from udp was not possible for the most part due to preread buffer
restriction. Passing to udp could occasionally work, but the connection would
still be bound to the original listen rbtree, which prevented it from being
deleted on connection closure.
Roman Arutyunyan
Maxim Dounin
Patrik Wall
Feng Wu
Sergey Kandaurov
Vadim Zhestikov
Vladimir HomutovandRoman Arutyunyan
Eugene Grebenschikov
Aleksei Bavshin
sftcd
Roman Semenov
Dmitry Plotnikov
蕭澧邦
Mini Hawthorne
Ruslan Ermilov
Dmitry Volyntsev
