Commit Graph

6785 Commits

Author SHA1 Message Date
Sergey Kandaurov
01ed3c3591 Use .Mt to mark up email addresses. 2020-11-19 17:15:22 +00:00
Igor Ippolitov
7e1637a316 Core: "-e" command line option.
When installing or running from a non-root user it is sometimes required to
override default, compiled in error log path. There was no way to do this
without rebuilding the binary (ticket #147).

This patch introduced "-e" command line option which allows one to override
compiled in error log path.
2020-11-19 16:59:00 +00:00
Maxim Dounin
23597e97f5 Configure: shared sources for addon modules.
Addon modules, both dynamic and static, can now use shared source files.
Shared sources result in only one make rule even if specified several
times in different modules.
2020-11-10 17:13:20 +03:00
Maxim Dounin
3030907f5f Configure: initialization of NGX_ADDON_SRCS. 2020-11-10 17:13:17 +03:00
Maxim Dounin
82a56ff85a Configure: style. 2020-11-10 17:13:14 +03:00
Maxim Dounin
a3b5ccd056 Request body: removed error assumption (ticket #2058).
Before introduction of request body filter in 42d9beeb22db, the only
possible return code from the ngx_http_request_body_filter() call
without actual buffers was NGX_HTTP_INTERNAL_SERVER_ERROR, and
the code in ngx_http_read_client_request_body() hardcoded the only
possible error to simplify the code of initial call to set rb->rest.

This is no longer true after introduction of request body filters though,
as a request body filter might need to return other errors, such as 403.
Fix is to preserve the error code actually returned by the call
instead of assuming 500.
2020-11-09 22:41:54 +03:00
Maxim Dounin
8ed303c055 Request body: improved logging.
Added logging before returning NGX_HTTP_INTERNAL_SERVER_ERROR if there
are busy buffers after a request body flush.  This should never happen
with current code, though bugs can be introduced by 3rd party modules.
Make sure debugging will be easy enough.
2020-11-09 22:40:53 +03:00
Ruslan Ermilov
ad2b9944b0 SSL: fixed non-working SSL shutdown on lingering close.
When doing lingering close, the socket was first shut down for writing,
so SSL shutdown initiated after lingering close was not able to send
the close_notify alerts (ticket #2056).

The fix is to call ngx_ssl_shutdown() before shutting down the socket.
2020-11-06 23:44:54 +03:00
Ruslan Ermilov
aad0d1bf1c Removed dead code from ngx_http_set_keepalive().
The code removed became dead after 98f03cd8d6cc (0.8.14),
circa when the request reference counting was introduced.
2020-11-06 23:44:47 +03:00
Vladimir Homutov
3c04273733 Core: added format specifiers to output binary data as hex.
Now "s", "V", and "v" format specifiers may be prefixed with "x" (lowercase)
or "X" (uppercase) to output corresponding data in hexadecimal format.

In collaboration with Maxim Dounin.
2020-10-28 10:56:11 +03:00
Vladimir Homutov
c566d79993 Version bump. 2020-11-05 22:37:27 +03:00
Maxim Dounin
db65797133 release-1.19.4 tag 2020-10-27 18:09:20 +03:00
Maxim Dounin
0db948d240 nginx-1.19.4-RELEASE 2020-10-27 18:09:20 +03:00
Maxim Dounin
9cdb278454 SSL: ssl_reject_handshake directive (ticket #195).
In some cases it might be needed to reject SSL handshake based on SNI
server name provided, for example, to make sure an invalid certificate
is not returned to clients trying to contact a name-based virtual server
without SSL configured.  Previously, a "ssl_ciphers aNULL;" was used for
this.  This workaround, however, is not compatible with TLSv1.3, in
particular, when using BoringSSL, where it is not possible to configure
TLSv1.3 ciphers at all.

With this change, the ssl_reject_handshake directive is introduced,
which instructs nginx to reject SSL handshakes with an "unrecognized_name"
alert in a particular server block.

For example, to reject handshake with names other than example.com,
one can use the following configuration:

    server {
        listen 443 ssl;
        ssl_reject_handshake on;
    }

    server {
        listen 443 ssl;
        server_name example.com;
        ssl_certificate example.com.crt;
        ssl_certificate_key example.com.key;
    }

The following configuration can be used to reject all SSL handshakes
without SNI server name provided:

    server {
        listen 443 ssl;
        ssl_reject_handshake on;
    }

    server {
        listen 443 ssl;
        server_name ~^;
        ssl_certificate example.crt;
        ssl_certificate_key example.key;
    }

Additionally, the ssl_reject_handshake directive makes configuring
certificates for the default server block optional.  If no certificates
are configured in the default server for a given listening socket,
certificates must be defined in all non-default server blocks with
the listening socket in question.
2020-10-22 18:02:28 +03:00
Maxim Dounin
f9a37243c9 Stream: proxy_ssl_conf_command directive.
Similarly to ssl_conf_command, proxy_ssl_conf_command can be used to
set arbitrary OpenSSL configuration parameters as long as nginx is
compiled with OpenSSL 1.0.2 or later, when connecting to upstream
servers with SSL.  Full list of available configuration commands
can be found in the SSL_CONF_cmd manual page
(https://www.openssl.org/docs/man1.1.1/man3/SSL_CONF_cmd.html).
2020-10-22 18:00:27 +03:00
Maxim Dounin
9a72ac799c Upstream: proxy_ssl_conf_command and friends.
Similarly to ssl_conf_command, proxy_ssl_conf_command (grpc_ssl_conf_command,
uwsgi_ssl_conf_command) can be used to set arbitrary OpenSSL configuration
parameters as long as nginx is compiled with OpenSSL 1.0.2 or later,
when connecting to upstream servers with SSL.  Full list of available
configuration commands can be found in the SSL_CONF_cmd manual page
(https://www.openssl.org/docs/man1.1.1/man3/SSL_CONF_cmd.html).
2020-10-22 18:00:23 +03:00
Maxim Dounin
ac9c162282 SSL: ssl_conf_command directive.
With the ssl_conf_command directive it is now possible to set
arbitrary OpenSSL configuration parameters as long as nginx is compiled
with OpenSSL 1.0.2 or later.  Full list of available configuration
commands can be found in the SSL_CONF_cmd manual page
(https://www.openssl.org/docs/man1.1.1/man3/SSL_CONF_cmd.html).

In particular, this allows configuring PrioritizeChaCha option
(ticket #1445):

    ssl_conf_command Options PrioritizeChaCha;

It can be also used to configure TLSv1.3 ciphers in OpenSSL,
which fails to configure them via the SSL_CTX_set_cipher_list()
interface (ticket #1529):

    ssl_conf_command Ciphersuites TLS_CHACHA20_POLY1305_SHA256;

Configuration commands are applied after nginx own configuration
for SSL, so they can be used to override anything set by nginx.
Note though that configuring OpenSSL directly with ssl_conf_command
might result in a behaviour nginx does not expect, and should be
done with care.
2020-10-22 18:00:22 +03:00
Maxim Dounin
af05f05f91 Core: ngx_conf_set_keyval_slot() now accepts NGX_CONF_UNSET_PTR.
With this change, it is now possible to use ngx_conf_merge_ptr_value()
to merge keyval arrays.  This change actually follows much earlier
changes in ngx_conf_merge_ptr_value() and ngx_conf_set_str_array_slot()
in 1452:cd586e963db0 (0.6.10) and 1701:40d004d95d88 (0.6.22).

To preserve compatibility with existing 3rd party modules, both NULL
and NGX_CONF_UNSET_PTR are accepted for now.
2020-10-22 18:00:20 +03:00
Ruslan Ermilov
2ba670a925 Cpp test: added stream. 2020-10-13 07:44:09 +03:00
Maxim Dounin
9381ecb185 Limit req: unlocking of nodes on complex value errors.
Previously, if there were multiple limits configured, errors in
ngx_http_complex_value() during processing of a non-first limit
resulted in reference count leak in shared memory nodes of already
processed limits.  Fix is to explicity unlock relevant nodes, much
like we do when rejecting requests.
2020-10-08 17:44:34 +03:00
Maxim Dounin
1e92a0a4ce Mail: proxy_smtp_auth directive.
The proxy_smtp_auth directive instructs nginx to authenticate users
on backend via the AUTH command (using the PLAIN SASL mechanism),
similar to what is normally done for IMAP and POP3.

If xclient is enabled along with proxy_smtp_auth, the XCLIENT command
won't try to send the LOGIN parameter.
2020-10-03 21:04:57 +03:00
Maxim Dounin
8523ef8e86 Version bump. 2020-10-03 21:01:12 +03:00
Maxim Dounin
135b707ab0 release-1.19.3 tag 2020-09-29 17:32:10 +03:00
Maxim Dounin
3e299b1576 nginx-1.19.3-RELEASE 2020-09-29 17:32:10 +03:00
Maxim Dounin
249e126beb Updated OpenSSL used for win32 builds. 2020-09-29 15:56:16 +03:00
Maxim Dounin
e64f7fe7c9 Proxy: error checking for array init, missed in 7716:d6a5e14aa3e4.
Found by Coverity (CID 1467637).
2020-09-29 15:54:09 +03:00
Maxim Dounin
718d589091 Userid: userid_flags fixup.
In 7717:e3e8b8234f05, the 1st bit was incorrectly used.  It shouldn't
be used for bitmask values, as it is used by NGX_CONF_BITMASK_SET.

Additionally, special value "off" added to make it possible to clear
inherited userid_flags value.
2020-09-29 15:52:18 +03:00
Maxim Dounin
279efaab66 Resolver: improved error messages (ticket #2024). 2020-09-28 17:41:22 +03:00
Maxim Dounin
c511f3de3e Userid: userid_flags directive to set cookie flags. 2020-09-28 17:07:48 +03:00
Ruslan Ermilov
21b903f8e3 Proxy: added the "proxy_cookie_flags" directive. 2020-09-27 23:21:11 +03:00
Ruslan Ermilov
8b3f778cbc Proxy: changed interface of some internal functions.
This is in preparation for the next change.

Also, moved optimization from ngx_http_proxy_rewrite_regex_handler()
to ngx_http_proxy_rewrite().
2020-09-27 23:21:10 +03:00
Ruslan Ermilov
b2b8f226f1 Proxy: strengthen syntax checking for some directives.
The "false" parameter of the proxy_redirect directive is deprecated.
Warning has been emitted since c2230102df6f (0.7.54).

The "off" parameter of the proxy_redirect, proxy_cookie_domain, and
proxy_cookie_path directives tells nginx not to inherit the
configuration from the previous configuration level.

Previously, after specifying the directive with the "off" parameter,
any other directives were ignored, and syntax checking was disabled.

The syntax was enforced to allow either one directive with the "off"
parameter, or several directives with other parameters.

Also, specifying "proxy_redirect default foo" no longer works like
"proxy_redirect default".
2020-09-27 23:21:09 +03:00
Pavel Pautov
c85d6fec21 Stream: set module.
Adds 'set' directive to the stream server context.
2020-08-28 14:10:54 -07:00
Sergey Kandaurov
3bbeb1b8de SSL: abort handshake on SSL_set_SSL_CTX() errors.
In rare cases, such as memory allocation failure, SSL_set_SSL_CTX() returns
NULL, which could mean that a different SSL configuration has not been set.
Note that this new behaviour seemingly originated in OpenSSL-1.1.0 release.
2020-09-24 13:51:29 +01:00
Maxim Dounin
6c89d752c8 HTTP/2: run posted requests after reading body.
HTTP/2 code failed to run posted requests after calling the request body
handler, and this resulted in connection hang if a subrequest was created
in the body handler and no other actions were made.
2020-09-23 19:52:31 +03:00
Maxim Dounin
7c67ff7363 HTTP/2: fixed segfault on DATA frames after 400 errors.
If 400 errors were redirected to an upstream server using the error_page
directive, DATA frames from the client might cause segmentation fault
due to null pointer dereference.  The bug had appeared in 6989:2c4dbcd6f2e4
(1.13.0).

Fix is to skip such frames in ngx_http_v2_state_read_data() (similarly
to 7561:9f1f9d6e056a).  With the fix, behaviour of 400 errors in HTTP/2
is now similar to one in HTTP/1.x, that is, nginx doesn't try to read the
request body.

Note that proxying 400 errors, as well as other early stage errors, to
upstream servers might not be a good idea anyway.  These errors imply
that reading and processing of the request (and the request headers)
wasn't complete, and proxying of such incomplete request might lead to
various errors.

Reported by Chenglong Zhang.
2020-09-23 19:50:49 +03:00
Maxim Dounin
e9a8612c13 SSL: disabled shutdown when there are buffered data.
This fixes "SSL_shutdown() failed (SSL: ... bad write retry)" errors
as observed on the second SSL_shutdown() call after SSL shutdown fixes in
09fb2135a589 (1.19.2), notably when HTTP/2 connections are closed due
to read timeouts while there are incomplete writes.
2020-09-16 18:26:25 +03:00
Maxim Dounin
f6c28f93af SSL: disabled shutdown after connection errors.
This fixes "SSL_shutdown() failed (SSL: ... bad write retry)" errors
as observed on the second SSL_shutdown() call after SSL shutdown fixes in
09fb2135a589 (1.19.2), notably when sending fails in ngx_http_test_expect(),
similarly to ticket #1194.

Note that there are some places where c->error is misused to prevent
further output, such as ngx_http_v2_finalize_connection() if there
are pending streams, or in filter finalization.  These places seem
to be extreme enough to don't care about missing shutdown though.
For example, filter finalization currently prevents keepalive from
being used.
2020-09-16 18:26:24 +03:00
Maxim Dounin
a1864c2586 SSL: fixed event handling during shutdown.
The c->read->ready and c->write->ready flags need to be cleared to ensure
that appropriate read or write events will be reported by kernel.  Without
this, SSL shutdown might wait till the timeout after blocking on writing
or reading even if there is a socket activity.
2020-09-16 18:26:23 +03:00
Maxim Dounin
82cf625ab5 SSL: workaround for incorrect SSL_write() errors in OpenSSL 1.1.1.
OpenSSL 1.1.1 fails to return SSL_ERROR_SYSCALL if an error happens
during SSL_write() after close_notify alert from the peer, and returns
SSL_ERROR_ZERO_RETURN instead.  Broken by this commit, which removes
the "i == 0" check around the SSL_RECEIVED_SHUTDOWN one:

https://git.openssl.org/?p=openssl.git;a=commitdiff;h=8051ab2

In particular, if a client closed the connection without reading
the response but with properly sent close_notify alert, this resulted in
unexpected "SSL_write() failed while ..." critical log message instead
of correct "SSL_write() failed (32: Broken pipe)" at the info level.

Since SSL_ERROR_ZERO_RETURN cannot be legitimately returned after
SSL_write(), the fix is to convert all SSL_ERROR_ZERO_RETURN errors
after SSL_write() to SSL_ERROR_SYSCALL.
2020-09-16 18:26:22 +03:00
Sergey Kandaurov
dc1b14126e Cache: keep c->body_start when Vary changes (ticket #2029).
If the variant hash doesn't match one we used as a secondary cache key,
we switch back to the original key.  In this case, c->body_start was kept
updated from an existing cache node overwriting the new response value.
After file cache update, it led to discrepancy between a cache node and
cache file seen as critical errors "file cache .. has too long header".
2020-09-09 19:26:27 +03:00
Sergey Kandaurov
ed0b19cdd4 Cache: reset c->body_start when reading a variant on Vary mismatch.
Previously, a variant not present in shared memory and stored on disk using a
secondary key was read using c->body_start from a variant stored with a main
key.  This could result in critical errors "cache file .. has too long header".
2017-08-04 19:37:37 +03:00
Sergey Kandaurov
b1274232db HTTP/2: rejecting invalid stream identifiers with PROTOCOL_ERROR.
Prodded by Xu Yang.
2020-09-02 23:13:36 +03:00
Roman Arutyunyan
554916301c Cache: ignore stale-if-error for 4xx and 5xx codes.
Previously the stale-if-error extension of the Cache-Control upstream header
triggered the return of a stale response for all error conditions that can be
specified in the proxy_cache_use_stale directive.  The list of these errors
includes both network/timeout/format errors, as well as some HTTP codes like
503, 504, 403, 429 etc.  The latter prevented a cache entry from being updated
by a response with any of these HTTP codes during the stale-if-error period.

Now stale-if-error only works for network/timeout/format errors and ignores
the upstream HTTP code.  The return of a stale response for certain HTTP codes
is still possible using the proxy_cache_use_stale directive.

This change also applies to the stale-while-revalidate extension of the
Cache-Control header, which triggers stale-if-error if it is missing.

Reported at
http://mailman.nginx.org/pipermail/nginx/2020-July/059723.html.
2020-07-29 13:28:04 +03:00
Roman Arutyunyan
d961012799 Version bump. 2020-08-14 12:45:52 +03:00
Maxim Dounin
b2d09a4cdd release-1.19.2 tag 2020-08-11 17:52:30 +03:00
Maxim Dounin
e4c6918e08 nginx-1.19.2-RELEASE 2020-08-11 17:52:30 +03:00
Maxim Dounin
348bc94086 Core: reusing connections in advance.
Reworked connections reuse, so closing connections is attempted in
advance, as long as number of free connections is less than 1/16 of
worker connections configured.  This ensures that new connections can
be handled even if closing a reusable connection requires some time,
for example, for a lingering close (ticket #2017).

The 1/16 ratio is selected to be smaller than 1/8 used for disabling
accept when working with accept mutex, so nginx will try to balance
new connections to different workers first, and will start reusing
connections only if this won't help.
2020-08-10 18:53:07 +03:00
Maxim Dounin
e240d88d44 Core: added a warning about reusing connections.
Previously, reusing connections happened silently and was only
visible in monitoring systems.  This was shown to be not very user-friendly,
and administrators often didn't realize there were too few connections
available to withstand the load, and configured timeouts (keepalive_timeout
and http2_idle_timeout) were effectively reduced to keep things running.

To provide at least some information about this, a warning is now logged
(at most once per second, to avoid flooding the logs).
2020-08-10 18:52:59 +03:00
Maxim Dounin
eae2b2fdf1 SSL: disabled sending shutdown after ngx_http_test_reading().
Sending shutdown when ngx_http_test_reading() detects the connection is
closed can result in "SSL_shutdown() failed (SSL: ... bad write retry)"
critical log messages if there are blocked writes.

Fix is to avoid sending shutdown via the c->ssl->no_send_shutdown flag,
similarly to how it is done in ngx_http_keepalive_handler() for kqueue
when pending EOF is detected.

Reported by Jan Prachař
(http://mailman.nginx.org/pipermail/nginx-devel/2018-December/011702.html).
2020-08-10 18:52:34 +03:00