Commit Graph

5349 Commits

Author SHA1 Message Date
Roman Arutyunyan
94fa2bb4cb Modules compatibility: down flag in ngx_peer_connection_t. 2019-01-31 17:25:03 +03:00
Ruslan Ermilov
34a8b4506a Use %s for errors returned from configuration parsing handlers. 2018-12-25 15:26:58 +03:00
Sergey Kandaurov
a05e7555ce Removed --test-build-eventport workaround for old FreeBSD versions. 2019-01-28 14:34:02 +00:00
Sergey Kandaurov
f153bcebb5 Fixed portability issues with union sigval.
AIO support in nginx was originally developed against FreeBSD versions 4-6,
where the sival_ptr field was named as sigval_ptr (seemingly by mistake[1]),
which made nginx use the only name available then.  The standard-complaint
name was restored in 2005 (first appeared in FreeBSD 7.0, 2008), retaining
compatibility with previous versions[2][3].  In DragonFly, similar changes
were committed in 2009[4], with backward compatibility recently removed[5].

The change switches to the standard name, retaining compatibility with old
FreeBSD versions.

[1] https://svnweb.freebsd.org/changeset/base/48621
[2] https://svnweb.freebsd.org/changeset/base/152029
[3] https://svnweb.freebsd.org/changeset/base/174003
[4] https://gitweb.dragonflybsd.org/dragonfly.git/commit/3693401
[5] https://gitweb.dragonflybsd.org/dragonfly.git/commit/7875042
2019-01-28 14:33:31 +00:00
Maxim Dounin
38196b8ba6 Win32: detection of connect() errors in select().
On Windows, connect() errors are only reported via exceptfds descriptor set
from select().  Previously exceptfds was set to NULL, and connect() errors
were not detected at all, so connects to closed ports were waiting till
a timeout occurred.

Since ongoing connect() means that there will be a write event active,
except descriptor set is copied from the write one.  While it is possible
to construct except descriptor set as a concatenation of both read and write
descriptor sets, this looks unneeded.

With this change, connect() errors are properly detected now when using
select().  Note well that it is not possible to detect connect() errors with
WSAPoll() (see https://daniel.haxx.se/blog/2012/10/10/wsapoll-is-broken/).
2019-01-24 22:00:44 +03:00
Maxim Dounin
40b74475d0 Win32: added WSAPoll() support.
WSAPoll() is only available with Windows Vista and newer (and only
available during compilation if _WIN32_WINNT >= 0x0600).  To make
sure the code works with Windows XP, we do not redefine _WIN32_WINNT,
but instead load WSAPoll() dynamically if it is not available during
compilation.

Also, sockets are not guaranteed to be small integers on Windows.
So an index array is used instead of NGX_USE_FD_EVENT to map
events to connections.
2019-01-24 21:51:21 +03:00
Maxim Dounin
3dcad9c5d6 Events: fixed copying of old events in poll init.
Previously, the code incorrectly assumed "ngx_event_t *" elements
instead of "struct pollfd".

This is mostly cosmetic change, as this code is never called now.
2019-01-24 21:50:37 +03:00
Roman Arutyunyan
12645b46e9 Stream: do not split datagrams when limiting proxy rate.
Previously, when using proxy_upload_rate and proxy_download_rate, the buffer
size for reading from a socket could be reduced as a result of rate limiting.
For connection-oriented protocols this behavior is normal since unread data will
normally be read at the next iteration.  But for datagram-oriented protocols
this is not the case, and unread part of the datagram is lost.

Now buffer size is not limited for datagrams.  Rate limiting still works in this
case by delaying the next reading event.
2018-12-27 19:37:34 +03:00
Roman Arutyunyan
36a0713244 Prevented scheduling events on a shared connection.
A shared connection does not own its file descriptor, which means that
ngx_handle_read_event/ngx_handle_write_event calls should do nothing for it.
Currently the c->shared flag is checked in several places in the stream proxy
module prior to calling these functions.  However it was not done everywhere.
Missing checks could lead to calling
ngx_handle_read_event/ngx_handle_write_event on shared connections.

The problem manifested itself when using proxy_upload_rate and resulted in
either duplicate file descriptor error (e.g. with epoll) or incorrect further
udp packet processing (e.g. with kqueue).

The fix is to set and reset the event active flag in a way that prevents
ngx_handle_read_event/ngx_handle_write_event from scheduling socket events.
2019-01-14 20:36:23 +03:00
Vladimir Homutov
b6b39b2fb9 Added the ngx_http_test_required_predicates() function.
In contrast to ngx_http_test_predicates(), it requires all values to be
non-empty and not equal to "0".
2019-01-17 14:31:04 +03:00
Vladimir Homutov
de20d73ec5 Version bump. 2019-01-17 14:31:01 +03:00
Vladimir Homutov
910f330ad0 Autoindex: fixed possible integer overflow on 32-bit systems. 2018-12-25 12:59:24 +03:00
Maxim Dounin
aa741f8727 Win32: removed NGX_DIR_MASK concept.
Previous interface of ngx_open_dir() assumed that passed directory name
has a room for NGX_DIR_MASK at the end (NGX_DIR_MASK_LEN bytes).  While all
direct users of ngx_dir_open() followed this interface, this also implied
similar requirements for indirect uses - in particular, via ngx_walk_tree().

Currently none of ngx_walk_tree() uses provides appropriate space, and
fixing this does not look like a right way to go.  Instead, ngx_dir_open()
interface was changed to not require any additional space and use
appropriate allocations instead.
2018-12-24 21:07:05 +03:00
Sergey Kandaurov
499bb2655e Userid: using stub for AF_UNIX addresses.
Previously, AF_UNIX addresses misbehaved as AF_INET, which typically resulted
in $uid_set composed from the middle of sun_path.
2018-12-24 19:55:00 +03:00
Sergey Kandaurov
2a11bf0f77 SSL: avoid reading on pending SSL_write_early_data().
If SSL_write_early_data() returned SSL_ERROR_WANT_WRITE, stop further reading
using a newly introduced c->ssl->write_blocked flag, as otherwise this would
result in SSL error "ssl3_write_bytes:bad length".  Eventually, normal reading
will be restored by read event posted from successful SSL_write_early_data().

While here, place "SSL_write_early_data: want write" debug on the path.
2018-12-18 15:15:15 +03:00
Maxim Dounin
ce4a23d144 Geo: fixed handling of AF_UNIX client addresses (ticket #1684).
Previously, AF_UNIX client addresses were handled as AF_INET, leading
to unexpected results.
2018-12-14 18:11:06 +03:00
Ruslan Ermilov
cb4dd56771 Upstream: implemented $upstream_bytes_sent. 2018-12-13 17:23:07 +03:00
Roman Arutyunyan
8e2949e56a Resolver: report SRV resolve failure if all A resolves failed.
Previously, if an SRV record was successfully resolved, but all of its A
records failed to resolve, NXDOMAIN was returned to the caller, which is
considered a successful resolve rather than an error.  This could result in
losing the result of a previous successful resolve by the caller.

Now NXDOMAIN is only returned if at least one A resolve completed with this
code.  Otherwise the error state of the first A resolve is returned.
2018-12-11 19:41:22 +03:00
Roman Arutyunyan
746fba0d79 Copy regex unnamed captures to cloned subrequests.
Previously, unnamed regex captures matched in the parent request, were not
available in a cloned subrequest.  Now 3 fields related to unnamed captures
are copied to a cloned subrequest: r->ncaptures, r->captures and
r->captures_data.  Since r->captures cannot be changed by either request after
creating a clone, a new flag r->realloc_captures is introduced to force
reallocation of r->captures.

The issue was reported as a proxy_cache_background_update misbehavior in
http://mailman.nginx.org/pipermail/nginx/2018-December/057251.html.
2018-12-11 13:09:00 +03:00
Roman Arutyunyan
7b7f7c1458 Version bump. 2018-12-11 13:12:35 +03:00
Maxim Dounin
f4c70589ce Negative size buffers detection.
In the past, there were several security issues which resulted in
worker process memory disclosure due to buffers with negative size.
It looks reasonable to check for such buffers in various places,
much like we already check for zero size buffers.

While here, removed "#if 1 / #endif" around zero size buffer checks.
It looks highly unlikely that we'll disable these checks anytime soon.
2018-11-26 18:29:56 +03:00
Maxim Dounin
f5708e66c7 Mp4: fixed possible pointer overflow on 32-bit platforms.
On 32-bit platforms mp4->buffer_pos might overflow when a large
enough (close to 4 gigabytes) atom is being skipped, resulting in
incorrect memory addesses being read further in the code.  In most
cases this results in harmless errors being logged, though may also
result in a segmentation fault if hitting unmapped pages.

To address this, ngx_mp4_atom_next() now only increments mp4->buffer_pos
up to mp4->buffer_end.  This ensures that overflow cannot happen.
2018-11-21 20:23:16 +03:00
Maxim Dounin
aedc37fb3e Limit req: "delay=" parameter.
This parameter specifies an additional "soft" burst limit at which requests
become delayed (but not yet rejected as it happens if "burst=" limit is
exceeded).  Defaults to 0, i.e., all excess requests are delayed.

Originally inspired by Vladislav Shabanov
(http://mailman.nginx.org/pipermail/nginx-devel/2016-April/008126.html).
Further improved based on a patch by Peter Shchuchkin
(http://mailman.nginx.org/pipermail/nginx-devel/2018-October/011522.html).
2018-11-21 18:56:50 +03:00
Maxim Dounin
56dffac3e3 Limit req: fixed error message wording. 2018-11-21 18:56:44 +03:00
Vladimir Homutov
c241467318 Upstream: revised upstream response time variables.
Variables now do not depend on presence of the HTTP status code in response.
If the corresponding event occurred, variables contain time between request
creation and the event, and "-" otherwise.

Previously, intermediate value of the $upstream_response_time variable held
unix timestamp.
2018-11-21 13:40:40 +03:00
Vladimir Homutov
0f669b23a8 Upstream: removed unused ngx_http_upstream_t.timeout field. 2018-11-21 13:40:36 +03:00
Maxim Dounin
6c3838f9ed Core: ngx_explicit_memzero(). 2018-11-15 21:28:02 +03:00
Ruslan Ermilov
841c899a9e Core: free shared memory on cycle initialization failure. 2018-11-15 15:28:54 +03:00
Vladimir Homutov
41a451e286 Stream: proxy_requests directive.
The directive allows to drop binding between a client and existing UDP stream
session after receiving a specified number of packets.  First packet from the
same client address and port will start a new session.  Old session continues
to exist and will terminate at moment defined by configuration: either after
receiving the expected number of responses, or after timeout, as specified by
the "proxy_responses" and/or "proxy_timeout" directives.

By default, proxy_requests is zero (disabled).
2018-11-12 16:29:30 +03:00
Vladimir Homutov
abf04ed87a Stream: session completion check code moved to a separate function.
The code refactored to simplify the ngx_stream_proxy_process() function
and facilitate adding new session termination conditions.
2018-11-12 12:05:03 +03:00
Vladimir Homutov
7e3041b79f Stream: fixed possible use of a freed connection.
The session handler may result in session termination, thus a connection
pool (from which c->udp was allocated) may be destroyed.
2018-11-07 13:22:14 +03:00
Vladimir Homutov
6351d8306f Version bump. 2018-11-13 14:42:47 +03:00
Maxim Dounin
42043b4ef7 gRPC: limited allocations due to ping and settings frames. 2018-11-06 16:29:59 +03:00
Ruslan Ermilov
60b93594cc HTTP/2: limit the number of idle state switches.
An attack that continuously switches HTTP/2 connection between
idle and active states can result in excessive CPU usage.
This is because when a connection switches to the idle state,
all of its memory pool caches are freed.

This change limits the maximum allowed number of idle state
switches to 10 * http2_max_requests (i.e., 10000 by default).
This limits possible CPU usage in one connection, and also
imposes a limit on the maximum lifetime of a connection.

Initially reported by Gal Goldshtein from F5 Networks.
2018-11-06 16:29:49 +03:00
Ruslan Ermilov
8ec4146e1a HTTP/2: flood detection.
Fixed uncontrolled memory growth in case peer is flooding us with
some frames (e.g., SETTINGS and PING) and doesn't read data.  Fix
is to limit the number of allocated control frames.
2018-11-06 16:29:35 +03:00
Roman Arutyunyan
9cd9526ba6 Mp4: fixed reading 64-bit atoms.
Previously there was no validation for the size of a 64-bit atom
in an mp4 file.  This could lead to a CPU hog when the size is 0,
or various other problems due to integer underflow when calculating
atom data size, including segmentation fault or worker process
memory disclosure.
2018-11-06 16:29:18 +03:00
Maxim Dounin
b66ee453cc Cache: improved keys zone size error reporting.
After this change, too small keys zones are explicitly reported as such,
much like in the other modules which use shared memory.
2018-10-31 16:49:40 +03:00
Maxim Dounin
f186a01901 Cache: fixed minimum cache keys zone size limit.
Size of a shared memory zones must be at least two pages - one page
for slab allocator internal data, and another page for actual allocations.
Using 8192 instead is wrong, as there are systems with page sizes other
than 4096.

Note well that two pages is usually too low as well.  In particular, cache
is likely to use two allocations of different sizes for global structures,
and at least four pages will be needed to properly allocate cache nodes.
Except in a few very special cases, with keys zone of just two pages nginx
won't be able to start.  Other uses of shared memory impose a limit
of 8 pages, which provides some room for global allocations.  This patch
doesn't try to address this though.

Inspired by ticket #1665.
2018-10-31 16:49:39 +03:00
chronolaw
f3ed2fc356 A minor code clean for macro ngx_event_get_conf in ngx_event.h. 2018-10-19 13:50:36 +08:00
Maxim Dounin
471d077fdd SSL: explicitly set maximum version (ticket #1654).
With maximum version explicitly set, TLSv1.3 will not be unexpectedly
enabled if nginx compiled with OpenSSL 1.1.0 (without TLSv1.3 support)
will be run with OpenSSL 1.1.1 (with TLSv1.3 support).
2018-10-23 22:11:48 +03:00
Vladimir Homutov
1305b8414d Upstream: proxy_socket_keepalive and friends.
The directives enable the use of the SO_KEEPALIVE option on
upstream connections.  By default, the value is left unchanged.
2018-10-03 14:08:51 +03:00
Vladimir Homutov
ae1e6e5ec0 Version bump. 2018-10-03 17:02:44 +03:00
Maxim Dounin
53803b4780 SSL: fixed segfault on renegotiation (ticket #1646).
In e3ba4026c02d (1.15.4) nginx own renegotiation checks were disabled
if SSL_OP_NO_RENEGOTIATION is available.  But since SSL_OP_NO_RENEGOTIATION
is only set on a connection, not in an SSL context, SSL_clear_option()
removed it as long as a matching virtual server was found.  This resulted
in a segmentation fault similar to the one fixed in a6902a941279 (1.9.8),
affecting nginx built with OpenSSL 1.1.0h or higher.

To fix this, SSL_OP_NO_RENEGOTIATION is now explicitly set in
ngx_http_ssl_servername() after adjusting options.  Additionally, instead
of c->ssl->renegotiation we now check c->ssl->handshaked, which seems
to be a more correct flag to test, and will prevent the segmentation fault
from happening even if SSL_OP_NO_RENEGOTIATION is not working.
2018-10-02 17:46:18 +03:00
Ruslan Ermilov
df0dfa634d Fixed off-by-one error in shared zone initialization.
On systems without atomic ops, not enough space was allocated
for mutex's file name during shared zone initialization.
2018-10-02 13:32:52 +03:00
Ruslan Ermilov
a50dec6d6a SSL: fixed unlocked access to sess_id->len. 2018-09-25 14:07:59 +03:00
Ruslan Ermilov
99e06c69c5 Version bump. 2018-09-27 13:05:39 +03:00
Maxim Dounin
b7edec61c3 SSL: logging level of "no suitable signature algorithm".
The "no suitable signature algorithm" errors are reported by OpenSSL 1.1.1
when using TLSv1.3 if there are no shared signature algorithms.  In
particular, this can happen if the client limits available signature
algorithms to something we don't have a certificate for, or to an empty
list.  For example, the following command:

    openssl s_client -connect 127.0.0.1:8443 -sigalgs rsa_pkcs1_sha1

will always result in the "no suitable signature algorithm" error
as the "rsa_pkcs1_sha1" algorithm refers solely to signatures which
appear in certificates and not defined for use in TLS 1.3 handshake
messages.

The SSL_R_NO_COMMON_SIGNATURE_ALGORITHMS error is what BoringSSL returns
in the same situation.
2018-09-25 14:00:04 +03:00
Maxim Dounin
31ef0c47ca SSL: logging level of "no suitable key share".
The "no suitable key share" errors are reported by OpenSSL 1.1.1 when
using TLSv1.3 if there are no shared groups (that is, elliptic curves).
In particular, it is easy enough to trigger by using only a single
curve in ssl_ecdh_curve:

    ssl_ecdh_curve secp384r1;

and using a different curve in the client:

    openssl s_client -connect 127.0.0.1:443 -curves prime256v1

On the client side it is seen as "sslv3 alert handshake failure",
"SSL alert number 40":

0:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:ssl/record/rec_layer_s3.c:1528:SSL alert number 40

It can be also triggered with default ssl_ecdh_curve by using a curve
which is not in the default list (X25519, prime256v1, X448, secp521r1,
secp384r1):

    openssl s_client -connect 127.0.0.1:8443 -curves brainpoolP512r1

Given that many clients hardcode prime256v1, these errors might become
a common problem with TLSv1.3 if ssl_ecdh_curve is redefined.  Previously
this resulted in not using ECDH with such clients, but with TLSv1.3 it
is no longer possible and will result in a handshake failure.

The SSL_R_NO_SHARED_GROUP error is what BoringSSL returns in the same
situation.

Seen at:

https://serverfault.com/questions/932102/nginx-ssl-handshake-error-no-suitable-key-share
2018-09-25 13:59:53 +03:00
Gena Makhomed
1065455289 Cache: status must be less then 599 in *_cache_valid directives.
Previously, configurations with typo, for example

    fastcgi_cache_valid 200301 302 5m;

successfully pass configuration test. Adding check for status
codes > 599, and such configurations are now properly rejected.
2018-09-24 20:26:46 +03:00
Nova DasSarma
8117b3f5a0 Removed bgcolor attribute on body in error pages and autoindex.
The bgcolor attribute overrides compatibility settings in browsers
and leads to undesirable behavior when the default font color is set
to white in the browser, since font-color is not also overridden.
2018-09-19 09:26:47 -05:00
Sergey Kandaurov
ab9038af7e SSL: support for TLSv1.3 early data with OpenSSL.
In collaboration with Maxim Dounin.
2018-09-21 20:49:12 +03:00
Maxim Dounin
61cec6f01b SSL: disabled renegotiation checks with SSL_OP_NO_RENEGOTIATION.
Following 7319:dcab86115261, as long as SSL_OP_NO_RENEGOTIATION is
defined, it is OpenSSL library responsibility to prevent renegotiation,
so the checks are meaningless.

Additionally, with TLSv1.3 OpenSSL tends to report SSL_CB_HANDSHAKE_START
at various unexpected moments - notably, on KeyUpdate messages and
when sending tickets.  This change prevents unexpected connection
close on KeyUpdate messages and when finishing handshake with upcoming
early data changes.
2018-09-21 20:31:32 +03:00
Maxim Dounin
a834b8aa09 Rewrite: removed r->err_status special handling (ticket #1634).
Trying to look into r->err_status in the "return" directive
makes it behave differently than real errors generated in other
parts of the code, and is an endless source of various problems.
This behaviour was introduced in 726:7b71936d5299 (0.4.4) with
the comment "fix: "return" always overrode "error_page" response code".
It is not clear if there were any real cases this was expected to fix,
but there are several cases which are broken due to this change, some
previously fixed (4147:7f64de1cc2c0).

In ticket #1634, the problem is that when r->err_status is set to
a non-special status code, it is not possible to return a response
by simply returning r->err_status.  If this is the case, the only
option is to return script's e->status instead.  An example
configuration:

    location / {
        error_page 404 =200 /err502;
        return 404;
    }

    location = /err502 {
        return 502;
    }

After the change, such a configuration will properly return
standard 502 error, much like it happens when a 502 error is
generated by proxy_pass.

This also fixes the following configuration to properly close
connection as clearly requested by "return 444":

    location / {
        error_page 404 /close;
        return 404;
    }

    location = /close {
        return 444;
    }

Previously, this required "error_page 404 = /close;" to work
as intended.
2018-09-21 15:59:33 +03:00
Maxim Dounin
e4a3211e2f Fixed socket leak with "return 444" in error_page (ticket #274).
Socket leak was observed in the following configuration:

    error_page 400 = /close;

    location = /close {
        return 444;
    }

The problem is that "return 444" triggers termination of the request,
and due to error_page termination thinks that it needs to use a posted
request to clear stack.  But at the early request processing where 400
errors are generated there are no ngx_http_run_posted_requests() calls,
so the request is only terminated after an external event.

Variants of the problem include "error_page 497" instead (ticket #695)
and various other errors generated during early request processing
(405, 414, 421, 494, 495, 496, 501, 505).

The same problem can be also triggered with "return 499" and "return 408"
as both codes trigger ngx_http_terminate_request(), much like "return 444".

To fix this, the patch adds ngx_http_run_posted_requests() calls to
ngx_http_process_request_line() and ngx_http_process_request_headers()
functions, and to ngx_http_v2_run_request() and ngx_http_v2_push_stream()
functions in HTTP/2.

Since the ngx_http_process_request() function is now only called via
other functions which call ngx_http_run_posted_requests(), the call
there is no longer needed and was removed.
2018-09-21 15:59:30 +03:00
Maxim Dounin
05029e775f SSL: restore handlers after blocking.
It is possible that after SSL_read() will return SSL_ERROR_WANT_WRITE,
further calls will return SSL_ERROR_WANT_READ without reading any
application data.  We have to call ngx_handle_write_event() and
switch back to normal write handling much like we do if there are some
application data, or the write there will be reported again and again.

Similarly, we have to switch back to normal read handling if there
is saved read handler and SSL_write() returns SSL_ERROR_WANT_WRITE.
2018-09-10 18:57:39 +03:00
Maxim Dounin
c2f90de0c5 SSL: corrected SSL_ERROR_WANT_WRITE / SSL_ERROR_WANT_READ logging.
While SSL_read() most likely to return SSL_ERROR_WANT_WRITE (and SSL_write()
accordingly SSL_ERROR_WANT_READ) during an SSL renegotiation, it is
not necessary mean that a renegotiation was started.  In particular,
it can never happen during a renegotiation or can happen multiple times
during a renegotiation.

Because of the above, misleading "peer started SSL renegotiation" info
messages were replaced with "SSL_read: want write" and "SSL_write: want read"
debug ones.

Additionally, "SSL write handler" and "SSL read handler" are now logged
by the SSL write and read handlers, to make it easier to understand that
temporary SSL handlers are called instead of normal handlers.
2018-09-10 18:57:19 +03:00
Maxim Dounin
278be041dd Lingering close changed to handle NGX_AGAIN.
The "do { c->recv() } while (c->read->ready)" form used in the
ngx_http_lingering_close_handler() is not really correct, as for
example with SSL c->read->ready may be still set when returning NGX_AGAIN
due to SSL_ERROR_WANT_WRITE.  Therefore the above might be an infinite loop.

This doesn't really matter in lingering close, as we shutdown write side
of the socket anyway and also disable renegotiation (and even without shutdown
and with renegotiation it requires using very large certificate chain and
tuning socket buffers to trigger SSL_ERROR_WANT_WRITE).  But for the sake of
correctness added an NGX_AGAIN check.
2018-09-10 18:57:13 +03:00
Maxim Dounin
07dbfd00ff gRPC: disabled keepalive when sending control frames was blocked.
If sending request body was not completed (u->request_body_sent is not set),
the upstream keepalive module won't save such a connection.  However, it
is theoretically possible (though highly unlikely) that sending of some
control frames can be blocked after the request body was sent.  The
ctx->output_blocked flag introduced to disable keepalive in such cases.
2018-09-03 19:34:02 +03:00
Maxim Dounin
71e152debe gRPC: improved keepalive handling.
The code is now able to parse additional control frames after
the response is received, and can send control frames as well.
This fixes keepalive problems as observed with grpc-c, which can
send window update and ping frames after the response, see
http://mailman.nginx.org/pipermail/nginx/2018-August/056620.html.
2018-09-03 19:34:01 +03:00
Maxim Dounin
b71792b110 Uwsgi: added a check on maximum uwsgi request size.
Requested by Chris Caputo.
2018-09-03 19:17:06 +03:00
Maxim Dounin
07c3e79b64 Uwsgi: style. 2018-09-03 19:17:01 +03:00
Roman Arutyunyan
09cb553c7f Version bump. 2018-08-30 14:42:15 +03:00
Roman Arutyunyan
d9908c6c9a Stream: avoid potential infinite loop at preread phase.
Previously the preread phase code ignored NGX_AGAIN value returned from
c->recv() and relied only on c->read->ready.  But this flag is not reliable and
should only be checked for optimization purposes.  For example, when using
SSL, c->read->ready may be set when no input is available.  This can lead to
calling preread handler infinitely in a loop.
2018-08-29 15:56:42 +03:00
Vladimir Homutov
62821f1c99 Upstream: fixed request chain traversal (ticket #1618).
The problem does not manifest itself currently, because in case of
non-buffered reading, chain link created by u->create_request method
consists of a single element.

Found by PVS-Studio.
2018-08-24 12:19:37 +03:00
Maxim Dounin
d817ceae72 Upstream keepalive: keepalive_requests directive.
The directive configures maximum number of requests allowed on
a connection kept in the cache.  Once a connection reaches the number
of requests configured, it is no longer saved to the cache.
The default is 100.

Much like keepalive_requests for client connections, this is mostly
a safeguard to make sure connections are closed periodically and the
memory allocated from the connection pool is freed.
2018-08-10 21:54:46 +03:00
Maxim Dounin
7de808990b Upstream keepalive: keepalive_timeout directive.
The directive configures maximum time a connection can be kept in the
cache.  By configuring a time which is smaller than the corresponding
timeout on the backend side one can avoid the race between closing
a connection by the backend and nginx trying to use the same connection
to send a request at the same time.
2018-08-10 21:54:23 +03:00
Maxim Dounin
f3d1a925b5 Upstream keepalive: comment added. 2018-08-10 21:54:17 +03:00
Maxim Dounin
1b1b632eed SSL: fixed build with LibreSSL 2.8.0 (ticket #1605).
LibreSSL 2.8.0 "added const annotations to many existing APIs from OpenSSL,
making interoperability easier for downstream applications".  This includes
the const change in the SSL_CTX_sess_set_get_cb() callback function (see
9dd43f4ef67e), which breaks compilation.

To fix this, added a condition on how we redefine OPENSSL_VERSION_NUMBER
when working with LibreSSL (see 382fc7069e3a).  With LibreSSL 2.8.0,
we now set OPENSSL_VERSION_NUMBER to 0x1010000fL (OpenSSL 1.1.0), so the
appropriate conditions in the code will use "const" as it happens with
OpenSSL 1.1.0 and later versions.
2018-08-10 20:49:06 +03:00
Maxim Dounin
0d224602e9 HTTP/2: workaround for clients which fail on table size updates.
There are clients which cannot handle HPACK's dynamic table size updates
as added in 12cadc4669a7 (1.13.6).  Notably, old versions of OkHttp library
are known to fail on it (ticket #1397).

This change makes it possible to work with such clients by only sending
dynamic table size updates in response to SETTINGS_HEADER_TABLE_SIZE.  As
a downside, clients which do not use SETTINGS_HEADER_TABLE_SIZE will
continue to maintain default 4k table.
2018-08-09 20:12:17 +03:00
Maxim Dounin
dd8e1e178c Skipping spaces in configuration files (ticket #1557).
Previously, a chunk of spaces larger than NGX_CONF_BUFFER (4096 bytes)
resulted in the "too long parameter" error during parsing such a
configuration.  This was because the code only set start and start_line
on non-whitespace characters, and hence adjacent whitespace characters
were preserved when reading additional data from the configuration file.
Fix is to always move start and start_line if the last character was
a space.
2018-08-09 12:15:42 +03:00
Maxim Dounin
3b1589173f SSL: support for TLSv1.3 early data with BoringSSL.
Early data AKA 0-RTT mode is enabled as long as "ssl_early_data on" is
specified in the configuration (default is off).

The $ssl_early_data variable evaluates to "1" if the SSL handshake
isn't yet completed, and can be used to set the Early-Data header as
per draft-ietf-httpbis-replay-04.
2018-08-07 02:16:07 +03:00
Maxim Dounin
9f30fda1c2 SSL: enabled TLSv1.3 with BoringSSL.
BoringSSL currently requires SSL_CTX_set_max_proto_version(TLS1_3_VERSION)
to be able to enable TLS 1.3.  This is because by default max protocol
version is set to TLS 1.2, and the SSL_OP_NO_* options are merely used
as a blacklist within the version range specified using the
SSL_CTX_set_min_proto_version() and SSL_CTX_set_max_proto_version()
functions.

With this change, we now call SSL_CTX_set_max_proto_version() with an
explicit maximum version set.  This enables TLS 1.3 with BoringSSL.
As a side effect, this change also limits maximum protocol version to
the newest protocol we know about, TLS 1.3.  This seems to be a good
change, as enabling unknown protocols might have unexpected results.

Additionally, we now explicitly call SSL_CTX_set_min_proto_version()
with 0.  This is expected to help with Debian system-wide default
of MinProtocol set to TLSv1.2, see
http://mailman.nginx.org/pipermail/nginx-ru/2017-October/060411.html.

Note that there is no SSL_CTX_set_min_proto_version macro in BoringSSL,
so we call SSL_CTX_set_min_proto_version() and SSL_CTX_set_max_proto_version()
as long as the TLS1_3_VERSION macro is defined.
2018-08-07 02:15:28 +03:00
Sergey Kandaurov
a6baf5e508 Dav: removed dead store after 8e7a5de61664.
Found by Clang Static Analyzer.
2018-08-02 13:19:48 +03:00
Maxim Dounin
04a6d9d3c9 Dav: changed COPY of a file to preserve access mask.
The behaviour is now in line with COPY of a directory with contents,
which preserves access masks on individual files, as well as the "cp"
command.

Requested by Roman Arutyunyan.
2018-08-01 02:12:21 +03:00
Maxim Dounin
9e0d1236f1 Dav: changed ngx_copy_file() to preserve access and mtime.
This fixes wrong permissions and file time after cross-device MOVE
in the DAV module (ticket #1577).  Broken in 8101d9101ed8 (0.8.9) when
cross-device copying was introduced in ngx_ext_rename_file().

With this change, ngx_copy_file() always calls ngx_set_file_time(),
either with the time provided, or with the time from the original file.
This is considered acceptable given that copying the file is costly anyway,
and optimizing cases when we do not need to preserve time will require
interface changes.
2018-08-01 02:12:11 +03:00
Maxim Dounin
6bff9c9967 Dav: fixed ngx_copy_file() to truncate destination file.
Previously, ngx_open_file(NGX_FILE_CREATE_OR_OPEN) was used, resulting
in destination file being partially rewritten if exists.  Notably,
this affected WebDAV COPY command (ticket #1576).
2018-08-01 02:11:58 +03:00
Sergey Kandaurov
a1e2c5566c Version bump. 2018-07-24 18:46:18 +03:00
Maxim Dounin
5a19c034f3 Fixed NGX_TID_T_FMT format specification for uint64_t.
Previously, "%uA" was used, which corresponds to ngx_atomic_uint_t.
Size of ngx_atomic_uint_t can be easily different from uint64_t,
leading to undefined results.
2018-07-22 04:03:40 +03:00
Sergey Kandaurov
b93931ae82 Stream ssl_preread: added SSLv2 Client Hello support.
In particular, it was not possible to obtain SSLv2 protocol version.
2018-07-18 18:51:25 +03:00
Ruslan Ermilov
372b624627 Fixed invalid access to location defined as an empty string. 2018-07-17 15:30:43 +03:00
Sergey Kandaurov
d5a27006e0 SSL: save sessions for upstream peers using a callback function.
In TLSv1.3, NewSessionTicket messages arrive after the handshake and
can come at any time.  Therefore we use a callback to save the session
when we know about it.  This approach works for < TLSv1.3 as well.
The callback function is set once per location on merge phase.

Since SSL_get_session() in BoringSSL returns an unresumable session for
TLSv1.3, peer save_session() methods have been updated as well to use a
session supplied within the callback.  To preserve API, the session is
cached in c->ssl->session.  It is preferably accessed in save_session()
methods by ngx_ssl_get_session() and ngx_ssl_get0_session() wrappers.
2018-07-17 12:53:23 +03:00
Maxim Dounin
e1bebd05cb SSL: use of the SSL_OP_NO_RENEGOTIATION option (ticket #1376).
The SSL_OP_NO_RENEGOTIATION option is available in OpenSSL 1.1.0h+ and can
save some CPU cycles on renegotiation attempts.
2018-07-16 17:47:48 +03:00
Maxim Dounin
1456129902 SSL: fixed SSL_clear_options() usage with OpenSSL 1.1.0+.
In OpenSSL 1.1.0 the SSL_CTRL_CLEAR_OPTIONS macro was removed, so
conditional compilation test on it results in SSL_clear_options()
and SSL_CTX_clear_options() not being used.  Notably, this caused
"ssl_prefer_server_ciphers off" to not work in SNI-based virtual
servers if server preference was switched on in the default server.

It looks like the only possible fix is to test OPENSSL_VERSION_NUMBER
explicitly.
2018-07-16 17:47:20 +03:00
Maxim Dounin
b1734fd800 SSL: logging levels of "unsupported protocol", "version too low".
Starting with OpenSSL 1.1.0, SSL_R_UNSUPPORTED_PROTOCOL instead of
SSL_R_UNKNOWN_PROTOCOL is reported when a protocol is disabled via
an SSL_OP_NO_* option.

Additionally, SSL_R_VERSION_TOO_LOW is reported when using MinProtocol
or when seclevel checks (as set by @SECLEVEL=n in the cipher string)
rejects a protocol, and this is what happens with SSLv3 and @SECLEVEL=1,
which is the default.

There is also the SSL_R_VERSION_TOO_HIGH error code, but it looks like
it is not possible to trigger it.
2018-07-16 17:47:18 +03:00
Maxim Dounin
85b44b46fb Events: added configuration check on the number of connections.
There should be at least one worker connection for each listening socket,
plus an additional connection for channel between worker and master,
or starting worker processes will fail.
2018-07-12 19:50:07 +03:00
Maxim Dounin
751bdd3bb2 Events: moved sockets cloning to ngx_event_init_conf().
Previously, listenings sockets were not cloned if the worker_processes
directive was specified after "listen ... reuseport".

This also simplifies upcoming configuration check on the number
of worker connections, as it needs to know the number of listening
sockets before cloning.
2018-07-12 19:50:02 +03:00
Roman Arutyunyan
a8e38e2a9c Stream ssl_preread: $ssl_preread_protocol variable.
The variable keeps the latest SSL protocol version supported by the client.
The variable has the same format as $ssl_protocol.

The version is read from the client_version field of ClientHello.  If the
supported_versions extension is present in the ClientHello, then the version
is set to TLSv1.3.
2018-07-11 17:56:51 +03:00
Ruslan Ermilov
527cac2979 Allow resetting connections closed by "return 444" (ticket #905).
If reset_timedout_connection is on, TCP connections closed by
"return 444" will be reset instead of a normal close.
2018-07-12 12:50:20 +03:00
Maxim Dounin
f62d460d5b Resolver: retry sending queries on errors (ticket #1511).
Errors when sending UDP datagrams can happen, e.g., when local IP address
changes (see fa0e093b64d7), or an unavailable DNS server on the LAN can cause
send() to fail with EHOSTDOWN on BSD systems.  If this happens during
initial query, retry sending immediately, to a different DNS server when
possible.  If this is not enough, allow normal resend to happen by ignoring
the return code of the second ngx_resolver_send_query() call, much like we
do in ngx_resolver_resend().
2018-07-05 22:21:14 +03:00
Maxim Dounin
f206a112c6 SSL: logging level of "https proxy request" errors.
The "http request" and "https proxy request" errors cannot happen
with HTTP due to pre-handshake checks in ngx_http_ssl_handshake(),
but can happen when SSL is used in stream and mail modules.
2018-07-05 20:45:29 +03:00
Maxim Dounin
c62f9c914f Version bump. 2018-07-05 20:44:58 +03:00
Maxim Dounin
a7186c8f1c Upstream: fixed tcp_nopush with gRPC.
With gRPC it is possible that a request sending is blocked due to flow
control.  Moreover, further sending might be only allowed once the
backend sees all the data we've already sent.  With such a backend
it is required to clear the TCP_NOPUSH socket option to make sure all
the data we've sent are actually delivered to the backend.

As such, we now clear TCP_NOPUSH in ngx_http_upstream_send_request()
also on NGX_AGAIN if c->write->ready is set.  This fixes a test (which
waits for all the 64k bytes as per initial window before allowing more
bytes) with sendfile enabled when the body was written to a file
in a different context.
2018-07-02 19:03:04 +03:00
Maxim Dounin
1331a99f5c Upstream: fixed unexpected tcp_nopush usage on peer connections.
Now tcp_nopush on peer connections is disabled if it is disabled on
the client connection, similar to how we handle c->sendfile.  Previously,
tcp_nopush was always used on upstream connections, regardless of
the "tcp_nopush" directive.
2018-07-02 19:02:31 +03:00
Maxim Dounin
d842b4e5e0 gRPC: clearing buffers in ngx_http_grpc_get_buf().
We copy input buffers to our buffers, so various flags might be
unexpectedly set in buffers returned by ngx_chain_get_free_buf().

In particular, the b->in_file flag might be set when the body was
written to a file in a different context.  With sendfile enabled this
in turn might result in protocol corruption if such a buffer was reused
for a control frame.

Make sure to clear buffers and set only fields we really need to be set.
2018-07-02 19:02:08 +03:00
Ruslan Ermilov
468e37734c Added FreeBSD support for "listen ... reuseport". 2018-07-02 13:54:33 +03:00
Vladimir Homutov
0c4ccbea23 Upstream: ngx_http_upstream_random module.
The module implements random load-balancing algorithm with optional second
choice.  In the latter case, the best of two servers is chosen, accounting
number of connections and server weight.

Example:

upstream u {
    random [two [least_conn]];

    server 127.0.0.1:8080;
    server 127.0.0.1:8081;
    server 127.0.0.1:8082;
    server 127.0.0.1:8083;
}
2018-06-15 11:46:14 +03:00
Ruslan Ermilov
2eab9efbe4 Upstream: improved peer selection concurrency for hash and ip_hash. 2018-06-14 07:03:50 +03:00
Ruslan Ermilov
63e8a1d926 Resolver: require name servers. 2018-06-13 22:37:49 +03:00
Ruslan Ermilov
4542898917 Resolver: allocate resolver from configuration pool.
Before 4a8c9139e579, ngx_resolver_create() didn't use configuration
pool, and allocations were done using malloc().

In 016352c19049, when resolver gained support of several servers,
new allocations were done from the pool.
2018-06-13 22:37:42 +03:00
Maxim Dounin
9b6bc8a578 Upstream: disable body cleanup with preserve_output (ticket #1565).
With u->conf->preserve_output set the request body file might be used
after the response header is sent, so avoid cleaning it.  (Normally
this is not a problem as u->conf->preserve_output is only set with
r->request_body_no_buffering, but the request body might be already
written to a file in a different context.)
2018-06-13 15:28:11 +03:00
Ruslan Ermilov
fb3a9e28b2 HTTP/2: use scheme from original request for pushes (closes #1549).
Instead of the connection scheme, use scheme from the original request.
This fixes pushes when SSL is terminated by a proxy server in front of
nginx.
2018-06-07 20:04:22 +03:00
Ruslan Ermilov
f11a9cbdd0 Added r->schema.
For HTTP/1, it keeps scheme from the absolute form of URI.
For HTTP/2, the :scheme request pseudo-header field value.
2018-06-07 20:01:41 +03:00
Ruslan Ermilov
70b6e7a299 Removed extraneous check while processing request line. 2018-06-07 19:53:43 +03:00
Ruslan Ermilov
94a2ce426f HTTP/2: validate client request scheme.
The scheme is validated as per RFC 3986, Section 3.1.
2018-06-07 11:47:10 +03:00
Ruslan Ermilov
c804eb7748 Allowed digits, '+', '-', and '.' in scheme names as per RFC 3986. 2018-05-24 12:06:35 +03:00
Sergey Kandaurov
1ef7b1ef61 SSL: removed extra prototype. 2018-06-06 13:31:05 +03:00
Maxim Dounin
8dc0f75d0b Added missing space after ngx_close_socket_n. 2018-06-05 17:41:34 +03:00
Maxim Dounin
82b0fe21d6 Version bump. 2018-06-05 17:13:17 +03:00
Roman Arutyunyan
96b6f215b8 Stream: udp streams.
Previously, only one client packet could be processed in a udp stream session
even though multiple response packets were supported.  Now multiple packets
coming from the same client address and port are delivered to the same stream
session.

If it's required to maintain a single stream of data, nginx should be
configured in a way that all packets from a client are delivered to the same
worker.  On Linux and DragonFly BSD the "reuseport" parameter should be
specified for this.  Other systems do not currently provide appropriate
mechanisms.  For these systems a single stream of udp packets is only
guaranteed in single-worker configurations.

The proxy_response directive now specifies how many packets are expected in
response to a single client packet.
2018-06-04 19:50:00 +03:00
Roman Arutyunyan
1028d71695 Events: moved ngx_recvmsg() to new file src/event/ngx_event_udp.c. 2018-06-01 16:55:49 +03:00
Roman Arutyunyan
20f8bfab34 Events: get remote addresses before creating udp connection.
Previously, ngx_event_recvmsg() got remote socket addresses after creating
the connection object.  In preparation to handling multiple UDP packets in a
single session, this code was moved up.
2018-06-01 13:12:57 +03:00
Roman Arutyunyan
26a57486f0 Events: fixed handling zero-length client address.
On Linux recvmsg() syscall may return a zero-length client address when
receiving a datagram from an unbound unix datagram socket.  It is usually
assumed that socket address has at least the sa_family member.  Zero-length
socket address caused buffer over-read in functions which receive socket
address, for example ngx_sock_ntop().  Typically the over-read resulted in
unexpected socket family followed by session close.  Now a fake socket address
is allocated instead of a zero-length client address.
2018-06-01 16:53:02 +03:00
Sergey Kandaurov
dac90a4bff Leave chain in ngx_chain_add_copy() in consistent state on errors. 2018-06-04 18:47:54 +03:00
Maxim Dounin
7b9bc5d4d1 Limit req: improved handling of negative times.
Negative times can appear since workers only update time on an event
loop iteration start.  If a worker was blocked for a long time during
an event loop iteration, it is possible that another worker already
updated the time stored in the node.  As such, time since last update
of the node (ms) will be negative.

Previous code used ngx_abs(ms) in the calculations.  That is, negative
times were effectively treated as positive ones.  As a result, it was
not possible to maintain high request rates, where the same node can be
updated multiple times from during an event loop iteration.
In particular, this affected setups with many SSL handshakes, see
http://mailman.nginx.org/pipermail/nginx/2018-May/056291.html.

Fix is to only update the last update time stored in the node if the
new time is larger than previously stored one.  If a future time is
stored in the node, we preserve this time as is.

To prevent breaking things on platforms without monotonic time available
if system time is updated backwards, a safety limit of 60 seconds is
used.  If the time stored in the node is more than 60 seconds in the future,
we assume that the time was changed backwards and update lr->last
to the current time.
2018-05-30 15:40:34 +03:00
Ruslan Ermilov
e4d173cc88 Core: fixed comment about ngx_current_msec after 81fae70d6cb8.
The value is no longer guaranteed to be based on milliseconds
elapsed since Epoch.
2018-05-29 16:15:19 +03:00
Ruslan Ermilov
4271e48911 Win32: fixed comment in ngx_gettimeofday() calculations. 2018-05-29 11:47:32 +03:00
Maxim Dounin
1ef5c9ce6f Removed glibc crypt_r() bug workaround (ticket #1469).
The bug in question was fixed in glibc 2.3.2 and is no longer expected
to manifest itself on real servers.  On the other hand, the workaround
causes compilation problems on various systems.  Previously, we've
already fixed the code to compile with musl libc (fd6fd02f6a4d), and
now it is broken on Fedora 28 where glibc's crypt library was replaced
by libxcrypt.  So the workaround was removed.
2018-05-23 16:38:16 +03:00
Vladimir Homutov
5568a6598d Syslog: install cleanup handler only once.
If a socket was re-opened due to an error (02c2352d5b01 and fa0e093b64d7),
additional cleanup handler was installed each time.
2018-05-14 22:50:57 +03:00
Ruslan Ermilov
4ac8036e78 Resolver: close UDP socket on error or incomplete send. 2018-05-23 10:41:38 +03:00
Ruslan Ermilov
4fca5bf1e1 Resolver: style. 2018-05-23 10:41:29 +03:00
Maxim Dounin
4f9d83d6d7 Core: silenced getsockopt(TCP_FASTOPEN) messages on FreeBSD.
FreeBSD returns EINVAL when getsockopt(TCP_FASTOPEN) is called on a unix
domain socket, resulting in "getsockopt(TCP_FASTOPEN) ... failed" messages
during binary upgrade when unix domain listen sockets are present in
the configuration.  Added EINVAL to the list of ignored error codes.
2018-05-21 23:11:27 +03:00
Vladimir Homutov
644d26e841 Syslog: re-open syslog udp socket on send error (ticket #1477).
Previously, only unix domain sockets were reopened to tolerate cases when
local syslog server was restarted.  It makes sense to treat other cases
(for example, local IP address changes) similarly.
2018-05-08 19:35:56 +03:00
Sergey Kandaurov
68b50f71e1 Silenced -Wcast-function-type warnings (closes #1546).
Cast to intermediate "void *" to lose compiler knowledge about the original
type and pass the warning.  This is not a real fix but rather a workaround.

Found by gcc8.
2018-05-07 09:54:37 +00:00
Ruslan Ermilov
658a84f425 SSL: deprecated the "ssl" directive. 2018-04-25 14:57:24 +03:00
Maxim Dounin
76be1ea9de SSL: detect "listen ... ssl" without certificates (ticket #178).
In mail and stream modules, no certificate provided is a fatal condition,
much like with the "ssl" and "starttls" directives.

In http, "listen ... ssl" can be used in a non-default server without
certificates as long as there is a certificate in the default one, so
missing certificate is only fatal for default servers.
2018-04-24 15:29:01 +03:00
Maxim Dounin
5d3a854ebd Mail: fixed error message about missing ssl_certificate_key.
In 51e1f047d15d, the "ssl" directive name was incorrectly hardcoded
in the error message shown when there are some SSL keys defined, but
not for all certificates.  Right approach is to use the "mode" variable,
which can be either "ssl" or "starttls".
2018-04-24 15:28:58 +03:00
Maxim Dounin
8b4c7725d2 Cache: fixed cache valid slot to reject incorrect statuses.
Previously, result of ngx_atoi() was assigned to an ngx_uint_t variable,
and errors reported by ngx_atoi() became positive, so the following check
in "status < 100" failed to catch them.  This resulted in the configurations
like "proxy_cache_valid 2xx 30s" being accepted as correct, while they
in fact do nothing.  Changing type to ngx_int_t fixes this, and such
configurations are now properly rejected.
2018-04-18 16:11:41 +03:00
Maxim Dounin
59d806b56f Version bump. 2018-04-18 16:09:08 +03:00
Maxim Dounin
f9e43a3154 Upstream: fixed u->conf->preserve_output (ticket #1519).
Previously, ngx_http_upstream_process_header() might be called after
we've finished reading response headers and switched to a different read
event handler, leading to errors with gRPC proxying.  Additionally,
the u->conf->read_timeout timer might be re-armed during reading response
headers (while this is expected to be a single timeout on reading
the whole response header).
2018-04-05 16:56:12 +03:00
Maxim Dounin
923c9d5f3b Version bump. 2018-04-05 16:53:27 +03:00
Maxim Dounin
d90a75b7cd Upstream: fixed ngx_http_upstream_test_next() conditions.
Previously, ngx_http_upstream_test_next() used an outdated condition on
whether it will be possible to switch to a different server or not.  It
did not take into account restrictions on non-idempotent requests, requests
with non-buffered request body, and the next upstream timeout.

For such requests, switching to the next upstream server was rejected
later in ngx_http_upstream_next(), resulting in nginx own error page
being returned instead of the original upstream response.
2018-04-03 02:43:18 +03:00
Vladimir Homutov
cadc8ca306 Core: fixed build, broken by 63e91f263a49.
Both Solaris and Windows define "s_addr" as a macro.
2018-04-02 20:38:43 +03:00
Ruslan Ermilov
55f08fc9f7 Core: revised the PROXY protocol v2 code.
- use normal prefixes for types and macros
- removed some macros and types
- revised debug messages
- removed useless check of ngx_sock_ntop() returning 0
- removed special processing of AF_UNSPEC
2018-04-02 18:40:04 +03:00
Vladimir Homutov
ecd6e243b6 Core: style. 2018-03-27 18:39:38 +03:00
Vladimir Homutov
9207cc84b2 Core: added processing of version 2 of the PROXY protocol.
The protocol used on inbound connection is auto-detected and corresponding
parser is used to extract passed addresses.  TLV parameters are ignored.

The maximum supported size of PROXY protocol header is 107 bytes
(similar to version 1).
2018-03-22 15:55:28 +03:00
Roman Arutyunyan
f39d5e8b33 Stream: set action before each recv/send while proxying.
Now it's clear from log error message if the error occurred on client or
upstream side.
2018-03-22 18:43:49 +03:00
Maxim Dounin
0ac57648eb gRPC: fixed possible sign extension of error and setting_value.
All cases are harmless and should not happen on valid values, though can
result in bad values being shown incorrectly in logs.

Found by Coverity (CID 1430311, 1430312, 1430313).
2018-03-22 19:26:25 +03:00
Ruslan Ermilov
8e8734ec82 Improved code readability (closes #1512). 2018-03-22 18:13:33 +03:00
Maxim Dounin
25a26bbce4 Version bump. 2018-03-22 15:55:52 +03:00
Sergey Kandaurov
e232421266 gRPC: fixed missing state save in frame header parsing.
Previously, frame state wasn't saved if HEADERS frame payload
that begins with header fragment was not received at once.
2018-03-20 15:58:11 +03:00
Ruslan Ermilov
74ea120f7d HTTP/2: improved frame info debugging. 2018-03-19 21:32:15 +03:00
Sergey Kandaurov
f3ad346952 gRPC: fixed parsing response headers split on CONTINUATION frames. 2018-03-19 16:42:56 +03:00
Ruslan Ermilov
c09aa142d6 Fixed checking ngx_tcp_push() and ngx_tcp_nopush() return values.
No functional changes.
2018-03-19 16:28:23 +03:00
Ruslan Ermilov
510986b806 Upstream: fixed comments after 13f8dec720b5.
The fields "uri", "location", and "url" from ngx_http_upstream_conf_t
moved to ngx_http_proxy_loc_conf_t and ngx_http_proxy_vars_t, reflect
this change in create_loc_conf comments.
2018-03-19 16:22:09 +03:00
Maxim Dounin
6559a42013 gRPC: special handling of "trailer only" responses.
The gRPC protocol makes a distinction between HEADERS frame with
the END_STREAM flag set, and a HEADERS frame followed by an empty
DATA frame with the END_STREAM flag.  The latter is not permitted,
and results in errors not being propagated through nginx.  Instead,
gRPC clients complain that "server closed the stream without sending
trailers" (seen in grpc-go) or "13: Received RST_STREAM with error
code 2" (seen in grpc-c).

To fix this, nginx now returns HEADERS with the END_STREAM flag if
the response length is known to be 0, and we are not expecting
any trailer headers to be added.  And the response length is
explicitly set to 0 in the gRPC proxy if we see initial HEADERS frame
with the END_STREAM flag set.
2018-03-17 23:04:26 +03:00
Maxim Dounin
6a0d9e5b2d gRPC: special handling of the TE request header.
According to the gRPC protocol specification, the "TE" header is used
to detect incompatible proxies, and at least grpc-c server rejects
requests without "TE: trailers".

To preserve the logic, we have to pass "TE: trailers" to the backend if
and only if the original request contains "trailers" in the "TE" header.
Note that no other TE values are allowed in HTTP/2, so we have to remove
anything else.
2018-03-17 23:04:25 +03:00
Maxim Dounin
56ad960e7a The gRPC proxy module.
The module allows passing requests to upstream gRPC servers.
The module is built by default as long as HTTP/2 support is compiled in.
Example configuration:

    grpc_pass 127.0.0.1:9000;

Alternatively, the "grpc://" scheme can be used:

    grpc_pass grpc://127.0.0.1:9000;

Keepalive support is available via the upstream keepalive module.  Note
that keepalive connections won't currently work with grpc-go as it fails
to handle SETTINGS_HEADER_TABLE_SIZE.

To use with SSL:

    grpc_pass grpcs://127.0.0.1:9000;

SSL connections use ALPN "h2" when available.  At least grpc-go works fine
without ALPN, so if ALPN is not available we just establish a connection
without it.

Tested with grpc-c++ and grpc-go.
2018-03-17 23:04:24 +03:00
Maxim Dounin
c67124190c Upstream: u->conf->preserve_output flag.
The flag can be used to continue sending request body even after we've
got a response from the backend.  In particular, this is needed for gRPC
proxying of bidirectional streaming RPCs, and also to send control frames
in other forms of RPCs.
2018-03-17 23:04:23 +03:00
Maxim Dounin
7be60194c1 Upstream: u->request_body_blocked flag.
The flag indicates whether last ngx_output_chain() returned NGX_AGAIN
or not.  If the flag is set, we arm the u->conf->send_timeout timer.
The flag complements c->write->ready test, and allows to stop sending
the request body in an output filter due to protocol-specific flow
control.
2018-03-17 23:04:22 +03:00
Maxim Dounin
7ef115a7e8 Upstream: trailers support, u->conf->pass_trailers flag.
Basic trailer headers support allows one to access response trailers
via the $upstream_trailer_* variables.

Additionally, the u->conf->pass_trailers flag was introduced.  When the
flag is set, trailer headers from the upstream response are passed to
the client.  Like normal headers, trailer headers will be hidden
if present in u->conf->hide_headers_hash.
2018-03-17 23:04:21 +03:00