Commit Graph

6748 Commits

Author SHA1 Message Date
Maxim Dounin
f2cfcc09a0 Win32: enabled both select and poll on Windows by default.
Since we now have both select and poll on Windows, it is enough to
do not set EVENT_FOUND, auto/modules will enable both automatically.
2019-01-24 22:00:13 +03:00
Maxim Dounin
40b74475d0 Win32: added WSAPoll() support.
WSAPoll() is only available with Windows Vista and newer (and only
available during compilation if _WIN32_WINNT >= 0x0600).  To make
sure the code works with Windows XP, we do not redefine _WIN32_WINNT,
but instead load WSAPoll() dynamically if it is not available during
compilation.

Also, sockets are not guaranteed to be small integers on Windows.
So an index array is used instead of NGX_USE_FD_EVENT to map
events to connections.
2019-01-24 21:51:21 +03:00
Maxim Dounin
c6a0003af1 Win32: properly enabled select on Windows.
Previously, select was compiled in by default, but the NGX_HAVE_SELECT
macro was not set, resulting in iocp being used by default unless
the "--with-select_module" configure option was explicitly specified.
Since the iocp module is not finished and does not work properly, this
effectively meant that the "--with-select_module" option was mandatory.

With the change NGX_HAVE_SELECT is properly set, making "--with-select_module"
optional.  Accordingly, it is removed from misc/GNUmakefile win32 target.
2019-01-24 21:51:00 +03:00
Maxim Dounin
3dcad9c5d6 Events: fixed copying of old events in poll init.
Previously, the code incorrectly assumed "ngx_event_t *" elements
instead of "struct pollfd".

This is mostly cosmetic change, as this code is never called now.
2019-01-24 21:50:37 +03:00
Roman Arutyunyan
12645b46e9 Stream: do not split datagrams when limiting proxy rate.
Previously, when using proxy_upload_rate and proxy_download_rate, the buffer
size for reading from a socket could be reduced as a result of rate limiting.
For connection-oriented protocols this behavior is normal since unread data will
normally be read at the next iteration.  But for datagram-oriented protocols
this is not the case, and unread part of the datagram is lost.

Now buffer size is not limited for datagrams.  Rate limiting still works in this
case by delaying the next reading event.
2018-12-27 19:37:34 +03:00
Roman Arutyunyan
36a0713244 Prevented scheduling events on a shared connection.
A shared connection does not own its file descriptor, which means that
ngx_handle_read_event/ngx_handle_write_event calls should do nothing for it.
Currently the c->shared flag is checked in several places in the stream proxy
module prior to calling these functions.  However it was not done everywhere.
Missing checks could lead to calling
ngx_handle_read_event/ngx_handle_write_event on shared connections.

The problem manifested itself when using proxy_upload_rate and resulted in
either duplicate file descriptor error (e.g. with epoll) or incorrect further
udp packet processing (e.g. with kqueue).

The fix is to set and reset the event active flag in a way that prevents
ngx_handle_read_event/ngx_handle_write_event from scheduling socket events.
2019-01-14 20:36:23 +03:00
Vladimir Homutov
b6b39b2fb9 Added the ngx_http_test_required_predicates() function.
In contrast to ngx_http_test_predicates(), it requires all values to be
non-empty and not equal to "0".
2019-01-17 14:31:04 +03:00
Vladimir Homutov
de20d73ec5 Version bump. 2019-01-17 14:31:01 +03:00
Maxim Dounin
d95e34c558 release-1.15.8 tag 2018-12-25 17:53:03 +03:00
Maxim Dounin
2eedb3ff0c nginx-1.15.8-RELEASE 2018-12-25 17:53:03 +03:00
Vladimir Homutov
910f330ad0 Autoindex: fixed possible integer overflow on 32-bit systems. 2018-12-25 12:59:24 +03:00
Gena Makhomed
8610ddd092 Contrib: vim syntax, update core and 3rd party module directives. 2018-12-24 16:30:10 +02:00
Maxim Dounin
aa741f8727 Win32: removed NGX_DIR_MASK concept.
Previous interface of ngx_open_dir() assumed that passed directory name
has a room for NGX_DIR_MASK at the end (NGX_DIR_MASK_LEN bytes).  While all
direct users of ngx_dir_open() followed this interface, this also implied
similar requirements for indirect uses - in particular, via ngx_walk_tree().

Currently none of ngx_walk_tree() uses provides appropriate space, and
fixing this does not look like a right way to go.  Instead, ngx_dir_open()
interface was changed to not require any additional space and use
appropriate allocations instead.
2018-12-24 21:07:05 +03:00
Sergey Kandaurov
499bb2655e Userid: using stub for AF_UNIX addresses.
Previously, AF_UNIX addresses misbehaved as AF_INET, which typically resulted
in $uid_set composed from the middle of sun_path.
2018-12-24 19:55:00 +03:00
Sergey Kandaurov
2a11bf0f77 SSL: avoid reading on pending SSL_write_early_data().
If SSL_write_early_data() returned SSL_ERROR_WANT_WRITE, stop further reading
using a newly introduced c->ssl->write_blocked flag, as otherwise this would
result in SSL error "ssl3_write_bytes:bad length".  Eventually, normal reading
will be restored by read event posted from successful SSL_write_early_data().

While here, place "SSL_write_early_data: want write" debug on the path.
2018-12-18 15:15:15 +03:00
Maxim Dounin
ce4a23d144 Geo: fixed handling of AF_UNIX client addresses (ticket #1684).
Previously, AF_UNIX client addresses were handled as AF_INET, leading
to unexpected results.
2018-12-14 18:11:06 +03:00
Ruslan Ermilov
cb4dd56771 Upstream: implemented $upstream_bytes_sent. 2018-12-13 17:23:07 +03:00
Roman Arutyunyan
8e2949e56a Resolver: report SRV resolve failure if all A resolves failed.
Previously, if an SRV record was successfully resolved, but all of its A
records failed to resolve, NXDOMAIN was returned to the caller, which is
considered a successful resolve rather than an error.  This could result in
losing the result of a previous successful resolve by the caller.

Now NXDOMAIN is only returned if at least one A resolve completed with this
code.  Otherwise the error state of the first A resolve is returned.
2018-12-11 19:41:22 +03:00
Roman Arutyunyan
746fba0d79 Copy regex unnamed captures to cloned subrequests.
Previously, unnamed regex captures matched in the parent request, were not
available in a cloned subrequest.  Now 3 fields related to unnamed captures
are copied to a cloned subrequest: r->ncaptures, r->captures and
r->captures_data.  Since r->captures cannot be changed by either request after
creating a clone, a new flag r->realloc_captures is introduced to force
reallocation of r->captures.

The issue was reported as a proxy_cache_background_update misbehavior in
http://mailman.nginx.org/pipermail/nginx/2018-December/057251.html.
2018-12-11 13:09:00 +03:00
Roman Arutyunyan
7b7f7c1458 Version bump. 2018-12-11 13:12:35 +03:00
Maxim Dounin
e2673071e4 release-1.15.7 tag 2018-11-27 17:40:21 +03:00
Maxim Dounin
0febe9a095 nginx-1.15.7-RELEASE 2018-11-27 17:40:20 +03:00
Maxim Dounin
abbde03cf6 Updated OpenSSL used for win32 builds. 2018-11-27 17:02:56 +03:00
Maxim Dounin
f4c70589ce Negative size buffers detection.
In the past, there were several security issues which resulted in
worker process memory disclosure due to buffers with negative size.
It looks reasonable to check for such buffers in various places,
much like we already check for zero size buffers.

While here, removed "#if 1 / #endif" around zero size buffer checks.
It looks highly unlikely that we'll disable these checks anytime soon.
2018-11-26 18:29:56 +03:00
Maxim Dounin
f5708e66c7 Mp4: fixed possible pointer overflow on 32-bit platforms.
On 32-bit platforms mp4->buffer_pos might overflow when a large
enough (close to 4 gigabytes) atom is being skipped, resulting in
incorrect memory addesses being read further in the code.  In most
cases this results in harmless errors being logged, though may also
result in a segmentation fault if hitting unmapped pages.

To address this, ngx_mp4_atom_next() now only increments mp4->buffer_pos
up to mp4->buffer_end.  This ensures that overflow cannot happen.
2018-11-21 20:23:16 +03:00
Maxim Dounin
aedc37fb3e Limit req: "delay=" parameter.
This parameter specifies an additional "soft" burst limit at which requests
become delayed (but not yet rejected as it happens if "burst=" limit is
exceeded).  Defaults to 0, i.e., all excess requests are delayed.

Originally inspired by Vladislav Shabanov
(http://mailman.nginx.org/pipermail/nginx-devel/2016-April/008126.html).
Further improved based on a patch by Peter Shchuchkin
(http://mailman.nginx.org/pipermail/nginx-devel/2018-October/011522.html).
2018-11-21 18:56:50 +03:00
Maxim Dounin
56dffac3e3 Limit req: fixed error message wording. 2018-11-21 18:56:44 +03:00
Vladimir Homutov
c241467318 Upstream: revised upstream response time variables.
Variables now do not depend on presence of the HTTP status code in response.
If the corresponding event occurred, variables contain time between request
creation and the event, and "-" otherwise.

Previously, intermediate value of the $upstream_response_time variable held
unix timestamp.
2018-11-21 13:40:40 +03:00
Vladimir Homutov
0f669b23a8 Upstream: removed unused ngx_http_upstream_t.timeout field. 2018-11-21 13:40:36 +03:00
Maxim Dounin
6c3838f9ed Core: ngx_explicit_memzero(). 2018-11-15 21:28:02 +03:00
Ruslan Ermilov
841c899a9e Core: free shared memory on cycle initialization failure. 2018-11-15 15:28:54 +03:00
Vladimir Homutov
41a451e286 Stream: proxy_requests directive.
The directive allows to drop binding between a client and existing UDP stream
session after receiving a specified number of packets.  First packet from the
same client address and port will start a new session.  Old session continues
to exist and will terminate at moment defined by configuration: either after
receiving the expected number of responses, or after timeout, as specified by
the "proxy_responses" and/or "proxy_timeout" directives.

By default, proxy_requests is zero (disabled).
2018-11-12 16:29:30 +03:00
Vladimir Homutov
abf04ed87a Stream: session completion check code moved to a separate function.
The code refactored to simplify the ngx_stream_proxy_process() function
and facilitate adding new session termination conditions.
2018-11-12 12:05:03 +03:00
Vladimir Homutov
7e3041b79f Stream: fixed possible use of a freed connection.
The session handler may result in session termination, thus a connection
pool (from which c->udp was allocated) may be destroyed.
2018-11-07 13:22:14 +03:00
Vladimir Homutov
6351d8306f Version bump. 2018-11-13 14:42:47 +03:00
Maxim Dounin
774d69cf4a release-1.15.6 tag 2018-11-06 16:32:09 +03:00
Maxim Dounin
ac071c1490 nginx-1.15.6-RELEASE 2018-11-06 16:32:08 +03:00
Maxim Dounin
42043b4ef7 gRPC: limited allocations due to ping and settings frames. 2018-11-06 16:29:59 +03:00
Ruslan Ermilov
60b93594cc HTTP/2: limit the number of idle state switches.
An attack that continuously switches HTTP/2 connection between
idle and active states can result in excessive CPU usage.
This is because when a connection switches to the idle state,
all of its memory pool caches are freed.

This change limits the maximum allowed number of idle state
switches to 10 * http2_max_requests (i.e., 10000 by default).
This limits possible CPU usage in one connection, and also
imposes a limit on the maximum lifetime of a connection.

Initially reported by Gal Goldshtein from F5 Networks.
2018-11-06 16:29:49 +03:00
Ruslan Ermilov
8ec4146e1a HTTP/2: flood detection.
Fixed uncontrolled memory growth in case peer is flooding us with
some frames (e.g., SETTINGS and PING) and doesn't read data.  Fix
is to limit the number of allocated control frames.
2018-11-06 16:29:35 +03:00
Roman Arutyunyan
9cd9526ba6 Mp4: fixed reading 64-bit atoms.
Previously there was no validation for the size of a 64-bit atom
in an mp4 file.  This could lead to a CPU hog when the size is 0,
or various other problems due to integer underflow when calculating
atom data size, including segmentation fault or worker process
memory disclosure.
2018-11-06 16:29:18 +03:00
Maxim Dounin
b66ee453cc Cache: improved keys zone size error reporting.
After this change, too small keys zones are explicitly reported as such,
much like in the other modules which use shared memory.
2018-10-31 16:49:40 +03:00
Maxim Dounin
f186a01901 Cache: fixed minimum cache keys zone size limit.
Size of a shared memory zones must be at least two pages - one page
for slab allocator internal data, and another page for actual allocations.
Using 8192 instead is wrong, as there are systems with page sizes other
than 4096.

Note well that two pages is usually too low as well.  In particular, cache
is likely to use two allocations of different sizes for global structures,
and at least four pages will be needed to properly allocate cache nodes.
Except in a few very special cases, with keys zone of just two pages nginx
won't be able to start.  Other uses of shared memory impose a limit
of 8 pages, which provides some room for global allocations.  This patch
doesn't try to address this though.

Inspired by ticket #1665.
2018-10-31 16:49:39 +03:00
chronolaw
f3ed2fc356 A minor code clean for macro ngx_event_get_conf in ngx_event.h. 2018-10-19 13:50:36 +08:00
Maxim Dounin
471d077fdd SSL: explicitly set maximum version (ticket #1654).
With maximum version explicitly set, TLSv1.3 will not be unexpectedly
enabled if nginx compiled with OpenSSL 1.1.0 (without TLSv1.3 support)
will be run with OpenSSL 1.1.1 (with TLSv1.3 support).
2018-10-23 22:11:48 +03:00
Vladimir Homutov
1305b8414d Upstream: proxy_socket_keepalive and friends.
The directives enable the use of the SO_KEEPALIVE option on
upstream connections.  By default, the value is left unchanged.
2018-10-03 14:08:51 +03:00
Vladimir Homutov
ae1e6e5ec0 Version bump. 2018-10-03 17:02:44 +03:00
Maxim Dounin
31af68e6ca release-1.15.5 tag 2018-10-02 18:13:52 +03:00
Maxim Dounin
c040ab4bbf nginx-1.15.5-RELEASE 2018-10-02 18:13:51 +03:00
Maxim Dounin
53803b4780 SSL: fixed segfault on renegotiation (ticket #1646).
In e3ba4026c02d (1.15.4) nginx own renegotiation checks were disabled
if SSL_OP_NO_RENEGOTIATION is available.  But since SSL_OP_NO_RENEGOTIATION
is only set on a connection, not in an SSL context, SSL_clear_option()
removed it as long as a matching virtual server was found.  This resulted
in a segmentation fault similar to the one fixed in a6902a941279 (1.9.8),
affecting nginx built with OpenSSL 1.1.0h or higher.

To fix this, SSL_OP_NO_RENEGOTIATION is now explicitly set in
ngx_http_ssl_servername() after adjusting options.  Additionally, instead
of c->ssl->renegotiation we now check c->ssl->handshaked, which seems
to be a more correct flag to test, and will prevent the segmentation fault
from happening even if SSL_OP_NO_RENEGOTIATION is not working.
2018-10-02 17:46:18 +03:00