Commit Graph

6076 Commits

Author SHA1 Message Date
Dmitry Volyntsev
87aaac4ac5 Stream: postpone session initialization under accept mutex.
Previously, it was possible that some system calls could be
invoked while holding the accept mutex.  This is clearly
wrong as it prevents incoming connections from being accepted
as quickly as possible.
2016-09-06 21:28:13 +03:00
Vladimir Homutov
68a7b9b5a3 Stream: log module. 2016-09-05 17:50:16 +03:00
Vladimir Homutov
443b52db59 Stream: upstream response time variables.
The $upstream_connect_time, $upstream_first_byte_time and
$upstream_session_time variables keep corresponding times.
2016-09-02 18:27:12 +03:00
Vladimir Homutov
64223df670 Stream: $upstream_bytes_sent and $upstream_bytes_received. 2016-09-02 18:27:08 +03:00
Vladimir Homutov
c6d456da87 Stream: the $upstream_addr variable.
Keeps the full address of the upstream server.  If several servers were
contacted during proxying, their addresses are separated by commas,
e.g. "192.168.1.1:80, 192.168.1.2:80".
2016-09-02 18:27:05 +03:00
Roman Arutyunyan
be6024f9b7 Stream: the $status variable.
The stream session status is one of the following:

200 - normal completion
403 - access forbidden
500 - internal server error
502 - bad gateway
503 - limit conn
2016-08-11 20:22:23 +03:00
Maxim Dounin
ea9a1d745b Event pipe: do not set file's thread_handler if not needed.
This fixes a problem with aio threads and sendfile with aio_write switched
off, as observed with range requests after fc72784b1f52 (1.9.13).  Potential
problems with sendfile in threads were previously described in 9fd738b85fad,
and this seems to be one of them.

The problem occurred as file's thread_handler was set to NULL by event pipe
code after a sendfile thread task was scheduled.  As a result, no sendfile
completion code was executed, and the same buffer was additionally sent
using non-threaded sendfile.  Fix is to avoid modifying file's thread_handler
if aio_write is switched off.

Note that with "aio_write on" it is still possible that sendfile will use
thread_handler as set by event pipe.  This is believed to be safe though,
as handlers used are compatible.
2016-09-01 20:05:23 +03:00
Maxim Dounin
4e8a73adaa Event pipe: process data after recv_chain() errors.
When c->recv_chain() returns an error, it is possible that we already
have some data previously read, e.g., in preread buffer.  And in some
cases it may be even a complete response.  Changed c->recv_chain() error
handling to process the data, much like it is already done if kevent
reports about an error.

This change, in particular, fixes processing of small responses
when an upstream fails to properly close a connection with lingering and
therefore the connection is reset, but the response is already fully
obtained by nginx (see ticket #1037).
2016-09-01 18:29:55 +03:00
Roman Arutyunyan
c6ba4aae38 Realip: fixed uninitialized memory access.
Previously, the realip module could be left with uninitialized context after an
error in the ngx_http_realip_set_addr() function.  That context could be later
accessed by $realip_remote_addr and $realip_remote_port variable handlers.
2016-09-01 14:33:51 +03:00
Vladimir Homutov
048ee94130 Stream: the $protocol variable.
The variable keeps protocol used by the client, "TCP" or "UDP".
2016-08-26 15:33:07 +03:00
Vladimir Homutov
f04b65358e Stream: the $session_time variable.
The variable keeps time spent on processing the stream session.
2016-08-26 15:33:04 +03:00
Vladimir Homutov
1258126f0c Stream: the $bytes_received variable.
The variable keeps the number of bytes received from the client.
2016-08-26 15:33:02 +03:00
Piotr Sikora
14b1b6e10a Thread pools: create threads in detached state.
This prevents theoretical resource leak, since those threads are never joined.

Found with ThreadSanitizer.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
2016-08-15 05:52:04 -07:00
Sergey Kandaurov
78031177f4 Geo: fixed indentation. 2016-08-25 12:59:39 +03:00
Maxim Dounin
0b97b1ec6f Contrib: unicode2nginx compatibility with recent Perl versions.
In recent Perl versions unpack("C*") unpacks wide characters by default,
likely since perl 5.10 (seen at least in perl 5.20).  Replaced with
unpack("U0C*") instead to unpack bytes.

While here, improved style and updated my email.
2016-08-24 15:53:17 +03:00
Ruslan Ermilov
9208875db1 Geo: fixed warnings when removing nonexistent ranges.
geo $geo {
    ranges;

    10.0.0.0-10.0.0.255 test;

    delete 10.0.1.0-10.0.1.255;     # should warn

    delete 10.0.0.0-10.0.0.255;
    delete 10.0.0.0-10.0.0.255;     # should warn
}
2016-08-23 15:59:42 +03:00
Ruslan Ermilov
20de5f14e5 Geo: fixed insertion of ranges specified in descending order. 2016-08-23 15:59:14 +03:00
Ruslan Ermilov
f927ab882a Geo: fixed removing a range in certain cases.
If the range includes two or more /16 networks and does
not start at the /16 boundary, the last subrange was not
removed (see 91cff7f97a50 for details).
2016-08-23 15:59:06 +03:00
Ruslan Ermilov
36e1c887db Geo: fixed overflow when iterating over ranges. 2016-08-23 15:57:29 +03:00
Sergey Kandaurov
218c8d493f SSL: adopted session ticket handling for OpenSSL 1.1.0.
Return 1 in the SSL_CTX_set_tlsext_ticket_key_cb() callback function
to indicate that a new session ticket is created, as per documentation.
Until 1.1.0, OpenSSL didn't make a distinction between non-negative
return values.

See https://git.openssl.org/?p=openssl.git;a=commitdiff;h=5c753de for details.
2016-08-22 18:53:21 +03:00
Piotr Sikora
c7c5c43c04 SSL: remove no longer needed workaround for BoringSSL.
BoringSSL added a no-op stub for OPENSSL_config() on 2016-01-26.

Requested by David Benjamin.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
2016-08-18 14:49:48 -07:00
Ruslan Ermilov
d4f7011688 Events: fixed setting of IP_BIND_ADDRESS_NO_PORT/SO_REUSEADDR.
The IP_BIND_ADDRESS_NO_PORT option is set on upstream sockets
if proxy_bind does not specify a port.  The SO_REUSEADDR option
is set on UDP upstream sockets if proxy_bind specifies a port.

Due to checking of the wrong port, IP_BIND_ADDRESS_NO_PORT was
never set, and SO_REUSEPORT was always set.
2016-08-22 11:40:10 +03:00
Roman Arutyunyan
dabbf30377 Ensure "listen" exists in a mail or stream server (ticket #1049). 2016-08-17 11:26:51 +03:00
Roman Arutyunyan
eecc68ce17 Fixed wrong type of the "line" field.
The new type ngx_uint_t was supposed when formatting the line number.
2016-08-18 17:13:07 +03:00
Valentin Bartenev
535550b406 Geo: fixed access to already freed memory.
Previously, in "ranges" mode when all added ranges were deleted,
the ctx.high.low[i] was left pointing to a temporary array.
2016-08-18 17:11:03 +03:00
Vladimir Homutov
50ba1a2cde Upstream: the $upstream_bytes_received variable.
Unlike $upstream_response_length that only counts the body size,
the new variable also counts the size of response header and data
received after switching protocols when proxying WebSockets.
2016-08-10 16:46:39 +03:00
Ruslan Ermilov
47e72bf7e0 Simplified extraction of current time. 2016-08-08 17:11:29 +03:00
Sergey Kandaurov
af9e72533a SSL: guarded SSL_R_NO_CIPHERS_PASSED not present in OpenSSL 1.1.0.
It was removed in OpenSSL 1.1.0 Beta 3 (pre-release 6).  It was
not used since OpenSSL 1.0.1n and 1.0.2b.
2016-08-08 13:44:49 +03:00
Ruslan Ermilov
42f6e1f78e Always seed PRNG with PID, seconds, and milliseconds. 2016-08-04 23:43:10 +03:00
Ruslan Ermilov
31389e4921 Fixed undefined behavior when left shifting signed integer. 2016-08-04 23:42:00 +03:00
Ruslan Ermilov
e62a16e8f6 Win32: added per-thread random seeding.
The change in b91bcba29351 was not enough to fix random() seeding.
On Windows, the srand() seeds the PRNG only in the current thread,
and worse, is not inherited from the calling thread.  Due to this,
worker threads were not properly seeded.

Reported by Marc Bevand.
2016-08-04 01:15:41 +03:00
Vladimir Homutov
38ca99cf98 Stream: fixed build without stream_ssl_module (ticket #1032). 2016-07-26 19:34:12 +03:00
Vladimir Homutov
386e3e80bb Version bump. 2016-07-26 19:07:18 +03:00
Maxim Dounin
5fb57611b5 release-1.11.3 tag 2016-07-26 16:58:58 +03:00
Maxim Dounin
1d2f499889 nginx-1.11.3-RELEASE 2016-07-26 16:58:58 +03:00
Vladimir Homutov
161fcf4bdd Fixed regex captures handling without PCRE.
If PCRE is disabled, captures were treated as normal variables in
ngx_http_script_compile(), while code calculating flushes array length in
ngx_http_compile_complex_value() did not account captures as variables.
This could lead to write outside of the array boundary when setting
last element to -1.

Found with AddressSanitizer.
2016-07-06 14:33:40 +03:00
Vladimir Homutov
74305af672 Stream: variables in proxy_pass and proxy_ssl_name. 2016-06-14 18:29:46 +03:00
Valentin Bartenev
a85edfeef6 HTTP/2: flushing of the SSL buffer in transition to the idle state.
It fixes potential connection leak if some unsent data was left in the SSL
buffer.  Particularly, that could happen when a client canceled the stream
after the HEADERS frame has already been created.  In this case no other
frames might be produced and the HEADERS frame alone didn't flush the buffer.
2016-07-19 20:34:17 +03:00
Valentin Bartenev
3c81c08cea HTTP/2: refactored ngx_http_v2_send_output_queue().
Now it returns NGX_AGAIN if there's still data to be sent.
2016-07-19 20:34:02 +03:00
Valentin Bartenev
3b2f54bc2e HTTP/2: fixed send timer handling.
Checking for return value of c->send_chain() isn't sufficient since there
are data can be left in the SSL buffer.  Now the wew->ready flag is used
instead.

In particular, this fixed a connection leak in cases when all streams were
closed, but there's still some data to be sent in the SSL buffer and the
client forgot about the connection.
2016-07-19 20:31:09 +03:00
Valentin Bartenev
ce6eb33d15 HTTP/2: avoid sending output queue if there's nothing to send.
Particularly this fixes alerts on OS X and NetBSD systems when HTTP/2 is
configured over plain TCP sockets.

On these systems calling writev() with no data leads to EINVAL errors
being logged as "writev() failed (22: Invalid argument) while processing
HTTP/2 connection".
2016-07-19 20:30:21 +03:00
Valentin Bartenev
da852aa468 HTTP/2: always handle streams in error state.
Previously, a stream could be closed by timeout if it was canceled
while its send window was exhausted.
2016-07-19 20:22:44 +03:00
Valentin Bartenev
19de85a4d7 HTTP/2: prevented output of the HEADERS frame for canceled streams.
It's useless to generate HEADERS if the stream has been canceled already.
2016-07-19 20:22:44 +03:00
Valentin Bartenev
ab5401d204 HTTP/2: always send GOAWAY while worker is shutting down.
Previously, if the worker process exited, GOAWAY was sent to connections in
idle state, but connections with active streams were closed without GOAWAY.
2016-07-19 20:22:44 +03:00
Valentin Bartenev
5c2dd3913a Events: support for EPOLLEXCLUSIVE.
This flag appeared in Linux 4.5 and is useful for avoiding thundering herd
problem.

The current Linux kernel implementation walks the list of exclusive waiters,
and queues an event to each epfd, until it finds the first waiter that has
threads blocked on it via epoll_wait().
2016-07-15 15:18:57 +03:00
Valentin Bartenev
b60534e0d8 Style: sorted epoll flags. 2016-07-15 15:18:57 +03:00
Valentin Bartenev
46dd747c9e Events: the "accept_mutex" directive is turned off by default.
Now it is believed that the accept mutex brings more harm than benefits.
Especially in various benchmarks it often results in situation where only
one worker grabs all connections.
2016-07-15 15:18:57 +03:00
Vladimir Homutov
6c2b086d0e Stream: split_clients module. 2016-07-12 17:34:52 +03:00
Vladimir Homutov
bb790f5d30 Stream: geo module. 2016-06-30 16:12:50 +03:00
Vladimir Homutov
4cf0e28483 Stream: geoip module. 2016-07-12 17:34:43 +03:00