With this change, all script copy operations now check if there is
enough room in the buffer. To do so, the script engine now provides the
e->end pointer, which specifies expected buffer end, and each copy
operation is checked against it with the ngx_http_script_check_length()
function.
The e->end pointer is optional and only checked when set, thus
introducing no incompatible API changes. All standard functions were
updated to use it, notably ngx_http_complex_value(), ngx_http_script_run(),
ngx_http_script_regex_start_code(), ngx_http_script_complex_value_code().
Direct script evaluation in the proxy, fastcgi, scgi, uwsgi, grpc proxy,
index, and try_files modules will be updated by a separate patch.
In particular, this catches issues as observed when evaluating variables
with side effects, such as in the following configuration:
map $uri $map {
~(?<capture>.*) $capture;
}
set $capture "";
set $temp "$capture $map";
As well as when evaluating non-cacheable variables, where length of a
variable might change between length and copy codes, such as in the
following configuration:
map prefix:$capture $map_volatile {
volatile;
~(?<capture>.*) $capture;
}
set $capture "";
set $temp "$map_volatile";
Similar changes were made in the stream module.
Signed-off-by: Roman Arutyunyan <arut@nginx.com>
Origin: <https://freenginx.org/hg/nginx/rev/d172f506ab5f>
If a complex value is expected to be of type size_t, and the compiled
value is constant, the constant size_t value is remembered at compile
time.
The value is accessed through ngx_http_complex_value_size() which
either returns the remembered constant or evaluates the expression
and parses it as size_t.
There are lots of C4244 warnings (conversion from 'type1' to 'type2',
possible loss of data), so they were disabled.
The same applies to C4267 warnings (conversion from 'size_t' to 'type',
possible loss of data), most notably - conversion from ngx_str_t.len to
ngx_variable_value_t.len (which is unsigned:28). Additionally, there
is at least one case when it is not possible to fix the warning properly
without introducing win32-specific code: recv() on win32 uses "int len",
while POSIX defines "size_t len".
The ssize_t type now properly defined for 64-bit compilation with MSVC.
Caught by warning C4305 (truncation from '__int64' to 'ssize_t'), on
"cutoff = NGX_MAX_SIZE_T_VALUE / 10" in ngx_atosz()).
Several C4334 warnings (result of 32-bit shift implicitly converted to 64 bits)
were fixed by adding explicit conversions.
Several C4214 warnings (nonstandard extension used: bit field types other
than int) in ngx_http_script.h fixed by changing bit field types from
uintptr_t to unsigned.
*) Feature: the "dav_access" directive.
*) Feature: the "if" directive supports the "-d", "!-d", "-e", "!-e",
"-x", and "!-x" operators.
*) Bugfix: a segmentation fault occurred if a request returned a
redirect and some sent to client header lines were logged in the
access log.
*) Feature: the "bind" option of the "listen" directive in IMAP/POP3
proxy.
*) Bugfix: if the same capture in the "rewrite" directive was used more
then once.
*) Bugfix: the $sent_http_content_type, $sent_http_content_length,
$sent_http_last_modified, $sent_http_connection,
$sent_http_keep_alive, and $sent_http_transfer_encoding variables
were not written to access log.
*) Bugfix: the $sent_http_cache_control returned value of the single
"Cache-Control" response header line.
*) Feature: the ngx_http_dav_module supports the MKCOL method.
*) Feature: the "create_full_put_path" directive.
*) Feature: the "$limit_rate" variable.
*) Feature: the "limit_except" directive.
*) Feature: the "if" directive supports the "!~", "!~*", "-f", and
"!-f" operators.
*) Feature: the ngx_http_perl_module supports the $r->request_body
method.
*) Bugfix: in the ngx_http_addition_filter_module.
*) Feature: the "http_503" parameter of the "proxy_next_upstream" or
"fastcgi_next_upstream" directives.
*) Bugfix: ngx_http_perl_module did not work with inlined in the
configuration code, if it was not started with the "sub" word.
*) Bugfix: in the "post_action" directive.
*) Change: the "valid_referers" directive and the "$invalid_referer"
variable were moved to the new ngx_http_referer_module from the
ngx_http_rewrite_module.
*) Change: the "$apache_bytes_sent" variable name was changed to
"$body_bytes_sent".
*) Feature: the "$sent_http_..." variables.
*) Feature: the "if" directive supports the "=" and "!=" operations.
*) Feature: the "proxy_pass" directive supports the HTTPS protocol.
*) Feature: the "proxy_set_body" directive.
*) Feature: the "post_action" directive.
*) Feature: the ngx_http_empty_gif_module.
*) Feature: the "worker_cpu_affinity" directive for Linux.
*) Bugfix: the "rewrite" directive did not unescape URI part in
redirect, now it is unescaped except the %00-%25 and %7F-%FF
characters.
*) Bugfix: nginx could not be built by the icc 9.0 compiler.
*) Bugfix: if the SSI was enabled for zero size static file, then the
chunked response was encoded incorrectly.
*) Security: nginx now checks URI got from a backend in
"X-Accel-Redirect" header line or in SSI file for the "/../" paths
and zeroes.
*) Change: nginx now does not treat the empty user name in the
"Authorization" header line as valid one.
*) Feature: the "ssl_session_timeout" directives of the
ngx_http_ssl_module and ngx_imap_ssl_module.
*) Feature: the "auth_http_header" directive of the
ngx_imap_auth_http_module.
*) Feature: the "add_header" directive.
*) Feature: the ngx_http_realip_module.
*) Feature: the new variables to use in the "log_format" directive:
$bytes_sent, $apache_bytes_sent, $status, $time_gmt, $uri,
$request_time, $request_length, $upstream_status,
$upstream_response_time, $gzip_ratio, $uid_got, $uid_set,
$connection, $pipe, and $msec. The parameters in the "%name" form
will be canceled soon.
*) Change: now the false variable values in the "if" directive are the
empty string "" and string starting with "0".
*) Bugfix: while using proxied or FastCGI-server nginx may leave
connections and temporary files with client requests in open state.
*) Bugfix: the worker processes did not flush the buffered logs on
graceful exit.
*) Bugfix: if the request URI was changes by the "rewrite" directive
and the request was proxied in location given by regular expression,
then the incorrect request was transferred to backend; the bug had
appeared in 0.2.6.
*) Bugfix: the "expires" directive did not remove the previous
"Expires" header.
*) Bugfix: nginx may stop to accept requests if the "rtsig" method and
several worker processes were used.
*) Bugfix: the "\"" and "\'" escape symbols were incorrectly handled in
SSI commands.
*) Bugfix: if the response was ended just after the SSI command and
gzipping was used, then the response did not transferred complete or
did not transferred at all.
*) Feature: the "limit_rate" directive is supported in in proxy and
FastCGI mode.
*) Feature: the "X-Accel-Limit-Rate" response header line is supported
in proxy and FastCGI mode.
*) Feature: the "break" directive.
*) Feature: the "log_not_found" directive.
*) Bugfix: the response status code was not changed when request was
redirected by the ""X-Accel-Redirect" header line.
*) Bugfix: the variables set by the "set" directive could not be used
in SSI.
*) Bugfix: the segmentation fault may occurred if the SSI page has more
than one remote subrequest.
*) Bugfix: nginx treated the backend response as invalid if the status
line in the header was transferred in two packets; the bug had
appeared in 0.1.29.
*) Feature: the "ssi_types" directive.
*) Feature: the "autoindex_exact_size" directive.
*) Bugfix: the ngx_http_autoindex_module did not support the long file
names in UTF-8.
*) Feature: the IMAP/POP3 proxy.
*) Bugfix: nginx could not be built with the --without-pcre parameter;
the bug had appeared in 0.1.29.
*) Bugfix: 3, 4, 7, and 8 the "proxy_set_header" directives in one
level cause the bus fault on start up.
*) Bugfix: the HTTP protocol was specified in the HTTPS redirects.
*) Bugfix: if the "rewrite" directive used the captures inside the "if"
directive, then the 500 error code was returned.
*) Bugfix: the arguments were omitted in the redirects, issued by the
"rewrite" directive; the bug had appeared in 0.1.29.
*) Feature: the "if" directive supports the captures in regular
expressions.
*) Feature: the "set" directive supports the variables and the captures
of regular expressions.
*) Feature: the "X-Accel-Redirect" response header line is supported in
proxy and FastCGI mode.
*) Feature: the ngx_http_ssi_module supports "include virtual" command.
*) Feature: the ngx_http_ssi_module supports the condition command like
'if expr="$NAME"' and "else" and "endif" commands. Only one nested
level is supported.
*) Feature: the ngx_http_ssi_module supports the DATE_LOCAL and
DATE_GMT variables and "config timefmt" command.
*) Feature: the "ssi_ignore_recycled_buffers" directive.
*) Bugfix: the "echo" command did not show the default value for the
empty QUERY_STRING variable.
*) Change: the ngx_http_proxy_module was rewritten.
*) Feature: the "proxy_redirect", "proxy_pass_request_headers",
"proxy_pass_request_body", and "proxy_method" directives.
*) Feature: the "proxy_set_header" directive. The "proxy_x_var" was
canceled and must be replaced with the proxy_set_header directive.
*) Change: the "proxy_preserve_host" is canceled and must be replaced
with the "proxy_set_header Host $host" and the "proxy_redirect off"
directives, the "proxy_set_header Host $host:$proxy_port" directive
and the appropriate proxy_redirect directives.
*) Change: the "proxy_set_x_real_ip" is canceled and must be replaced
with the "proxy_set_header X-Real-IP $remote_addr" directive.
*) Change: the "proxy_add_x_forwarded_for" is canceled and must be
replaced with
the "proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for"
directive.
*) Change: the "proxy_set_x_url" is canceled and must be replaced with
the "proxy_set_header X-URL http://$host:$server_port$request_uri"
directive.
*) Feature: the "fastcgi_param" directive.
*) Change: the "fastcgi_root", "fastcgi_set_var" and "fastcgi_params"
directive are canceled and must be replaced with the fastcgi_param
directives.
*) Feature: the "index" directive can use the variables.
*) Feature: the "index" directive can be used at http and server levels.
*) Change: the last index only in the "index" directive can be absolute.
*) Feature: the "rewrite" directive can use the variables.
*) Feature: the "internal" directive.
*) Feature: the CONTENT_LENGTH, CONTENT_TYPE, REMOTE_PORT, SERVER_ADDR,
SERVER_PORT, SERVER_PROTOCOL, DOCUMENT_ROOT, SERVER_NAME,
REQUEST_METHOD, REQUEST_URI, and REMOTE_USER variables.
*) Change: nginx now passes the invalid lines in a client request
headers or a backend response header.
*) Bugfix: if the backend did not transfer response for a long time and
the "send_timeout" was less than "proxy_read_timeout", then nginx
returned the 408 response.
*) Bugfix: the segmentation fault was occurred if the backend sent an
invalid line in response header; the bug had appeared in 0.1.26.
*) Bugfix: the segmentation fault may occurred in FastCGI fault
tolerance configuration.
*) Bugfix: the "expires" directive did not remove the previous
"Expires" and "Cache-Control" headers.
*) Bugfix: nginx did not take into account trailing dot in "Host"
header line.
*) Bugfix: the ngx_http_auth_module did not work under Linux.
*) Bugfix: the rewrite directive worked incorrectly, if the arguments
were in a request.
*) Bugfix: nginx could not be built on MacOS X.
*) Feature: the autoconfiguration directives:
--http-client-body-temp-path=PATH, --http-proxy-temp-path=PATH, and
--http-fastcgi-temp-path=PATH
*) Change: the directory name for the temporary files with the client
request body is specified by directive client_body_temp_path, by
default it is <prefix>/client_body_temp.
*) Feature: the ngx_http_fastcgi_module and the directives:
fastcgi_pass, fastcgi_root, fastcgi_index, fastcgi_params,
fastcgi_connect_timeout, fastcgi_send_timeout, fastcgi_read_timeout,
fastcgi_send_lowat, fastcgi_header_buffer_size, fastcgi_buffers,
fastcgi_busy_buffers_size, fastcgi_temp_path,
fastcgi_max_temp_file_size, fastcgi_temp_file_write_size,
fastcgi_next_upstream, and fastcgi_x_powered_by.
*) Bugfix: the "[alert] zero size buf" error; the bug had appeared in
0.1.3.
*) Change: the URI must be specified after the host name in the
proxy_pass directive.
*) Change: the %3F symbol in the URI was considered as the argument
string start.
*) Feature: the unix domain sockets support in the
ngx_http_proxy_module.
*) Feature: the ssl_engine and ssl_ciphers directives.
Thanks to Sergey Skvortsov for SSL-accelerator.
Maxim Dounin
Ruslan Ermilov
Vladimir Homutov
Maxim Konovalov
Igor Sysoev