Fix error-handling of strtoimax boundary conditions

strtoimax is only required to set errno if there is an
underflow/overflow.  In those conditions, strtoimax returns
INTMAX_MIN/INTMAX_MAX respectively, so that's the only time we should be
checking the value of errno.

Even in those conditions, errno needs to be set to a known good value
before calling strtoimax to differentiate between "value is actually
INTMAX_MAX/MIN" and "value over/underflows".

Closes #5279
This commit is contained in:
James McCoy 2016-09-02 10:02:49 -04:00
parent f175b281cf
commit a371f1027e

View File

@ -1739,8 +1739,11 @@ char_u* skiptowhite_esc(char_u *p) {
/// @return Number read from the string.
intmax_t getdigits(char_u **pp)
{
errno = 0;
intmax_t number = strtoimax((char *)*pp, (char **)pp, 10);
if (number == INTMAX_MAX || number == INTMAX_MIN) {
assert(errno != ERANGE);
}
return number;
}