This is an attempt to fix#360. The algorithm produces
decimal representations which are guaranteed to roundtrip
and in ~99.8% actually produces the shortest possible
representation. So this is a nice compromise between using
a precision of digits10 and max_digits10.
Note 1:
The implementation only works for IEEE single/double precision
numbers. So the old implementation is kept for compatibility
with non-IEEE implementations and 'long double'.
Note 2:
If number_float_t is 'float', not all serialized numbers can
be recovered using strtod (strtof works, though). (There is
exactly one such number and the result is off by 1 ulp.)
This can be avoided by changing the implementation (the fix
is trivial), but then the resulting decimal numbers are not
exactly short.
Note 3:
The code should be replaced by std::to_chars once it is available.
o We assume the same character int_type as the unerlying std::istream
o There are no assumptions on the value of eof(), other than that it
will not be a valid unsigned char value.
o To retain performance, we do not allow swapping out the underlying
std::streambuf during our use of the std::istream for parsing.
o We can retain -Weffc++ and specify default initializers by using
initializer lists. The risks are low (of additional non-conformat
compilers), because there is already one other such initialization
used in the code-base.
o An (-'ve valued, typically -1) EOF must never be allowed in
token_string, as it be converted to 255 -- a legitimate value.
o Comparing against a specific eof() (-1, typically) is more costly than
detecting +'ve/-'ve. Since EOF is the only non-positive value allowed
we can use the simpler test.
o Removed unnecessary test for token_string size, as it is already
tested in the method, and must never occur in correct code; used an
assert instead.
o Return its contents when necessary. In many cases, this avoids
construction of multiple copies of the yytext token. Exceeds
performance of current develop branch.
o Use std::streambuf I/O instead of std::istream; does not maintain
(unused) istream flags.
o Further simplify get/unget handling.
o Restore original handling of NUL in input stream; ignored during
token_string escaping.
