Default math scale to 6

This changes the behavior of builtin math to floating point by default.
If the result of a computation is an integer, then it will be printed as an
integer; otherwise it will be printed as a floating point decimal with up to
'scale' digits past the decimal point (default is 6, matching printf).
Trailing zeros are trimmed. Values are rounded following printf semantics.

Fixes #4478
This commit is contained in:
ridiculousfish
2018-07-24 00:00:06 -07:00
parent 78cac07d3c
commit d2bee105c9
5 changed files with 81 additions and 26 deletions

View File

@@ -2,6 +2,7 @@ logmsg Validate basic expressions
math 3 / 2
math 10/6
math -s0 10 / 6
math 'floor(10 / 6)'
math -s3 10/6
math '10 % 6'
math -s0 '10 % 6'
@@ -14,6 +15,19 @@ math 5 \* -2
math -- -4 / 2
math -- '-4 * 2'
logmsg Validate some integral computations
math 1
math 10
math 100
math 1000
math '10^15'
math '-10^14'
math '-10^15'
math -s0 '1.0 / 2.0'
math -s0 '3.0 / 2.0'
math -s0 '10^15 / 2.0'
logmsg Validate how variables in an expression are handled
math $x + 1
set x 1
@@ -21,7 +35,7 @@ math $x + 1
set x 3
set y 1.5
math "-$x * $y"
math -s1 "-$x * $y"
math -s0 "-$x * $y"
logmsg Validate math error reporting
not math '2 - '
@@ -31,5 +45,5 @@ not math 'sin()'
not math '2 + 2 4'
not math
not math -s 12
not math 2^650
not math 2^999999
not math 1 / 0