Home My Page Projects Code Snippets Project Openings SML/NJ Bugs
Summary Activity Tracker Lists

[#254] Real.fromLargeInt produces negative results

Date:
2020-04-14 19:28
Priority:
3
State:
Closed
Submitted by:
Bug Submitter (webuser)
Assigned to:
John Reppy (jhr)
Machine Architecture:
x86
Operating System:
MacOS X
Component:
Basis Library
Resolution:
Fixed
Severity:
Major
OS Version:
10.15.3
SML/NJ Version:
110.96
Keywords:
URL:
Transcript (of reproduction):
Standard ML of New Jersey (64-bit) v110.96 [built: Fri Dec 13 14:59:34 2019] - valOf Int.maxInt; [autoloading] [library $SMLNJ-BASIS/basis.cm is stable] [library $SMLNJ-BASIS/(basis.cm):basis-common.cm is stable] [autoloading done] val it = 4611686018427387903 : int - Real.fromLargeInt 4611686018427387903; val it = 4.61168601843E18 : real - Real.fromLargeInt 4611686018427387904; val it = ~4.61168601843E18 : real
Source (for reproduction):
Summary:
Real.fromLargeInt produces negative results

Detailed description
Real.fromInt returns a negative result when the number is bigger than Int.maxInt
Additional comments:
A student had the same problem on Windows 10, where maxInt was 2^30-1, and the wrap-around occurred for 2^30..

Submitted via web form by Dan Licata <dlicata@wesleyan.edu>

Comments:

Message  ↓
Date: 2020-12-19 19:19
Sender: John Reppy

The problem is because the digit size for ``IntInf.int is only one bit smaller than the default int and the scaling factor `rbase` was being computed using the `InlineT.Real64.from_int` function (so `rbase` ends up being negative). Thus it would return incorrect results whenever the IntInf representation involved more than one digit. This is a bug on both 32-bit and 64-bit systems. The fix is to switch to `InlineT.Real64.from_int{32,64}`.

Fixed for 110.99

Attached Files:

Changes

Field Old Value Date By
status_idOpen2020-12-19 19:19jhr
close_dateNone2020-12-19 19:19jhr
summaryReal.fromLargeInt in 110.962020-12-19 19:19jhr
ResolutionAccepted As Bug2020-12-19 19:19jhr
assigned_tonone2020-04-14 22:19jhr
detailsReal.fromInt returns a negative result when the number is bigger than Int.maxInt Additional comments: A student had the same problem on Windows 10, where maxInt was 2^30-1, and the wrap-around occurred for 2^30.. Submitted via web form by Dan Licata <dlicata@wesleyan.edu> 2020-04-14 22:19jhr
ResolutionNone2020-04-14 22:19jhr
Transcript (of reproduction)Standard ML of New Jersey (64-bit) v110.96 [built: Fri Dec 13 14:59:34 2019] - valOf Int.maxInt; [autoloading] [library $SMLNJ-BASIS/basis.cm is stable] [library $SMLNJ-BASIS/(basis.cm):basis-common.cm is stable] [autoloading done] val it = 4611686018427387903 : int - Real.fromLargeInt 4611686018427387903; val it = 4.61168601843E18 : real - Real.fromLargeInt 4611686018427387904; val it = ~4.61168601843E18 : real2020-04-14 22:19jhr