Home My Page Projects Code Snippets Project Openings SML/NJ Bugs
Summary Activity Tracker Lists

[#222] CM exports from imported library rather than from defined library

Date:
2019-05-21 14:54
Priority:
3
State:
Closed
Submitted by:
Bug Submitter (webuser)
Assigned to:
Matthias Blume (blume)
Machine Architecture:
x86
Operating System:
Linux
Component:
CM
Resolution:
Accepted As Bug
Severity:
Major
OS Version:
Ubuntu 16.04
SML/NJ Version:
110.88
Keywords:
CM
URL:
Transcript (of reproduction):
Standard ML of New Jersey v110.88 [built: Wed May 15 15:59:12 2019] - CM.make "sources.cm"; [autoloading] [library $smlnj/cm/cm.cm is stable] [library $smlnj/internal/cm-sig-lib.cm is stable] [library $/pgraph.cm is stable] [library $smlnj/internal/srcpath-lib.cm is stable] [library $SMLNJ-BASIS/basis.cm is stable] [library $SMLNJ-BASIS/(basis.cm):basis-common.cm is stable] [autoloading done] [scanning sources.cm] [parsing (sources.cm):int.sml] [creating directory .cm/SKEL] [parsing (sources.cm):foo.sml] [compiling (sources.cm):int.sml] [creating directory .cm/GUID] [creating directory .cm/x86-unix] [code: 120, env: 93 bytes] [compiling (sources.cm):foo.sml] [code: 64, env: 93 bytes] [New bindings added.] val it = true : bool - Foo.foo; val it = fn : int -> int - Int.inc; stdIn:3.1-3.8 Error: unbound variable or constructor: inc in path Int.inc - Int.maxInt; val it = SOME 1073741823 : int option Standard ML of New Jersey v110.87 [built: Thu May 09 14:56:50 2019] - CM.make "sources.cm"; [autoloading] [library $smlnj/cm/cm.cm is stable] [library $smlnj/internal/cm-sig-lib.cm is stable] [library $/pgraph.cm is stable] [library $smlnj/internal/srcpath-lib.cm is stable] [library $SMLNJ-BASIS/basis.cm is stable] [library $SMLNJ-BASIS/(basis.cm):basis-common.cm is stable] [autoloading done] [scanning sources.cm] [parsing (sources.cm):int.sml] [creating directory .cm/SKEL] [parsing (sources.cm):foo.sml] [compiling (sources.cm):int.sml] [creating directory .cm/GUID] [creating directory .cm/x86-unix] [code: 120, env: 93 bytes] [compiling (sources.cm):foo.sml] [code: 64, env: 93 bytes] [New bindings added.] val it = true : bool - Foo.foo; val it = fn : int -> int - Int.inc; val it = fn : int -> int - Int.maxInt; stdIn:4.1-4.11 Error: unbound variable or constructor: maxInt in path Int.maxInt
Source (for reproduction):
(* sources.cm *) Library structure Char (* commenting out this export results in the correct `Int` being exported *) structure Int structure Foo is $/basis.cm int.sml foo.sml (* int.sml *) structure Int = struct (* open Int *) fun inc (x: int) = x + 1 end (* foo.sml *) structure Foo = struct val foo = Int.inc end
Summary:
CM exports from imported library rather than from defined library

Detailed description
With SML/NJ 110.88, there seems to be a problem with CM sometimes
exporting a module from an imported library rather than exporting
the locally defined module of the same name.

Additional comments:
It seems to be related to the library exporting both a module directly from the
imported library and exporting a shadowing module; as noted above, if the
`structure Char` export is commented, then the correct `structure Int` is
exported. Curiously, though, some direct exports from the imported library
don't cause the wrong `structure Int` to be exported; for example, if `structure
Char` is replaced by `structure List`, then the correct `structure Int` is
exported.

Also, note that uses of `structure Int` within the library are correctly
resolved; `structure Foo` is correctly defined using the library-defined
`structure Int` (rather than the imported `structure Int` from `$/basis.cm`).

This bug occurs "in the wild" with MLton's source code; in particular, MLton's
utility library redefines/extends much of the Basis Library, but a few
structures are exported directly from the Basis Library.

Submitted via web form by Matthew Fluet <Matthew.Fluet@gmail.com>

Comments:

Message  ↓
Date: 2019-06-08 23:06
Sender: Matthew Fluet

Thanks Matthias!
-Matthew

Date: 2019-06-08 14:56
Sender: Matthias Blume

Svn commit 5348 fixes this.

Date: 2019-06-07 05:29
Sender: Matthias Blume

Update: I have convinced myself that this is, indeed, a bug in CM - probably one that has been there for 20 years. The reorganization of bindings in the Basis has exposed it.
I am still working on understanding the exact mechanics of what's going on. Once I have this I will provide a fix.

I suspect that symbol filtering at the "root" of a traversal (i.e., where CM.make eventually introduces bindings into the interactive environment) is broken while the same scenario at an internal node of the traversal is handled correctly.

Additional info: I figured out what the significance of structure Char is: Both structure Char and structure Int are now defined in the same source file. Re-exporting any other structure defined in that same file has the same effect (triggering the bug). Re-exporting a symbol that is exported by the Basis but not defined in that same file (e.g., structure Array) does not trigger the effect.

Date: 2019-06-05 01:20
Sender: Matthias Blume

This bug first appears in version 110.88. It was absent in 110.87.

My guess is that something is wrong with the new primop implementation, and that that has some subtle effect on the way the compiler's internal environments work.

Side note: The compiler's internal representation of environments seems ancient and in need of modernization. For example, we could use red-black maps from the smlnj library. The current implementation internally makes use of Array and Vector and some index arithmetic.
See base/compiler/ElabData/basics/env.sml

Date: 2019-05-27 18:11
Sender: Matthias Blume

I'm looking into this now.

First observation: This seems to happen only at the interactive toplevel.
When I create another library that wraps the above sources.cm, then within that library's source the correct version of structure Int is visible.

(* Other files as above, sources.cm re-exporting Char *)

(* bar.cm *)
Library Bar
structure Foo
structure Int
structure Bar
is
sources.cm
bar.sml

(* bar.sml *)
structure Bar = struct
val foo = Foo.foo
val inc = Int.inc
end


Date: 2019-05-27 14:00
Sender: John Reppy

There were no changes to CM in 110.88 (in fact, there have not been any changes to CM in the last year). Therefore, I expect that this bug was triggered by the reorganization of the Basis Library to make it more target-size neutral.

Attached Files:

Changes

Field Old Value Date By
status_idOpen2019-06-08 14:56blume
close_dateNone2019-06-08 14:56blume
assigned_tonone2019-05-27 14:00jhr
detailsWith SML/NJ 110.88, there seems to be a problem with CM sometimes exporting a module from an imported library rather than exporting the locally defined module of the same name. Additional comments: It seems to be related to the library exporting both a module directly from the imported library and exporting a shadowing module; as noted above, if the `structure Char` export is commented, then the correct `structure Int` is exported. Curiously, though, some direct exports from the imported library don't cause the wrong `structure Int` to be exported; for example, if `structure Char` is replaced by `structure List`, then the correct `structure Int` is exported. Also, note that uses of `structure Int` within the library are correctly resolved; `structure Foo` is correctly defined using the library-defined `structure Int` (rather than the imported `structure Int` from `$/basis.cm`). This bug occurs "in the wild" with MLton's source code; in particular, MLton's utility library redefines/extends much of the Basis Library, but a few structures are exported directly from the Basis Library. Submitted via web form by Matthew Fluet <Matthew.Fluet@gmail.com> 2019-05-27 14:00jhr
ResolutionNone2019-05-27 14:00jhr
Transcript (of reproduction)Standard ML of New Jersey v110.88 [built: Wed May 15 15:59:12 2019] - CM.make "sources.cm"; [autoloading] [library $smlnj/cm/cm.cm is stable] [library $smlnj/internal/cm-sig-lib.cm is stable] [library $/pgraph.cm is stable] [library $smlnj/internal/srcpath-lib.cm is stable] [library $SMLNJ-BASIS/basis.cm is stable] [library $SMLNJ-BASIS/(basis.cm):basis-common.cm is stable] [autoloading done] [scanning sources.cm] [parsing (sources.cm):int.sml] [creating directory .cm/SKEL] [parsing (sources.cm):foo.sml] [compiling (sources.cm):int.sml] [creating directory .cm/GUID] [creating directory .cm/x86-unix] [code: 120, env: 93 bytes] [compiling (sources.cm):foo.sml] [code: 64, env: 93 bytes] [New bindings added.] val it = true : bool - Foo.foo; val it = fn : int -> int - Int.inc; stdIn:3.1-3.8 Error: unbound variable or constructor: inc in path Int.inc - Int.maxInt; val it = SOME 1073741823 : int option Standard ML of New Jersey v110.87 [built: Thu May 09 14:56:50 2019] - CM.make "sources.cm"; [autoloading] [library $smlnj/cm/cm.cm is stable] [library $smlnj/internal/cm-sig-lib.cm is stable] [library $/pgraph.cm is stable] [library $smlnj/internal/srcpath-lib.cm is stable] [library $SMLNJ-BASIS/basis.cm is stable] [library $SMLNJ-BASIS/(basis.cm):basis-common.cm is stable] [autoloading done] [scanning sources.cm] [parsing (sources.cm):int.sml] [creating directory .cm/SKEL] [parsing (sources.cm):foo.sml] [compiling (sources.cm):int.sml] [creating directory .cm/GUID] [creating directory .cm/x86-unix] [code: 120, env: 93 bytes] [compiling (sources.cm):foo.sml] [code: 64, env: 93 bytes] [New bindings added.] val it = true : bool - Foo.foo; val it = fn : int -> int - Int.inc; val it = fn : int -> int - Int.maxInt; stdIn:4.1-4.11 Error: unbound variable or constructor: maxInt in path Int.maxInt2019-05-27 14:00jhr
Source (for reproduction)(* sources.cm *) Library structure Char (* commenting out this export results in the correct `Int` being exported *) structure Int structure Foo is $/basis.cm int.sml foo.sml (* int.sml *) structure Int = struct (* open Int *) fun inc (x: int) = x + 1 end (* foo.sml *) structure Foo = struct val foo = Int.inc end 2019-05-27 14:00jhr