Home My Page Projects Code Snippets Project Openings SML/NJ
Summary Activity Forums Tracker Lists Tasks Docs Surveys News SCM Files

SCM Repository

[smlnj] Diff of /sml/trunk/src/system/README
ViewVC logotype

Diff of /sml/trunk/src/system/README

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 537, Fri Feb 18 17:20:16 2000 UTC revision 652, Tue Jun 6 02:14:56 2000 UTC
# Line 1  Line 1 
1  Compiler Hacker's Guide to the new CM...  Compiler Hacker's Guide to the new CM...
2  ========================================  ========================================
3    
4  Last change: 2/2000  Last change: 12/5/2000
5    
6  * Libraries  * Libraries
7  -----------  -----------
# Line 19  Line 19 
19  the compiler ("*" means that I consider the library potentially useful  the compiler ("*" means that I consider the library potentially useful
20  in its own right):  in its own right):
21    
22          * basis.cm      - The SML'97 basis library    * $/basis.cm                          SML'97 Basis Library (pre-loaded)
23          - cm-hook.cm    - an internal library for organizational purposes    * $/smlnj-lib.cm                      SML/NJ Utility Library
24          - cm-lib.cm     - the library that implements CM's functionality    * $/html-lib.cm                       SML/NJ HTML Library
25          * comp-lib.cm   - a helper library for the compiler, MLRISC, and CM    * $/pp-lib.cm                         SML/NJ Pretty-print Library
26          * full-cm.cm    - the library that exports the public interface to  
27                            the compilation manager (i.e., structure CM)    * $/ml-yacc-lib.cm                    SML/NJ ML-Yacc runtime library
28          * minimal-cm.cm - exports a reduced version of structure CM;  
29                            this one is pre-registered at the top level    * $smlnj/compiler/{alpha32,hppa,ppc,sparc,x86}.cm
30          * host-cmb.cm   - the library that exports the public interface to                                          cross-compiler libraries, exporting
31                            the bootstrap compiler (i.e., structure CMB)                                          structure <Arch>Compiler
32          - host-compiler-0.cm    * $smlnj/compiler/current.cm          structure Compiler (current arch)
33                          - an internal library for organizational purposes    * $smlnj/compiler/all.cm              all cross-compilers and all cross-CMBs
34          * host-compiler.cm  
35                          - the library that exports the public interface to    * $smlnj/cm/full.cm                   structure CM (see manual)
36                            the visible compiler (i.e., structure Compiler)    * $smlnj/cm/tools.cm                  CM tools library
37          - intsys.cm     - an internal library for organizational purposes  
38                            (In fact, its the "root" of the main hierarchy.)    * $smlnj/cmb/{alpha32,hppa,ppc,sparc,x86}-unix.cm
39          * ml-yacc-lib.cm - needs no further comment                                          cross-bootstrap-compilers for Unix
40          * smlnj-lib.cm  - needs no further comment                                          (structure <Arch>UnixCMB)
41          * <arch>-compiler.cm - exports the Compiler structure for the    * $smlnj/cmb/ppc-macos.cm             ...for Mac (structure PPCMacosCMB)
42                             given architecture (<Arch>Compiler).    * $smlnj/cmb/x86-win32.cm             ...for Windoze (structure X86Win32CMB)
43          * <arch>-<os>.cm - exports <Arch><OS>CMB for the given architecture-    * $smlnj/cmb/current.cm               structure CMB (current arch/os)
44                             OS combination.  
45          * target-compilers.cm    * $smlnj/compiler.cm                  abbrev. for $smlnj/compiler/current.cm
46                          - library exporting target-specific versions of    * $smlnj/cm.cm                        abbrev. for $smlnj/cm/full.cm
47                            structure Compiler and of structure CMB.    * $smlnj/cmb.cm                       abbrev. for $smlnj/cmb/current.cm
48                            This is a summary of all the <arch>-compiler.cm  
49                            and <arch>-<os>.cm libraries above.    * $comp-lib.cm                        Utility library for compiler
50                            (The existence of these libraries is the moral  
51                             equivalent of "CMB.retarget" in the old CM.)    - $smlnj/viscomp/core.cm              Compiler core functionality
52          * viscomp-core.cm - library that implements the machine-independent    - $smlnj/viscomp/{alpha32,hppa,ppc,sparc,x86}.cm
53                            core of the compiler.                                          Machine-specific parts of compiler
54          * viscomp-<arch>.cm - library that implements the visible compiler  
55                             for a given architecture.    - $smlnj/internal/{intsys,cm-lib,host-compiler-0}.cm
56                                            Glue that holds the interactive system
57                                            together
58    
59      * $MLRISC/{MLRISC,Control,Lib,ALPHA,HPPA,PPC,SPARC,IA32}.cm
60                                            Various MLRISC bits
61                                            (Other MLRISC libraries such as
62                                             Graph, Visual, etc. do not currently
63                                             take part in the SML/NJ build.)
64    
65      * ${mlyacc,mllex,mlburg}-tool.cm      CM plug-in libraries for common tools
66      * ${grm,lex,burg}-ext.cm              CM plug-in libraries for common file
67                                            extensions
68    
69    Paths of the form $/foo/... are shorthands for $foo/foo/..., paths of
70    the form $/singlearc can also be written as $singlearc (e.g.,
71    $basis.cm instead of $/basis.cm).  The usefulness of the latter
72    shorthand is controversial, so you should probably try to avoid it.
73    
74    A more complete explanation of the $-notation can be found later in
75    this document or in the CM manual.
76    
77    To learn about the definitions of the $-anchors (and, thus, where in
78    the source tree the above libraries are defined), consult the
79    "pathconfig" file here in this directory.
80    
81  * Before you can use the bootstrap compiler (CMB)...  * Before you can use the bootstrap compiler (CMB)...
82  ----------------------------------------------------  ----------------------------------------------------
83    
84  To be able to use CMB at all, you must first say  To be able to use CMB at all, you must first say
85    
86          CM.autoload "host-cmb.cm";          CM.autoload "$smlnj/cmb.cm";
87    
88  after you start sml.  after you start sml.  Alternatively -- and perhaps more conveniently --
89    you can provide "$smlnj/cmb.cm" as a command-line argument to sml:
90    
91  * Compiling the compiler -- a two-step procedure        $ sml '$smlnj/cmb.cm'
 ------------------------------------------------  
92    
93  Until now (with the old CM), once we managed to run CMB.make() to  (Be sure to protect the dollar symbol which usually has its own
94  completion we had a directory full of binfiles that were ready to be  special meaning to the shell.)
 used by the boot procedure.  This is no longer the case.  
   
 The boot procedure now wants to use stable libraries (except for the  
 part that makes up the pervasive environment).  Having stable  
 libraries around during development of these very libraries would be a  
 bit annoying because if CM sees a stable library it will no longer  
 bother to check the corresponding source files -- even if they have  
 changed.  Therefore, libraries are not stabilized until you think you  
 are ready for that.  Thus, you should run:  
   
         CMB.make ();  
   
 until you no longer get compile errors.  CMB.make will return true in  
 this case.  Then you say:  
   
         CMB.deliver ();  
   
 This command creates a second directory parallel to the "bin"  
 directory -- the "boot" directory.  It will hold everything necessary  
 to bootstrap a new heap image.  You will probably find that  
 CMB.deliver() compiles a number of additional files even though  
 CMB.make completed successfully.  This is because CMB.make compiles  
 just those modules that will actually go into the heap image, but  
 CMB.deliver must also build the remaining files -- files that are part  
 of libraries to be stabilized but which are not used by the compiler.  
95    
96  You can reduce the number of extra files compiled and stabilized  * Compiling the compiler
97  during CMB.deliver at the expense of not building any cross-compilers.  ------------------------
 For that, say  
         #set (CMB.symval "LIGHT") (SOME 1);  
 before running CMB.deliver.  
98    
99  After you have made the boot directory, if you want to continue  We are now back to the old scheme where a call to CMB.make() suffices to
100  developing the compiler (i.e., make changes to some sources,  build a bootable set of files (libraries in our case).  CMB.make maintains
101  recompile, etc.), you must first get rid of that boot directory.  two parallel hierarchies of derived files:
 Running the "makeml" script (see below) will automatically remove the  
 boot directory.  
102    
103  The names of "bin" and "boot" directories are    1. the binfile hierarchy ("binfiles"), containing compiled objects for
104         each individual ML source file; this hierarchy is rooted at
105          <prefix>.bin.<arch>-<opsys>
106      2. the stable library hierarchy ("boot files"), containing library files
107         for each library that participates in building SML/NJ; this hierarchy
108         is rooted at
109          <prefix>.boot.<arch>-<opsys>
110    
111          <prefix>.bin.<arch>-<os>  The default for <prefix> is "sml".  It can be changed by using
112    CMB.make' with the new <prefix> as the optional string argument.
113    
114  and  CMB.make reuses existing bootfiles after it has verified that they are
115    consistent with their corresponding binfiles.  Bootfiles do not need
116    to be deleted in order for CMB.make to work correctly.
117    
118          <prefix>.boot.<arch>-<os>  To bootstrap a new system (using the runtime system boot loader), the
119    bootfiles _must_ be present, the binfiles need not be present (but
120    their presence does not hurt either).
121    
122  respectively, with "sml" being the default for <prefix>.  To change  You can reduce the number of extra files compiled and stabilized
123  the prefix, use CMB.make' and CMB.deliver' with the new prefix  during CMB.make at the expense of not building any cross-compilers.
124  provided as the optional string argument to these functions.  For that, say
125            #set (CMB.symval "LIGHT") (SOME 1);
126    before running CMB.make.
127    
128  * Making the heap image  * Making the heap image
129  -----------------------  -----------------------
# Line 129  Line 136 
136  The "feel" of using makeml should be mostly as it used to.  However,  The "feel" of using makeml should be mostly as it used to.  However,
137  internally, there are some changes that you should be aware of:  internally, there are some changes that you should be aware of:
138    
139  1. The script will make a heap image and also move its associated  1. The script will make a heap image and build a separate library directory
140     libraries into a separate directory.     that contains (hard) links to the library files in the bootfile directory.
141    
142  2. There is no "-full" option anymore.  This functionality should  2. There is no "-full" option anymore.  This functionality should
143     eventually be provided by a library with a sufficiently rich export     eventually be provided by a library with a sufficiently rich export
# Line 142  Line 149 
149     option to actually make the image.  The argument to "-rebuild"     option to actually make the image.  The argument to "-rebuild"
150     is the <prefix> for the new bin and boot directories (see above).     is the <prefix> for the new bin and boot directories (see above).
151    
152  4. Unless you use "-rebuild", makeml will delete the boot directory     [Note: When the -rebuild option is specified, then the boot procedure
153     (thus readying you for further "CMB.make();" runs).      will not read static environments from the boot directory.  Instead,
154        after the ML code has been loaded and linked, the system will invoke
155        CMB.make' with the argument that was given to -rebuild.  After
156        CMB.make' is done, the system quits.  In essence, makeml with -rebuild
157        acts as a bootstrap compiler that is not dependent on any usable
158        static environments.]
159    
160    Makeml will not destroy the bootfile directory.
161    
162  * Testing a newly generated heap image  * Testing a newly generated heap image
163  --------------------------------------  --------------------------------------
# Line 151  Line 165 
165  If you use a new heap image by saying "sml @SMLload=..." then things  If you use a new heap image by saying "sml @SMLload=..." then things
166  will not go as you may expect because along with the new heap image  will not go as you may expect because along with the new heap image
167  should go those new stable libraries, but unless you do something  should go those new stable libraries, but unless you do something
168  about it, the new CM will look for its stable libraries in places  about it, the newly booted system will look for its stable libraries
169  where you stored your _old_ stable libraries.  in places where you stored your _old_ stable libraries.  (After just
170    having done "makeml", these "places" would be within the boot file
171    hierarchy under <prefix>.boot.<arch>-<os>.)
172    
173  After you have made the new heap image, the new libraries are in a  After you have made the new heap image, the new libraries are in a
174  separate directory whose name is derived from the name of the heap  separate directory whose name is derived from the name of the heap
175  image.  The "testml" script that you also find here will run the heap  image.  (Actually, only the directory hierachy is separate, the
176  image and instruct it to look for its libraries in that new library  library files themselves are hard links.)  The "testml" script that
177  directory.  you also find here will run the heap image and instruct it to look for
178    its libraries in that new library directory by setting the
179    CM_PATHCONFIG environment variable to point to a different pathconfig
180    file under <prefix>.lib.
181    
182  "testml" takes the <prefix> of the heap image as its first  "testml" takes the <prefix> of the heap image as its first
183  argument. All other arguments are passed verbatim to the ML process.  argument. All other arguments are passed verbatim to the ML process.
184    
185  The <prefix> is the same as the one used when you did "makeml".  If  The <prefix> is the same as the one used when you did "makeml".  If
186  you run "testml" without arguments, <prefix> defaults to "sml".  you run "testml" without arguments, <prefix> defaults to "sml".
187  Thus, if you just said "makeml" without argument you can also say  Thus, if you just said "makeml" without arguments you can also say
188  "testml" without argument.  (Note that you _must_ supply the <prefix>  "testml" without arguments.  (Note that you _must_ supply the <prefix>
189  argument if you intend to pass any additional arguments.)  argument if you intend to pass any additional arguments.)
190    
191  * Installing a heap image for more permanent use  * Installing a heap image for more permanent use
192  ------------------------------------------------  ------------------------------------------------
193    
194  You can "install" a newly generated heap image by replacing the old  You can "install" a newly generated heap image by replacing the old
195  image with the new one _AND AT THE SAME TIME_ replacing the old stable  image with the new one AND AT THE SAME TIME replacing the old stable
196  libaries with the new ones.  To do this, run the "installml" script.  libaries with the new ones.  To do this, run the "installml" script.
197    
198  Like "testml", "installml" also expects the <prefix> as its first  Like "testml", "installml" also expects the <prefix> as its first
199  argument.  <prefix> defaults to "sml" if no argument is specified.  argument.  <prefix> defaults to "sml" if no argument is specified.
200    
201  "installml" patches the ../../lib/pathconfig file to reflect any  "installml" patches the ../../lib/pathconfig file to reflect any
202  changes or additions to the path name mapping.  changes or additions to the path name mapping.  (I say "patches"
203    because entries unrelated to the SML/NJ build process are retained in
204    their original form.)  If you want to use a destination directory that
205    is different from ../../lib, then you must do this by hand (i.e.,
206    installml does not have an option for that).
207    
208  Thus, after a successful CMB.deliver, you should say  Thus, after a successful CMB.make, you should say
209    
210          ./makeml          ./makeml
211    
# Line 199  Line 222 
222  * Cross-compiling  * Cross-compiling
223  -----------------  -----------------
224    
225  All cross-compilers live in the "target-compilers.cm" library.  You  All cross-compilers live in the "$smlnj/compiler/all.cm" library.
226  must first say  (The source tree for the "$smlnj" anchor -- see "pathconfig" -- is
227    src/system/smlnj, but this should normally not concern you.)
228    You must first say
229    
230          CM.autoload "target-compilers.cm";          CM.autoload "$smlnj/compiler/all.cm";
231    
232  before you can access them.  (This step corresponds to the old  before you can access them.  (This step corresponds to the old
233  CMB.retarget call.)  After that, _all_ cross-compilers are available  CMB.retarget call.)  After that, _all_ cross-compilers are available
234  at the same time.  However, the ones that you are not using don't take  at the same time.  However, the ones that you are not using don't take
235  up any undue space because they only get loaded once you actually  up any undue space because they only get loaded once you actually
236  mention them at the top-level.  The names of the structures currently  mention them at top level.  The names of the structures currently
237  exported by target-compilers.cm are:  exported by $smlnj/compiler/all.cm are:
238    
239          structure Alpha32UnixCMB          structure Alpha32UnixCMB
240          structure HppaUnixCMB          structure HppaUnixCMB
# Line 229  Line 254 
254  implementation of the basis library for the MacOS.)  implementation of the basis library for the MacOS.)
255    
256  Alternatively, you can select just the one single structure that you  Alternatively, you can select just the one single structure that you
257  are interested in by auto-loading <arch>-compiler.cm or <arch>-<os>.cm.  are interested in by auto-loading $smlnj/compiler/<arch>.cm or
258    $smlnj/cmb/<arch>-<os>.cm.
259  <arch> currently ranges over "alpha32", "hppa", "ppc", "sparc", and "x86.  <arch> currently ranges over "alpha32", "hppa", "ppc", "sparc", and "x86.
260  <os> can be either "unix" or "macos" or "win32".  <os> can be either "unix" or "macos" or "win32".
261  (Obviously, not all combinations are valid.)  (Obviously, not all combinations are valid.)
262    
263    Again, as with $smlnj/cmb.cm, you can specify the .cm file as an
264    argument to the sml command:
265    
266        $ sml '$smlnj/compiler/all.cm'
267    
268    or
269    
270        $ sml '$smlnj/cmb/alpha32-unix.cm'
271    
272    [Note: The command line for the "sml" command accepts configuration
273    parameters of the form "@SMLxxx...", mode switches of the form "-m"
274    and "-a", names of ML files -- which are passed to "use" -- and
275    arguments suitable for CM.make or CM.autoload.  CM.autoload is the
276    default; the "-m" and "-a" mode switches can be used to change the
277    default -- even several times within the same command line.
278    A single argument "@CMslave" is also accepted, but it should not be
279    used directly as it is intended for use by the parallel compilation
280    facility within CM.]
281    
282  * Path configuration  * Path configuration
283  --------------------  --------------------
284    
# Line 267  Line 312 
312  be changed, but one must do so explicitly.  In effect, it does not  be changed, but one must do so explicitly.  In effect, it does not
313  depend on the contents of the file system.  Here is how it works:  depend on the contents of the file system.  Here is how it works:
314    
315  If I specify a relative pathname in one of CM's description files  If I specify a pathname that starts with a "$", then the first arc
316  where the first component (the first arc) of that pathname is known to  between "$" and the first "/" is taken as the name of a so-called
317  CM as a configuration anchor, then the corresponding directory  "anchor".  CM knows a mapping from anchor names to directory names and
318  (according to CM's mapping) is prepended to the path.  Suppose the  replaces the prefix $<anchor> with the name of the corresponding
319  path name is "a/foo.sml" and "a" is a known anchor that maps to  directory.  Therefore, an anchored path has the general form
320  "/usr/lib/smlnj", then the resulting complete pathname is  
321  "/usr/lib/smlnj/a/foo.sml".  The pathname can be a single arc (but     $<anchor>/<path>
322  does not have to be).  For example, the anchor "basis.cm" is typically  
323  mapped to the directory where the basis library is stored.  It is important that there is at least one arc in <path>.  In other
324    words, the form $<anchor> is NOT valid.  (Actually, it currently is
325  Now, the important point is that one can change the mapping of the  valid, but CM interprets it in a different way.)
326  anchor, and the path name will also change accordingly -- even very  
327  late in the game.  CM avoids "elaborating" path names until it really  Examples:
328  needs them when it is time to open files.  CM is also willing to  
329  re-elaborate the same names if there is reason to do so. Thus, the     $smlnj/compiler/all.cm
330  "basis.cm" library that was analyzed "here" but then moved "there"     $basis.cm/basis.cm
331  will also be found "there" if the anchor has been re-set accordingly.     $MLRISC/Control.cm
332    
333    The special case where <anchor> coincides with the first arc of <path>
334    can be abbreviated by ommitting <anchor>.  This leads to the shorthand
335    
336      $/<anchor>/<more>...
337    
338    for the longer
339    
340      $<anchor>/<anchor>/<more>...
341    
342    Examples:
343    
344      $/foo/bar/baz.cm      (* same as $foo/foo/bar/baz.cm *)
345      $/basis.cm            (* same as $basis.cm/basis.cm *)
346    
347    Currently, CM accepts one additional shorthand for the case where
348    <path> has precisely one arc that coincides with <anchor>.  Here, the
349    slash "/" can be ommitted, too.
350    
351    Examples:
352    
353      $basis.cm             (* same as $/basis.cm or $basis.cm/basis.cm *)
354      $nw-ext.cm            (* same as $/nw-ext.cm or $nw-ext.cm/nw-ext.cm *)
355    
356    Previously, CM used "implicit" anchors where anchored paths simply
357    have the form
358    
359       <anchor>/<more>...
360    
361    The distinction between anchored paths and relative paths was made
362    based on whether or not <anchor> had a known mapping at the time it
363    was seen by CM.  Since this is hard to read and fragile, support for
364    implicit anchors (while still there) is considered obsolete and will
365    be faded out soon.  The meaning of an implicitly anchored path <path>
366    is the same as $/<path>.
367    
368    Recognition of implicit anchors can be turned off by issuing the
369    following command:
370    
371       CM.autoload "$smlnj/cm.cm";
372       #set CM.Control.implicit_anchors false;
373    
374    + Why anchored paths?
375    
376    The important point is that one can change the mapping of the anchor,
377    and the tranlation of the (anchored) path name will also change
378    accordingly -- even very late in the game.  CM avoids "elaborating"
379    path names until it really needs them when it is time to open files.
380    CM is also willing to re-elaborate the same names if there is reason
381    to do so. Thus, the "basis.cm" library that was analyzed "here" but
382    then moved "there" will also be found "there" if the anchor has been
383    re-set accordingly.
384    
385    The anchor mapping is (re-)initialized at startup time by reading two
386    configuration files.  Normally, those are the "../../lib/pathconfig"
387    file and the ".smlnj-pathconfig" file in your home directory (if such
388    exists).  During an ongoing session, function CM.Anchor.anchor can be
389    used to query and modify the anchor mapping.
390    
391  + Different configurations at different times:  + Different configurations at different times:
392    
393  During compilation of the compiler, CMB uses a path configuration that  During compilation of the compiler, CMB uses a path configuration that
394  is read from the file "pathconfig" located here in this directory.  is read from the file "pathconfig" located here in this directory.
395    
396  At bootstrap time, the same anchors are mapped to the corresponding  At bootstrap time (while running "makeml"), the same anchors are
397  sub-directory of the "boot" directory: basis.cm is mapped to  mapped to the corresponding sub-directory of the "boot" directory:
398  sml.boot.<arch>-<os>/basis.cm -- which means that CM will look for a  basis.cm is mapped to sml.boot.<arch>-<os>/basis.cm -- which means
399  library named sml.boot.<arch>-<os>/basis.cm/basis.cm -- and so forth.  that CM will look for a library named
400    sml.boot.<arch>-<os>/basis.cm/basis.cm -- and so forth.
401    
402    [Note, there are some anchors in "pathconfig" that have no
403    corresponding sub-directory of the boot director.  Examples are
404    "root.cm", "cm", and so on.  The reason is that there are no stable
405    libraries whose description files are named using these anchors;
406    everything anchored at "$cm" is a group but not a library.]
407    
408  By the way, you will perhaps notice that there is no file  By the way, you will perhaps notice that there is no file
409          sml.boot.<arch>-<os>/basis.cm/basis.cm          sml.boot.<arch>-<os>/basis.cm/basis.cm
# Line 304  Line 414 
414  This mapping (from anchors to names in the boot directory) is the one  This mapping (from anchors to names in the boot directory) is the one
415  that will get frozen into the generated heap image at boot time.  that will get frozen into the generated heap image at boot time.
416  Thus, unless it is changed, CM will look for its libraries in the boot  Thus, unless it is changed, CM will look for its libraries in the boot
417  directory.  The aforementioned "testml" script will make sure that  directory.  The aforementioned "testml" script will make sure (by
418  the mapping is changed to the one specified in a new "pathconfig" file  setting the environment variable CM_PATHCONFIG) that the mapping be
419  which was created by makeml and placed into the test library  changed to the one specified in a new "pathconfig" file which was
420  directory.  It points all anchors to the corresponding entry in the  created by makeml and placed into the test library directory.  It
421  test library directory.  Thus, "testml" will let a new heap image run  points all anchors to the corresponding entry in the test library
422  with its corresponding new libraries.  directory.  Thus, "testml" will let a new heap image run with its
423    corresponding new libraries.
424    
425  Normally, however, CM consults other pathconfig files at startup --  Normally, however, CM consults other pathconfig files at startup --
426  files that live in standard locations.  These files are used to modify  files that live in standard locations.  These files are used to modify
# Line 326  Line 437 
437  before making the heap image.  Therefore, heap images generated by  before making the heap image.  Therefore, heap images generated by
438  makeml will look for their global pathconfig file in  makeml will look for their global pathconfig file in
439    
440          `pwd`/../../lib/pathconfig          ../../lib/pathconfig
441    
442    [Note: The "makeml" script will not re-set the CM_PATHCONFIG_DEFAULT
443    variable if it was already set before.  If it does re-set the
444    variable, it uses an absolute path name instead of the relative path
445    that I used for illustration above.]
446    
447  For example, I always keep my "good" libraries in `pwd`/../../lib --  For example, I always keep my "good" libraries in `pwd`/../../lib --
448  where both the main "install" script and the "installml" script (see  where both the main "install" script (in config/install.sh) and the
449  above) also put them -- so I don't have to do anything special about  "installml" script (see above) also put them -- so I don't have to do
450  my pathconfig file.  anything special about my pathconfig file.
451    
452  Once I have new heap image and libraries working, I replace the old  Once I have new heap image and libraries working, I replace the old
453  "good" image with the new one:  "good" image with the new one:
454    
455    mv <image>.<arch>-<osvariant> ../../bin/.heap/sml.<arch>-<osvariant>    mv <image>.<arch>-<osvariant> ../../bin/.heap/sml.<arch>-<osvariant>
456    
457  and then:  After this I must also move all libraries from <image>.libs/* to their
458    corresponding position in ../../lib.
   rm -r ../../lib/*.cm  
   mv <image>.libs/*.cm ../../lib  
459    
460  For convenience, there is a script called "installml" that automates  Since this is cumbersome to do by hand, there is a script called
461  this task.  Using the script has the added advantage that it will not  "installml" that automates this task.  Using the script has the added
462  clobber libraries that belong to other than the current architecture.  advantage that it will not clobber libraries that belong to other than
463  (The rather heavy-handed "rm/mv" approach above will delete all stable  the current architecture.  (A rather heavy-handed "rm/mv" approach
464  libraries for all architectures.)  "installml" also patches the  will delete all stable libraries for all architectures.)
465  ../../lib/pathconfig file as necessary.  "installml" also patches the ../../lib/pathconfig file as necessary.
466    
467  Of course, you can organize things differently for yourself -- the  Of course, you can organize things differently for yourself -- the
468  path configuration mechanism should be sufficiently flexible.  path configuration mechanism should be sufficiently flexible.  If you
469    do so, you will have to set CM_PATHCONFIG.  This must be done before
470    you start sml.  If you want to change the pathname mapping at the time
471    sml is already running, then use the functions in CM.Anchor.
472    
473  * Libraries vs. Groups  * Libraries vs. Groups
474  ----------------------  ----------------------
# Line 371  Line 488 
488  other groups -- as long as all these other groups belong to the same  other groups -- as long as all these other groups belong to the same
489  owner library.  owner library.
490    
491  If you want to take a collection of files whose purpose fits that of a  Normally, collections of files that belong together should be made
492  library, then, please, make them into a library (i.e., not a group!).  into proper CM libraries.  CM groups (aka "library components") should
493  The purpose of groups is to deal with name-space issues _within_  be used only when there are namespace problems within a library.
 libraries.  
494    
495  Aside from the fact that I find this design quite natural, there is  Aside from the fact that I find this design quite natural, there is
496  actually a technical reason for it: when you stabilize a library  actually a technical reason for it: when you stabilize a library
# Line 384  Line 500 
500  library, n for n sub-groups), there will be just one file representing  library, n for n sub-groups), there will be just one file representing
501  the one stable archive (per architecture/os) for the whole thing.  For  the one stable archive (per architecture/os) for the whole thing.  For
502  example, I structured the standard basis into one library with two  example, I structured the standard basis into one library with two
503  sub-groups, but once you compile it (CMB.deliver) there is only one  sub-groups, but once you compile it (CMB.make) there is only one
504  stable file that represents the whole basis library.  If groups were  stable file that represents the whole basis library.  If groups were
505  allowed to appear in more than one library, then stabilization would  allowed to appear in more than one library, then stabilization would
506  duplicate the group (its code, its environment data structures, and  duplicate the group (its code, its environment data structures, and
# Line 396  Line 512 
512  keyword.  If the specification is missing (that's the "old" syntax),  keyword.  If the specification is missing (that's the "old" syntax),
513  then the the owner will be taken to be the interactive toplevel.  then the the owner will be taken to be the interactive toplevel.
514    
515  * Pervasive environment, core environment, the init group "init.cmi"  * Pervasive environment, core environment, the init library "init.cmi"
516  -------------------------------------------------------------------------  -------------------------------------------------------------------------
517    
518  CMB.make (or CMB.deliver) starts out by building and compiling the  CMB.make starts out by building and compiling the
519  "init group".  This group cannot be described in the "usual" way  "init library".  This library cannot be described in the "usual" way
520  because it uses "magic" in three ways:  because it uses "magic" in three ways:
521   - it is used to later tie in the runtime system   - it is used to later tie in the runtime system
522   - it builds the "core" environment   - it binds the "_Core" structure
523   - it builds the "pervasive" environment   - it exports the "pervasive" environment
524    
525  The pervasive environment no longer includes the entire basis library  The pervasive environment no longer includes the entire basis library
526  but only non-modular bindings (top-level bindings of variables and  but only non-modular bindings (top-level bindings of variables and
527  types).  types).
528    
529  CM cannot automatically determine dependencies for the init group  CM cannot automatically determine dependencies (or exports) for the
530  source files, but it still does use its regular cutoff recompilation  init library source files, but it still does use its regular cutoff
531  mechanism.  Therefore, dependencies must be given explicitly.  This is  recompilation mechanism.  Therefore, dependencies must be given
532  done by a special description file which currently lives in  explicitly.  This is done by a special description file which
533  Init/init.cmi.  See the long comment at the beginning of that file for  currently lives in smlnj/init/init.cmi (as an anchored path:
534  more details.  "$smlnj/init/init.cmi").  See the long comment at the beginning of
535    that file for more details.
536  After it is built, init.cmi can be used as an "ordinary" library by  
537  other libraries.  (This is done, for example, by the implementation of  After it is built, $smlnj/init/init.cmi can be used as an "ordinary"
538  the Basis library.)  Access to "init.cmi" is protected by the  library by other libraries.  (This is done, for example, by the
539  privilege named "primitive".  implementation of the Basis library.)  Access to
540    "$smlnj/init/init.cmi" is protected by the privilege named
541    "primitive".  Also, note that the .cmi-file is not automatically
542    recognized as as CM description file. ("cmi" should remind you of "CM
543    - Initial library".)  Therefore, it must be given an explicit member
544    class:
545    
546         $smlnj/init/init.cmi : cm
547    
548  * Autoloader  * Autoloader
549  ------------  ------------
550    
551  The new system heavily relies on the autoloader.  As a result, almost  The new system heavily relies on the autoloader.  As a result, almost
552  no static environments need to get unpickled at bootstap time.  The  no static environments need to get unpickled at bootstrap time.  The
553  construction of such environments is deferred until they become  construction of such environments is deferred until they become
554  necessary.  Because of this, I was able to reduce the size of the heap  necessary.  Thanks to this, it was possible to reduce the size of the
555  image by more than one megabyte (depending on the architecture).  The  heap image by more than one megabyte (depending on the architecture).
556  downside (although not really terribly bad) is that there is a short  The downside (although not really terribly bad) is that there is a
557  wait when you first touch an identifier that hasn't been touched  short wait when you first touch an identifier that hasn't been touched
558  before.  (I acknowledge that the notion of "short" may depend on your  before.  (I acknowledge that the notion of "short" may depend on your
559  sense of urgency. :-)  sense of urgency. :-)
560    
# Line 439  Line 562 
562  means that in order to be able to use the system, your paths must be  means that in order to be able to use the system, your paths must be
563  properly configured.  properly configured.
564    
565  Two libraries get pre-registered at bootstap time: the basis library  Several libraries get pre-registered at bootstap time.  Here, at least
566  ("basis.cm") and CM itself ("minimal-cm.cm").  The latter is crucial:  the following two should be included: the basis library ("$/basis.cm")
567  without it one wouldn't be able to register any other libraries  and CM itself ("$smlnj/cm.cm").  Currently, we also pre-register the
568  via CM.autoload.  The registration of basis.cm is a mere convenience.  library exporting structure Compiler ($smlnj/compiler.cm) and the
569    SML/NJ library ($/smlnj-lib.cm).
570    
571  Here are some other useful libraries that are not pre-registered but  Here are some other useful libraries that are not pre-registered but
572  which can easily be made accessible via CM.autoload (or, non-lazily,  which can easily be made accessible via CM.autoload (or, non-lazily,
573  via CM.make):  via CM.make):
574    
575          full-cm.cm              - provides the actual ("full") structure CM          $smlnj/cmb.cm           - provides "structure CMB"
576                                    as described in the CM manual          $smlnj/cmb/current.cm   - same as $smlnj/cmb.cm
577          host-compiler.cm        - provides "structure Compiler"          $smlnj/compiler/all.cm  - provides "structure <Arch>Compiler" and
         host-cmb.cm             - provides "structure CMB"  
         target-compilers.cm     - provides "structure <Arch>Compiler" and  
578                                    "structure <Arch><OS>CMB" for various                                    "structure <Arch><OS>CMB" for various
579                                    values of <Arch> and <OS>                                    values of <Arch> and <OS>
580          smlnj-lib.cm            - the SML/NJ library  
581    The file preloads.standard here in this directory currently includes
582    $smlnj/cmb.cm.  This means that by doing ./makeml one obtains a heap
583    image with the bootstrap compiler being pre-registered as well.  This
584    seems reasonable for compiler hackers.  (The config/install.sh script
585    uses config/preloads where $smlnj/cmb.cm is not pre-registered.  This
586    is appropriate as a setup for general users.)
587    
588    [****** Note:  The following is NO LONGER TRUE:
589    The fact that $smlnj/compiler.cm is not among the pre-registered
590    libraries seems like an oversight and could lead to some
591    inconveniences to users who want to, for example, set compiler flags.
592    However, pre-registration of this library significantly increases the
593    size of the heap image.  Moreover, since the library can easily be
594    loaded by giving the string as a command line argument, this does not
595    really appear to be a big burden to me.  Just create a shell alias or
596    a little wrapper script if you think you really need this.]
597    
598  * Internal sharing  * Internal sharing
599  ------------------  ------------------
# Line 497  Line 635 
635          defined(<namespace> <name>)          defined(<namespace> <name>)
636      is true if any of the included members preceeding this clause exports      is true if any of the included members preceeding this clause exports
637      a symbol "<namespace> <name>".      a symbol "<namespace> <name>".
638    - Within the "exports" section of the description (i.e., before "is):    - Within the "exports" section of the description (i.e., before "is"):
639      The same expression is true if _any_ of the members exports the      The same expression is true if _any_ of the members exports the
640      named symbol.      named symbol.
641      (It would be more logical if the exports section would follow the      (It would be more logical if the exports section would follow the
# Line 557  Line 695 
695  big loss because path anchors make up for most of it.  Moreover,  big loss because path anchors make up for most of it.  Moreover,
696  stable libraries can now easily be moved to convenient locations  stable libraries can now easily be moved to convenient locations
697  without having to move large source trees at the same time. (See my  without having to move large source trees at the same time. (See my
698  new build/install.sh script for examples of that.)  new config/install.sh script for examples of that.)
699    
700    It is possible to simulate aliases (in a way that is safer than the
701    original alias mechanism).  For example, the root.cm file (which is the
702    root of the whole system as far as CMB.make is concerned) acts as an
703    alias for $smlnj/internal/intsys.cm.  In this case, root.cm is a group
704    to avoid having a (trivial) stable library file built for it.
705    
706    A library can act as an "alias" for another library if it has a
707    verbatim copy of the export list and mentions the other library as its
708    only member.  Examples for this are $smlnj/cm.cm (for
709    $smlnj/cm/full.cm), $smlnj/compiler.cm (for $smlnj/compiler/current.cm),
710    etc.  The stable library file for such an "alias" is typically very
711    small because it basically just points to the other library.  (For
712    example, the file representing $smlnj/cm.cm is currently 234 bytes
713    long.)
714    
715  * Don't use relative or absolute pathnames to refer to libraries  * Don't use relative or absolute pathnames to refer to libraries
716  ----------------------------------------------------------------  ----------------------------------------------------------------
717    
718  Don't use relative or absolute pathnames to refer to libraries.  If  Don't use relative or absolute pathnames to refer to libraries.  If
719  you do it anyway, you'll get an appropriate warning at the time when  you do it anyway, you'll get an appropriate warning at the time when
720  you do CMB.deliver().  If you use relative or absolute pathnames to  you do CMB.make().  If you use relative or absolute pathnames to
721  refer to library B from library A, you will be committed to keeping B  refer to library B from library A, you will be committed to keeping B
722  in the same relative (to A) or absolute location.  This, clearly,  in the same relative (to A) or absolute location.  This, clearly,
723  would be undesirable.  would be undesirable in many situations (although perhaps not always).

Legend:
Removed from v.537  
changed lines
  Added in v.652

root@smlnj-gforge.cs.uchicago.edu
ViewVC Help
Powered by ViewVC 1.0.0