Home My Page Projects Code Snippets Project Openings SML/NJ
Summary Activity Forums Tracker Lists Tasks Docs Surveys News SCM Files

SCM Repository

[smlnj] Diff of /sml/trunk/src/system/README
ViewVC logotype

Diff of /sml/trunk/src/system/README

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 416, Fri Sep 3 23:50:31 1999 UTC revision 537, Fri Feb 18 17:20:16 2000 UTC
# Line 1  Line 1 
1  !!! ATTENTION !!!  Compiler Hacker's Guide to the new CM...
2  As an SML/NJ compiler developer, please read this document carefully.  ========================================
3  The new CM has a lot of good things to offer, but you must be aware  
4  of the many changes that it incurs to the process of compiling the  Last change: 2/2000
 SML/NJ compiler.  
                         Matthias Blume (July 1999)  
 -------------------------------------------------------------------------  
5    
6  * Libraries  * Libraries
7  -----------  -----------
# Line 26  Line 23 
23          - cm-hook.cm    - an internal library for organizational purposes          - cm-hook.cm    - an internal library for organizational purposes
24          - cm-lib.cm     - the library that implements CM's functionality          - cm-lib.cm     - the library that implements CM's functionality
25          * comp-lib.cm   - a helper library for the compiler, MLRISC, and CM          * comp-lib.cm   - a helper library for the compiler, MLRISC, and CM
26          * host-cm.cm    - the library that exports the public interface to          * full-cm.cm    - the library that exports the public interface to
27                            the compilation manager (i.e., structure CM)                            the compilation manager (i.e., structure CM)
28            * minimal-cm.cm - exports a reduced version of structure CM;
29                              this one is pre-registered at the top level
30          * host-cmb.cm   - the library that exports the public interface to          * host-cmb.cm   - the library that exports the public interface to
31                            the bootstrap compiler (i.e., structure CMB)                            the bootstrap compiler (i.e., structure CMB)
32          - host-compiler-0.cm          - host-compiler-0.cm
# Line 39  Line 38 
38                            (In fact, its the "root" of the main hierarchy.)                            (In fact, its the "root" of the main hierarchy.)
39          * ml-yacc-lib.cm - needs no further comment          * ml-yacc-lib.cm - needs no further comment
40          * smlnj-lib.cm  - needs no further comment          * smlnj-lib.cm  - needs no further comment
41            * <arch>-compiler.cm - exports the Compiler structure for the
42                               given architecture (<Arch>Compiler).
43            * <arch>-<os>.cm - exports <Arch><OS>CMB for the given architecture-
44                               OS combination.
45          * target-compilers.cm          * target-compilers.cm
46                          - library exporting target-specific versions of                          - library exporting target-specific versions of
47                            structure Compiler and of structure CMB                            structure Compiler and of structure CMB.
48                            (The existence of this library is the moral                            This is a summary of all the <arch>-compiler.cm
49                              and <arch>-<os>.cm libraries above.
50                              (The existence of these libraries is the moral
51                             equivalent of "CMB.retarget" in the old CM.)                             equivalent of "CMB.retarget" in the old CM.)
52          * viscomp-lib.cm - library that implements the compiler          * viscomp-core.cm - library that implements the machine-independent
53                            (At the moment, its interface is rather thin.  We                            core of the compiler.
54                             should think about how to structure the interface          * viscomp-<arch>.cm - library that implements the visible compiler
55                             in such a way that it becomes a useful equivalent                             for a given architecture.
                            to the old "full" compiler.)  
56    
57  * Before you can use the bootstrap compiler (CMB)...  * Before you can use the bootstrap compiler (CMB)...
58  ----------------------------------------------------  ----------------------------------------------------
# Line 90  Line 94 
94  CMB.deliver must also build the remaining files -- files that are part  CMB.deliver must also build the remaining files -- files that are part
95  of libraries to be stabilized but which are not used by the compiler.  of libraries to be stabilized but which are not used by the compiler.
96    
97    You can reduce the number of extra files compiled and stabilized
98    during CMB.deliver at the expense of not building any cross-compilers.
99    For that, say
100            #set (CMB.symval "LIGHT") (SOME 1);
101    before running CMB.deliver.
102    
103  After you have made the boot directory, if you want to continue  After you have made the boot directory, if you want to continue
104  developing the compiler (i.e., make changes to some sources,  developing the compiler (i.e., make changes to some sources,
105  recompile, etc.), you must first get rid of that boot directory.  recompile, etc.), you must first get rid of that boot directory.
# Line 104  Line 114 
114    
115          <prefix>.boot.<arch>-<os>          <prefix>.boot.<arch>-<os>
116    
117  respectively, with "comp" being the default for <prefix>.  To change  respectively, with "sml" being the default for <prefix>.  To change
118  the prefix, use CMB.make' and CMB.deliver' with the new prefix  the prefix, use CMB.make' and CMB.deliver' with the new prefix
119  provided as the optional string argument to these functions.  provided as the optional string argument to these functions.
120    
# Line 113  Line 123 
123    
124  The heap image is made by running the "makeml" script that you find  The heap image is made by running the "makeml" script that you find
125  here in this directory.  By default it will try to refer to the  here in this directory.  By default it will try to refer to the
126  comp.boot.<arch>-<os> directory.  You can change this using the -boot  sml.boot.<arch>-<os> directory.  You can change this using the -boot
127  argument (which takes the full name of the boot directory to be used).  argument (which takes the full name of the boot directory to be used).
128    
129  The "feel" of using makeml should be mostly as it used to.  However,  The "feel" of using makeml should be mostly as it used to.  However,
# Line 146  Line 156 
156    
157  After you have made the new heap image, the new libraries are in a  After you have made the new heap image, the new libraries are in a
158  separate directory whose name is derived from the name of the heap  separate directory whose name is derived from the name of the heap
159  image.  The "testrun" script that you also find here will run the heap  image.  The "testml" script that you also find here will run the heap
160  image and instruct it to look for its libraries in that new library  image and instruct it to look for its libraries in that new library
161  directory.  directory.
162  "testrun" takes the name of the heap image as its single argument.  It  
163  expects the library directory to be the one that makeml builds.  "testml" takes the <prefix> of the heap image as its first
164    argument. All other arguments are passed verbatim to the ML process.
165    
166    The <prefix> is the same as the one used when you did "makeml".  If
167    you run "testml" without arguments, <prefix> defaults to "sml".
168    Thus, if you just said "makeml" without argument you can also say
169    "testml" without argument.  (Note that you _must_ supply the <prefix>
170    argument if you intend to pass any additional arguments.)
171    
172  * Installing a heap image for more permanent use  * Installing a heap image for more permanent use
173  ------------------------------------------------  ------------------------------------------------
174    
175  Since you have been using the new CM already, it can be assumed that  You can "install" a newly generated heap image by replacing the old
176  you have already set up a correct pathname configuration.  (For more  image with the new one _AND AT THE SAME TIME_ replacing the old stable
177  information on pathname configurations, see below.)  With a correct  libaries with the new ones.  To do this, run the "installml" script.
178  pathname configuration in place, you can "install" a newly generated  
179  heap image by replacing the old image with the new one _AND AT THE  Like "testml", "installml" also expects the <prefix> as its first
180  SAME TIME_ replacing the old stable libaries with the new ones.  argument.  <prefix> defaults to "sml" if no argument is specified.
181    
182    "installml" patches the ../../lib/pathconfig file to reflect any
183    changes or additions to the path name mapping.
184    
185    Thus, after a successful CMB.deliver, you should say
186    
187            ./makeml
188    
189    to make the new heap image + libraries, then
190    
191            ./testml
192    
193    to make sure everything works, and finally
194    
195            ./installml
196    
197    to replace your existing compiler with the one you just built and tested.
198    
199  * Cross-compiling  * Cross-compiling
200  -----------------  -----------------
# Line 194  Line 228 
228  (PPCMacOSCMB is not very useful at the moment because there is no  (PPCMacOSCMB is not very useful at the moment because there is no
229  implementation of the basis library for the MacOS.)  implementation of the basis library for the MacOS.)
230    
231    Alternatively, you can select just the one single structure that you
232    are interested in by auto-loading <arch>-compiler.cm or <arch>-<os>.cm.
233    <arch> currently ranges over "alpha32", "hppa", "ppc", "sparc", and "x86.
234    <os> can be either "unix" or "macos" or "win32".
235    (Obviously, not all combinations are valid.)
236    
237  * Path configuration  * Path configuration
238  --------------------  --------------------
239    
# Line 218  Line 258 
258  in the path can upset the program that hopes to find something under  in the path can upset the program that hopes to find something under
259  the same name later on the path.  Even when ignoring security-issues  the same name later on the path.  Even when ignoring security-issues
260  like trojan horses and such, this definitely opens the door for  like trojan horses and such, this definitely opens the door for
261  various unpleasant surprises.  (Who has not ever named a test version  various unpleasant surprises.  (Who has never named a test version
262  of a program "test" an found that it acts strangely only to discover  of a program "test" an found that it acts strangely only to discover
263  later that /bin/test was run instead?)  later that /bin/test was run instead?)
264    
# Line 249  Line 289 
289    
290  During compilation of the compiler, CMB uses a path configuration that  During compilation of the compiler, CMB uses a path configuration that
291  is read from the file "pathconfig" located here in this directory.  is read from the file "pathconfig" located here in this directory.
 Warning: The names in that pathconfig file are relative pathnames and  
 will work only if you are in this directory.  (This will typically be  
 the case since you are compiling the compiler. Normally, however, path  
 configurations should map anchors to absolute pathnames.)  
292    
293  At bootstrap time, the same anchors are mapped to the corresponding  At bootstrap time, the same anchors are mapped to the corresponding
294  sub-directory of the "boot" directory: basis.cm is mapped to  sub-directory of the "boot" directory: basis.cm is mapped to
295  comp.boot.<arch>-<os>/basis.cm -- which means that CM will look for a  sml.boot.<arch>-<os>/basis.cm -- which means that CM will look for a
296  library named comp.boot.<arch>-<os>/basis.cm/basis.cm -- and so forth.  library named sml.boot.<arch>-<os>/basis.cm/basis.cm -- and so forth.
297    
298  By the way, you will perhaps notice that there is no file  By the way, you will perhaps notice that there is no file
299          comp.boot.<arch>-<os>/basis.cm/basis.cm          sml.boot.<arch>-<os>/basis.cm/basis.cm
300  but there _is_ the corresponding stable archive  but there _is_ the corresponding stable archive
301          comp.boot.<arch>-<os>/basis.cm/CM/<arch>-<os>/basis.cm          sml.boot.<arch>-<os>/basis.cm/CM/<arch>-<os>/basis.cm
302  CM always looks for stable archives first.  CM always looks for stable archives first.
303    
304  This mapping (from anchors to names in the boot directory) is the one  This mapping (from anchors to names in the boot directory) is the one
305  that will get frozen into the generated heap image at boot time.  that will get frozen into the generated heap image at boot time.
306  Thus, unless it is changed, CM will look for its libraries in the boot  Thus, unless it is changed, CM will look for its libraries in the boot
307  directory.  The aforementioned "testrun" script will make sure that  directory.  The aforementioned "testml" script will make sure that
308  the mapping is changed to the one specified in a new "pathconfig" file  the mapping is changed to the one specified in a new "pathconfig" file
309  which was created by makeml and placed into the test library  which was created by makeml and placed into the test library
310  directory.  It points all anchors to the corresponding entry in the  directory.  It points all anchors to the corresponding entry in the
311  test library directory.  Thus, "testrun" will let a new heap image run  test library directory.  Thus, "testml" will let a new heap image run
312  with its corresponding new libraries.  with its corresponding new libraries.
313    
314  Normally, however, CM consults other pathconfig files at startup --  Normally, however, CM consults other pathconfig files at startup --
# Line 290  Line 326 
326  before making the heap image.  Therefore, heap images generated by  before making the heap image.  Therefore, heap images generated by
327  makeml will look for their global pathconfig file in  makeml will look for their global pathconfig file in
328    
329          `pwd`/../../stable-libs/pathconfig          `pwd`/../../lib/pathconfig
330    
331  For example, I always keep my "good" libraries in  For example, I always keep my "good" libraries in `pwd`/../../lib --
332  `pwd`/../../stable-libs and let the pathconfig file point its anchors  where both the main "install" script and the "installml" script (see
333  to the members of that directory.  above) also put them -- so I don't have to do anything special about
334    my pathconfig file.
335    
336  Once I have new heap image and libraries working, I replace the old  Once I have new heap image and libraries working, I replace the old
337  "good" image with the new one:  "good" image with the new one:
# Line 303  Line 340 
340    
341  and then:  and then:
342    
343    rm -r ../../stable-libs/*.cm    rm -r ../../lib/*.cm
344    mv <image>.libs/*.cm ../../stable-libs    mv <image>.libs/*.cm ../../lib
345    
346    For convenience, there is a script called "installml" that automates
347    this task.  Using the script has the added advantage that it will not
348    clobber libraries that belong to other than the current architecture.
349    (The rather heavy-handed "rm/mv" approach above will delete all stable
350    libraries for all architectures.)  "installml" also patches the
351    ../../lib/pathconfig file as necessary.
352    
353  Of course, you can organize things differently for yourself -- the  Of course, you can organize things differently for yourself -- the
354  path configuration mechanism should be sufficiently flexible.  path configuration mechanism should be sufficiently flexible.
# Line 352  Line 396 
396  keyword.  If the specification is missing (that's the "old" syntax),  keyword.  If the specification is missing (that's the "old" syntax),
397  then the the owner will be taken to be the interactive toplevel.  then the the owner will be taken to be the interactive toplevel.
398    
399  There are several examples of this throughout the system's source  * Pervasive environment, core environment, the init group "init.cmi"
 hierarchy.  One notable case is MLRISC.  It should probably be made  
 into a library of its own, but I leave this job to Lal.  At the moment  
 MLRISC.cm is a sub-group of viscomp-lib.cm.  
   
 * Pervasive environment, core environment, other "primitive" environments  
400  -------------------------------------------------------------------------  -------------------------------------------------------------------------
401    
402  Just a handful of files is compiled at the beginning in order to  CMB.make (or CMB.deliver) starts out by building and compiling the
403  establish a number of "primitive" environments -- including the  "init group".  This group cannot be described in the "usual" way
404  "pervasive" environment and the "core" environment.  The pervasive  because it uses "magic" in three ways:
405  environment no longer includes the entire basis library but only   - it is used to later tie in the runtime system
406  non-modular bindings (top-level bindings of variables and types).   - it builds the "core" environment
407     - it builds the "pervasive" environment
408    
409    The pervasive environment no longer includes the entire basis library
410    but only non-modular bindings (top-level bindings of variables and
411    types).
412    
413  CM cannot automatically determine dependencies for these initial  CM cannot automatically determine dependencies for the init group
414  source files, but it still does use its regular cutoff recompilation  source files, but it still does use its regular cutoff recompilation
415  mechanism.  Therefore, dependencies must be given explicitly.  This is  mechanism.  Therefore, dependencies must be given explicitly.  This is
416  done by a special description file which currently lives in  done by a special description file which currently lives in
417  Init/init.cmi.  See the long comment at the beginning of that file for  Init/init.cmi.  See the long comment at the beginning of that file for
418  more details.  more details.
419    
420    After it is built, init.cmi can be used as an "ordinary" library by
421    other libraries.  (This is done, for example, by the implementation of
422    the Basis library.)  Access to "init.cmi" is protected by the
423    privilege named "primitive".
424    
425  * Autoloader  * Autoloader
426  ------------  ------------
427    
# Line 391  Line 440 
440  properly configured.  properly configured.
441    
442  Two libraries get pre-registered at bootstap time: the basis library  Two libraries get pre-registered at bootstap time: the basis library
443  ("basis.cm") and CM itself ("host-cm.cm").  The latter is crucial:  ("basis.cm") and CM itself ("minimal-cm.cm").  The latter is crucial:
444  without it one wouldn't be able to register any other libraries  without it one wouldn't be able to register any other libraries
445  via CM.autoload.  The registration of basis.cm is a mere convenience.  via CM.autoload.  The registration of basis.cm is a mere convenience.
446    
# Line 399  Line 448 
448  which can easily be made accessible via CM.autoload (or, non-lazily,  which can easily be made accessible via CM.autoload (or, non-lazily,
449  via CM.make):  via CM.make):
450    
451            full-cm.cm              - provides the actual ("full") structure CM
452                                      as described in the CM manual
453          host-compiler.cm        - provides "structure Compiler"          host-compiler.cm        - provides "structure Compiler"
454          host-cmb.cm             - provides "structure CMB"          host-cmb.cm             - provides "structure CMB"
455          target-compilers.cm     - provides "structure <Arch>Compiler" and          target-compilers.cm     - provides "structure <Arch>Compiler" and

Legend:
Removed from v.416  
changed lines
  Added in v.537

root@smlnj-gforge.cs.uchicago.edu
ViewVC Help
Powered by ViewVC 1.0.0