SCM Repository
Annotation of /sml/trunk/src/ml-lex/note
Parent Directory
| 
Revision Log
Revision 2 - (view) (download)
| 1 : | monnier | 2 | From: sufrin%prg.oxford.ac.uk@NSS.CS.UCL.AC.UK |
| 2 : | Message-Id: <8812101713.AA10966@uk.ac.oxford.prg.client13> | ||
| 3 : | Date: Sat Dec 10 17:13:18 1988 | ||
| 4 : | To: dbm <@RELAY.CS.NET:dbm@att.arpa> | ||
| 5 : | Subject: small suggestion for Andrew (re lexgen) | ||
| 6 : | Reply-To: Bernard Sufrin <sufrin%prg.oxford.ac.uk@NSS.CS.UCL.AC.UK> | ||
| 7 : | |||
| 8 : | |||
| 9 : | Dave: sorry to send stuff this way, | ||
| 10 : | but I can't find Andrew's email address. Bernard | ||
| 11 : | |||
| 12 : | Lexgen is very useful. A very simple impro- | ||
| 13 : | vement (which I've implemented here) is to | ||
| 14 : | give programmers the option of avoiding con- | ||
| 15 : | struction of a substring every time yytext | ||
| 16 : | is bound. Whilst "substring" is the simplest | ||
| 17 : | way to "internalise" a string, the program- | ||
| 18 : | mer may be able to do better by taking the | ||
| 19 : | "substring specification" (!yyb,i0,i-i0) and | ||
| 20 : | internalising it by some other means (which | ||
| 21 : | involves comparing the substring spec with | ||
| 22 : | already-internalised strings by any of the | ||
| 23 : | usual methods: I usually use open-hash + | ||
| 24 : | linear search). | ||
| 25 : | |||
| 26 : | |||
| 27 : | When employing this trick in a couple of | ||
| 28 : | compilers and our proof-support system I've | ||
| 29 : | found the consequent reduction of garbage | ||
| 30 : | collector cycles to be well worthwhile | ||
| 31 : | (instead of one substring construction per | ||
| 32 : | symbol of the text being analysed we get one | ||
| 33 : | substring constructed per DISTINCT symbol). | ||
| 34 : | |||
| 35 : | I've added an option in the lexgen defini- | ||
| 36 : | tions section: %nosubstring turns the binding | ||
| 37 : | of yytext from | ||
| 38 : | |||
| 39 : | val yytext = substring(!yyb,i0,i-i0) | ||
| 40 : | into | ||
| 41 : | val yytext = (!yyb,i0,i-i0:int) | ||
| 42 : | |||
| 43 : | |||
| 44 : | Here's the diff: | ||
| 45 : | |||
| 46 : | 92c94 | ||
| 47 : | < COUNT | REJECT | FULLCHARSET | STRUCT | ||
| 48 : | --- | ||
| 49 : | > COUNT | REJECT | FULLCHARSET | STRUCT | NOSUBSTRING | ||
| 50 : | 118c120 | ||
| 51 : | < val StrName = ref "Mlex" | ||
| 52 : | --- | ||
| 53 : | > val StrName = ref "Mlex"; | ||
| 54 : | 120,121c122 | ||
| 55 : | < val ResetFlags = fn () => (CountNewLines := false; HaveReject := false; | ||
| 56 : | < CharSetSize := 128; StrName := "Mlex") | ||
| 57 : | --- | ||
| 58 : | > (* Can obliterate substring operator *) | ||
| 59 : | 122a124,129 | ||
| 60 : | > val SubStrName = ref "substring"; | ||
| 61 : | > | ||
| 62 : | > val ResetFlags = fn () => | ||
| 63 : | > (CountNewLines := false; HaveReject := false; | ||
| 64 : | > CharSetSize := 128; StrName := "Mlex"; SubStrName := "substring" ) | ||
| 65 : | > | ||
| 66 : | 325a333,334 | ||
| 67 : | > else if command = "nosubstring" then | ||
| 68 : | > NOSUBSTRING | ||
| 69 : | 606a616,617 | ||
| 70 : | > | NOSUBSTRING => | ||
| 71 : | > (SubStrName := ""; ParseDefs()) | ||
| 72 : | 958c969 | ||
| 73 : | < sayln "\t\t\t(let val yytext = substring(!yyb,i0,i-i0)"; | ||
| 74 : | --- | ||
| 75 : | > say "\t\t\t(let val yytext = "; say(!SubStrName); sayln "(!yyb,i0,i-i0:int)"; | ||
| 76 : | |||
| 77 : |
| root@smlnj-gforge.cs.uchicago.edu | ViewVC Help |
| Powered by ViewVC 1.0.0 |
