1 
======================== 
NOTE: GLK's approximate ranking of 5 most important tagged with 
2 
SHORT TERM ============= (for curvaturebased VR) 
[GLK:1], [GLK:2], ... 

======================== 

3 


4 
vector fields: convolution on vector images 
============================== 
5 
(image orientation not quite working) 
other SHORT TERM ============= (including needed for LIC) 
6 

============================== 
7 


8 

Add a clamp function, which takes three arguments; either three scalars: 
9 

clamp(x, minval, maxval) = max(minval, min(maxval, x)) 
10 

or three vectors of the same size: 
11 

clamp([x,y], minvec, maxvec) = [max(minvec[0], min(maxvec[0], x)), 
12 

max(minvec[1], min(maxvec[1], y))] 
13 

This would be useful in many current Diderot programs. 
14 

One question: clamp(x, minval, maxval) is the argument order 
15 

used in OpenCL and other places, but clamp(minval, maxval, x) 
16 

would be more consistent with lerp(minout, maxout, x). 
17 


18 

Level of differentiability in field type should be statement about how 
19 

much differentiation the program *needs*, rather than what the kernel 
20 

*provides*. The needed differentiability can be less than or equal to 
21 

the provided differentiability. 
22 


23 

[GLK:1] Add sequence types (needed for evals & evecs) 
24 

syntax 
25 

types: ty '{' INT '}' 
26 

value construction: '{' e1 ',' … ',' en '}' 
27 

indexing: e '{' e '}' 
28 


29 
IL support for higherorder tensor values (matrices, etc). 
IL support for higherorder tensor values (matrices, etc). 
30 
tensor construction [DONE] 
tensor construction [DONE] 
32 
tensor slicing 
tensor slicing 
33 
verify that hessians work correctly [DONE] 
verify that hessians work correctly [DONE] 
34 



Outer products [DONE] 





Add M dot v, v dot M, and M dot N [DONE] 





Identity matrix [DONE] 





Zero tensor [DONE] 





trace [DONE] 





extend norm (exp) to tensor[3,3] [DONE] 





Matrix addition, subtraction, and scaling [DONE] 




35 
Use ∇⊗ etc. syntax 
Use ∇⊗ etc. syntax 
36 
syntax [DONE] 
syntax [DONE] 
37 
typechecking 
typechecking 
38 
IL and codegen 
IL and codegen 
39 


40 
Add sequence types 
test/uninit.diderot: 
41 
syntax 
documents need for better compiler error messages when output variables 
42 
types: ty '{' INT '}' 
are not initialized; the current messages are very cryptic 

value construction: '{' e1 ',' … ',' en '}' 


indexing: e '{' e '}' 





Infix dot product and cross product [DONE] 





lerp on scalars and vectors [DONE] 





Infix "^" operator for pow() [DONE] 





Code generation support for 1D image data, such as RGBA transfer functions 





============================== 


other SHORT TERM ============= (including needed for LIC) 


============================== 

43 


44 
determinant ("det") for tensor[3,3] 
determinant ("det") for tensor[3,3] 
45 



extend norm (exp) to all tensor types 




46 
expand trace in mid to low translation 
expand trace in mid to low translation 
47 


48 
valuenumbering optimization 
valuenumbering optimization 
55 
MEDIUM TERM ================== (including needed for streamlines & tractography) 
MEDIUM TERM ================== (including needed for streamlines & tractography) 
56 
============================== 
============================== 
57 


58 
ability to emit/track/record variables into dynamically resized 
[GLK:1] evals & evecs for symmetric tensor[3,3] (requires sequences) 

runtime buffer 





evals & evecs for symmetric tensor[3,3] 





tensor fields: convolution on general tensor images 





Use of Teem's "hest" commandline parser for getting 


any input variables that are not defined in the source file 

59 


60 
Save Diderot output to nrrd, instead of "mip.txt" 
[GLK:2] Save Diderot output to nrrd, instead of "mip.txt" 
61 
For grid of strands, save to similarlyshaped array 
For grid of strands, save to similarlyshaped array 
62 
For list of strands, save to long 1D (or 2D for nonscalar output) list 
For list of strands, save to long 1D (or 2D for nonscalar output) list 
63 
For ragged things (like tractography output), will need to save both 
For ragged things (like tractography output), will need to save both 
64 
complete list of values, as well as list of start indices and lengths 
complete list of values, as well as list of start indices and lengths 
65 
to index into complete list 
to index into complete list 
66 


67 

[GLK:3] Use of Teem's "hest" commandline parser for getting 
68 

any input variables that are not defined in the source file 
69 


70 

[GLK:4] ability to declare a field so that probe positions are 
71 

*always* "inside"; with various ways of mapping the known image values 
72 

to nonexistant index locations. One possible syntax emphasizes that 
73 

there is a index mapping function that logically precedes convolution: 
74 

F = bspln3 ⊛ (img ◦ clamp) 
75 

F = bspln3 ⊛ (img ◦ repeat) 
76 

F = bspln3 ⊛ (img ◦ mirror) 
77 

where "◦" or "∘" is used to indicate function composition 
78 


79 

extend norm (exp) to all tensor types [DONE for vectors and matrices] 
80 


81 

ability to emit/track/record variables into dynamically resized 
82 

runtime buffer 
83 


84 

Want: allow X *= Y, X /= Y, X += Y, X = Y to mean what they do in C, 
85 

provided that X*Y, X/Y, X+Y, XY are already supported. 
86 

Nearly every Diderot program would be simplified by this. 
87 


88 

Want: nontrivial field expressions & functions: 
89 

image(2)[2] Vimg = load(...); 
90 

field#0(2)[] Vlen = Vimg ⊛ bspln3; 
91 

to get a scalar field of vector length, or 
92 

field#2(2)[] F = Fimg ⊛ bspln3; 
93 

field#0(2)[] Gmag = ∇F; 
94 

to get a scalar field of gradient magnitude, or 
95 

field#2(2)[] F = Fimg ⊛ bspln3; 
96 

field#0(2)[] Gmsq = ∇F•∇F; 
97 

to get a scalar field of squared gradient magnitude, which is simpler 
98 

to differentiate. However, there is value in having these, even if 
99 

the differentiation of them is not supported (hence the indication 
100 

of "field#0" for these above) 
101 


102 

Want: ability to apply "normalize" to a field itself, e.g. 
103 

field#0(2)[2] V = normalize(Vimg ⊛ ctmr); 
104 

so that V(x) = normalize((Vimg ⊛ ctmr)(x)). 
105 

Having this would simplify expression of standard LIC method, and 
106 

would also help express other vector field expressions that arise 
107 

in vector field feature exraction. 
108 


109 

tensor fields: convolution on general tensor images 
110 


111 
============================== 
============================== 
112 
other MEDIUM TERM ============ (needed for particles) 
other MEDIUM TERM ============ (needed for particles) 
113 
============================== 
============================== 
114 


115 
runtime birth and death of strands 
Put small 1D and 2D fields, when reconstructed specifically by tent 
116 

and when differentiation is not needed, into faster texture buffers. 
117 

test/illustvr.diderot is good example of program that uses multiple 
118 

such 1D fields basically as lookuptablebased function evaluation 
119 


120 

runtime birth of strands 
121 


122 
"initially" supports lists 
"initially" supports lists 
123 


132 


133 
(F1 if x else F2)@pos 
(F1 if x else F2)@pos 
134 


135 
This will require duplication of the continuation of the conditional (but we should only 
This will require duplication of the continuation of the conditional 
136 
duplicate over the liverange of the result of the conditional. 
(but we should only duplicate over the liverange of the result of the 
137 

conditional. 
138 


139 

add ":" for tensor dot product (contracts out two indices 
140 

instead of one like •), valid for all pairs of tensors with 
141 

at least two indices 
142 


143 
============================== 
============================== 
144 
other MEDIUM TERM ============ 
other MEDIUM TERM ============ 
145 
============================== 
============================== 
146 


147 
want: warnings when D (and likely I) is declared as a 
want: warnings when "D" (reserved for differentiation) is declared as 
148 
variable name (now get confusing error messages now) 
a variable name (get confusing error messages now) 
149 


150 
support for Python interop and GUI 
support for Python interop and GUI 
151 


152 

Python/ctypes interface to runtime 
153 


154 
============================== 
============================== 
155 
LONG TERM ==================== 
LONG TERM ==================== 
156 
============================== 
============================== 
157 


158 
Better handling of variables that determines the scope of a variable based on its actual use, 
Better handling of variables that determines the scope of a variable 
159 
instead of where the user defined it. So, for example, we should lift strandinvariant variables 
based on its actual use, instead of where the user defined it. So, 
160 
to global scope. Also prune out useless variables, which should include field variables after the 
for example, we should lift strandinvariant variables to global 
161 
translation to midil. 
scope. Also prune out useless variables, which should include field 
162 

variables after the translation to midil. 
163 


164 
co vs contra index distinction 
co vs contra index distinction 
165 



add ":" for tensor dot product (contracts out two indices 


instead of one like •), valid for all pairs of tensors with 


at least two indices 




166 
some indication of tensor symmetry 
some indication of tensor symmetry 
167 
(have to identify the group of index permutations that are symmetries) 
(have to identify the group of index permutations that are symmetries) 
168 


174 


175 
"tensor comprehension" (like list comprehension) 
"tensor comprehension" (like list comprehension) 
176 



Python/ctypes interface to runtimez 




177 
====================== 
====================== 
178 
BUGS ================= 
BUGS ================= 
179 
====================== 
====================== 
180 


181 
test/zslicek1k2.diderot 
test/read2vecs.diderot: 
182 
If we allow "∇F@pos" we should allow "∇(∇F)@pos", 
// HEY (BUG?) shouldn't it be a type error to load this 2D array of 
183 
but this currently causes: 
// 2vectors into a 2D *scalar* field? Instead, get: 
184 
uncaught exception Fail [Fail: Error in compiling zslicek1k2.diderot] 
// uncaught exception Fail [Fail: Error in compiling lic.diderot] 
185 
raised at driver/main.sml:31.3931.76 
// raised at driver/main.sml:31.3931.76 
186 

image(2)[] Vimg = load("../data/vorttest.nrrd"); 

test/zslicek1k2.diderot (and other programs) 


As programs are getting more complicated, the annoyance and confusion 


caused by allowing variables to be redeclared is getting higher! 


GLK sees no valid purpose for redeclaring a variable, even if the type is consistent. 


In zslicek1k2.diderot, if the green image declaration is: 


field#0(2)[] G = imgG ⊛ tent; 


and the geometry tensor is later declared 


tensor[3,3] G = (P•H•P)/g; 


that redeclaration causes no problems, but then usage of G the field 


G@kk 


causes: 


[zslicek1k2.diderot:52.2832] Error: type error for binary operator "@" 


expected: (field#'diff0101('dim0100)'shp00FF * tensor['dim0100]) 


but found: (tensor[3,3] * vec2) 





test/mipcam.diderot: 


// generated C code looks like "(float)p_ui_00.5e0f" 


//real rayU = 2.0*camUmax*(real(ui)  0.5)/real(imgResU)  camUmax; 





test/vrkcomp.diderot: 


// HEY (scoping BUG): the strand parameters (e.g. ui, vi) ... 





test/zslice2.diderot: 


// but is this not valid syntax for creating field in one shot? 


//field#2(3)[] F = (load("../data/zimg112.nrrd")) ⊛ bspln3; 


// It is valid syntax, but there is a bug in the conversion from HighIL to MidIL 

187 


188 
test/zslice2.diderot: 
test/zslice2.diderot: 
189 
// HEY (bug) bspln5 leads to problems ... 
// HEY (bug) bspln5 leads to problems ... 
190 
// uncaught exception Size [size] 
// uncaught exception Size [size] 
191 
// raised at ctarget/ctarget.sml:47.1547.19 
// raised at ctarget/ctarget.sml:47.1547.19 
192 
//field#4(3)[] F = img ⊛ bspln5; 
//field#4(3)[] F = img ⊛ bspln5; 





