Home My Page Projects Code Snippets Project Openings diderot
Summary Activity Tracker Tasks SCM

SCM Repository

[diderot] Annotation of /branches/vis15/TODO
ViewVC logotype

Annotation of /branches/vis15/TODO

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1568 - (view) (download)
Original Path: trunk/TODO

1 : glk 1446 ***************************************************
2 :     ***************************************************
3 :     THIS TODO HAS BEEN MOVED TO THE DIDEROT WIKI:
4 : nseltzer 1445
5 : glk 1446 http://diderot-wiki.cs.uchicago.edu/index.php/Todo
6 :    
7 : jhr 1568 PLEASE USE THAT PAGE TO UPDATE PROBLEMS AND PROGRESS
8 : glk 1446 ***************************************************
9 :     ***************************************************
10 :    
11 : glk 1162 NOTE: GLK's approximate ranking of 8 most important tagged with
12 : jhr 1115 [GLK:1], [GLK:2], ...
13 :    
14 : glk 1156 ========================
15 :     SHORT TERM ============= (*needed* for streamlines & tractography)
16 :     ========================
17 : jhr 1115
18 : glk 1338 [GLK:2] Add sequence types (needed for evals & evecs)
19 : jhr 1115 syntax
20 :     types: ty '{' INT '}'
21 :     value construction: '{' e1 ',' … ',' en '}'
22 :     indexing: e '{' e '}'
23 :    
24 : glk 1338 [GLK:3] evals & evecs for symmetric tensor[2,2] and
25 : glk 1162 tensor[3,3] (requires sequences)
26 :    
27 : glk 1156 ability to emit/track/record variables into dynamically re-sized
28 : glk 1389 runtime output buffer
29 : jhr 1115
30 : glk 1442 [GLK:4] tensor fields from tensor images: Initially need at least
31 :     convolution on tensor[2,2] and tensor[3,3] (the same component-wise
32 :     convolution as for vectors).
33 : jhr 1115
34 : glk 1156 ========================
35 : glk 1162 SHORT-ISH TERM ========= (to make using Diderot less annoying to
36 :     ======================== program in, and slow to execute)
37 : jhr 1115
38 : glk 1167 Allow ".ddro" file extensions in addition to ".diderot"
39 :    
40 : glk 1204 Be able to output values of type tensor[2,2] and tensor[3,3];
41 :     (currently only scalars & vectors). Want to add some regression tests
42 :     based on this and currently can't
43 : glk 1167
44 : glk 1338 [GLK:1] Proper handling of stabilize method
45 : jhr 1115
46 : glk 1442 Convolution on general tensor images (order > 2)
47 :    
48 : glk 1162 allow "*" to represent "modulate": per-component multiplication of
49 :     vectors, and vectors only (not tensors of order 2 or higher). Once
50 :     sequences are implemented this should be removed: the operation is not
51 :     invariant WRT basis so it is not a legit vector computation.
52 :    
53 :     implicit type promotion of integers to reals where reals are
54 :     required (e.g. not exponentiation "^")
55 :    
56 : glk 1442 [Nick working on this] Save Diderot output to nrrd, instead of "mip.txt"
57 : jhr 1115 For grid of strands, save to similarly-shaped array
58 :     For list of strands, save to long 1-D (or 2-D for non-scalar output) list
59 :     For ragged things (like tractography output), will need to save both
60 :     complete list of values, as well as list of start indices and lengths
61 :     to index into complete list
62 :    
63 : glk 1442 [GLK:6] ability to declare a field so that probe positions are
64 : glk 1120 *always* "inside"; with various ways of mapping the known image values
65 :     to non-existant index locations. One possible syntax emphasizes that
66 :     there is a index mapping function that logically precedes convolution:
67 : glk 1162 F = bspln3 ⊛ (img ◦ clamp)
68 : glk 1120 F = bspln3 ⊛ (img ◦ repeat)
69 :     F = bspln3 ⊛ (img ◦ mirror)
70 :     where "◦" or "∘" is used to indicate function composition
71 : jhr 1115
72 : glk 1162 Level of differentiability in field type should be statement about how
73 :     much differentiation the program *needs*, rather than what the kernel
74 :     *provides*. The needed differentiability can be less than or equal to
75 :     the provided differentiability.
76 :    
77 : glk 1156 Use ∇⊗ etc. syntax
78 :     syntax [DONE]
79 :     typechecking
80 :     IL and codegen
81 : jhr 1115
82 : glk 1156 Add type aliases for color types
83 :     rgb = real{3}
84 :     rgba = real{4}
85 : jhr 1115
86 : glk 1389 Revisit how images are created within the language.
87 : jhr 1568 The "load" operator should probably go away, and its strange
88 : glk 1389 that strings are there only as a way to refer to nrrd filenames
89 :    
90 : jhr 1115 ==============================
91 : glk 1156 MEDIUM TERM ================== (*needed* for particles)
92 : jhr 1115 ==============================
93 :    
94 : glk 1442 [Lamont working on this] run-time birth of strands
95 : jhr 1115
96 :     "initially" supports lists
97 :    
98 : glk 1254 "initially" supports lists of positions output from different
99 :     initalization Diderot program (or output from the same program;
100 :     e.g. using output of iso2d.diderot for one isovalue to seed the input
101 :     to another invocation of the same program)
102 : jhr 1115
103 : glk 1442 [Lamont working on this] Communication between strands: they have to
104 :     be able to learn each other's state (at the previous iteration).
105 :     Early version of this can have the network of neighbors be completely
106 :     static (for running one strand/pixel image computations). Later
107 :     version with strands moving through the domain will require some
108 :     spatial data structure to optimize discovery of neighbors.
109 : jhr 1115
110 : glk 1156 ============================
111 :     MEDIUM-ISH TERM ============ (to make Diderot more useful/effective)
112 :     ============================
113 : jhr 1115
114 : glk 1442 [GLK:5] Want code-generation working for tensors of order three.
115 :     Order three matters for edge detection in scalar fields (to get
116 :     second derivatives of gradient magnitude), second derivatives
117 :     of vector fields (for some feature extraction), and first
118 :     derivatives of diffusion tensor fields.
119 :    
120 : glk 1156 Python/ctypes interface to run-time
121 : jhr 1115
122 : glk 1156 support for Python interop and GUI
123 : jhr 1115
124 : glk 1162 Allow integer exponentiation ("^2") to apply to square matrices,
125 :     to represent repeated matrix multiplication
126 :    
127 : glk 1156 Put small 1-D and 2-D fields, when reconstructed specifically by tent
128 :     and when differentiation is not needed, into faster texture buffers.
129 :     test/illust-vr.diderot is good example of program that uses multiple
130 :     such 1-D fields basically as lookup-table-based function evaluation
131 :    
132 :     extend norm (|exp|) to all tensor types [DONE for vectors and matrices]
133 :    
134 :     determinant ("det") for tensor[3,3]
135 :    
136 : jhr 1115 add ":" for tensor dot product (contracts out two indices
137 :     instead of one like •), valid for all pairs of tensors with
138 :     at least two indices
139 :    
140 : glk 1156 test/uninit.diderot:
141 :     documents need for better compiler error messages when output variables
142 :     are not initialized; the current messages are very cryptic
143 : jhr 1115
144 :     want: warnings when "D" (reserved for differentiation) is declared as
145 :     a variable name (get confusing error messages now)
146 :    
147 : glk 1156 ==============================
148 :     LONG TERM ==================== (make Diderot more interesting/attractive from
149 :     ============================== a research standpoint)
150 : jhr 1115
151 : glk 1156 IL support for higher-order tensor values (matrices, etc).
152 :     tensor construction [DONE]
153 :     tensor indexing [DONE]
154 :     tensor slicing
155 : jhr 1115
156 : glk 1156 Better handling of variables that determines the scope of a variable
157 :     based on its actual use, instead of where the user defined it. So,
158 :     for example, we should lift strand-invariant variables to global
159 :     scope. Also prune out useless variables, which should include field
160 :     variables after the translation to mid-il.
161 :    
162 :     test/vr-kcomp2.diderot: Add support for code like
163 :     (F1 if x else F2)@pos
164 :     This will require duplication of the continuation of the conditional
165 :     (but we should only duplicate over the live-range of the result of the
166 :     conditional.
167 :    
168 : glk 1338 [GLK:7] Want: non-trivial field expressions & functions.
169 : glk 1162 scalar fields from scalar fields F and G:
170 :     field#0(2)[] X = (sin(F) + 1.0)/2;
171 :     field#0(2)[] X = F*G;
172 :     scalar field of vector field magnitude:
173 : glk 1156 image(2)[2] Vimg = load(...);
174 :     field#0(2)[] Vlen = |Vimg ⊛ bspln3|;
175 : glk 1162 field of normalized vectors (for LIC and vector field feature extraction)
176 :     field#2(2)[2] F = ...
177 :     field#0(2)[2] V = normalize(F);
178 :     scalar field of gradient magnitude (for edge detection))
179 : glk 1156 field#2(2)[] F = Fimg ⊛ bspln3;
180 :     field#0(2)[] Gmag = |∇F|;
181 : glk 1162 scalar field of squared gradient magnitude (simpler to differentiate):
182 : glk 1156 field#2(2)[] F = Fimg ⊛ bspln3;
183 :     field#0(2)[] Gmsq = ∇F•∇F;
184 : glk 1162 There is value in having these, even if the differentiation of them is
185 :     not supported (hence the indication of "field#0" for these above)
186 : glk 1156
187 : jhr 1195 Introduce region types (syntax region(d), where d is the dimension of the
188 :     region. One useful operator would be
189 :     dom : field#k(d)[s] -> region(d)
190 :     Then the inside test could be written as
191 :     pos ∈ dom(F)
192 :     We could further extend this approach to allow geometric definitions of
193 :     regions. It might also be useful to do inside tests in world space,
194 :     instead of image space.
195 :    
196 : glk 1162 co- vs contra- index distinction
197 : glk 1156
198 : glk 1162 Permit field composition:
199 : glk 1156 field#2(3)[3] warp = bspln3 ⊛ warpData;
200 :     field#2(3)[] F = bspln3 ⊛ img;
201 :     field#2(3)[] Fwarp = F ◦ warp;
202 : glk 1162 So Fwarp(x) = F(warp(X)). Chain rule can be used for differentation.
203 :     This will be instrumental for expressing non-rigid registration
204 :     methods (but those will require co-vs-contra index distinction)
205 : glk 1156
206 : glk 1155 Allow the convolution to be specified either as a single 1D kernel
207 :     (as we have it now):
208 :     field#2(3)[] F = bspln3 ⊛ img;
209 :     or, as a tensor product of kernels, one for each axis, e.g.
210 :     field#0(3)[] F = (bspln3 ⊗ bspln3 ⊗ tent) ⊛ img;
211 : glk 1212 This is especially important for things like time-varying fields
212 :     and the use of scale-space in field visualization: one axis of the
213 :     must be convolved with a different kernel during probing.
214 :     What is very unclear is how, in such cases, we should notate the
215 :     gradient, when we only want to differentiate with respect to some
216 :     subset of the axes. One ambitious idea would be:
217 : glk 1162 field#0(3)[] Ft = (bspln3 ⊗ bspln3 ⊗ tent) ⊛ img; // 2D time-varying field
218 : glk 1204 field#0(2)[] F = lambda([x,y], Ft([x,y,42.0])) // restriction to time=42.0
219 :     vec2 grad = ∇F([x,y]); // 2D gradient
220 : glk 1155
221 : glk 1162 representation of tensor symmetry
222 : jhr 1115 (have to identify the group of index permutations that are symmetries)
223 :    
224 :     dot works on all tensors
225 :    
226 :     outer works on all tensors
227 :    
228 : glk 1204 Help for debugging Diderot programs: need to be able to uniquely
229 :     identify strands, and for particular strands that are known to behave
230 :     badly, do something like printf or other logging of their computations
231 :     and updates.
232 :    
233 :     Permit writing dimensionally general code: Have some statement of the
234 :     dimension of the world "W" (or have it be learned from one particular
235 :     field of interest), and then able to write "vec" instead of
236 :     "vec2/vec3", and perhaps "tensor[W,W]" instead of
237 :     "tensor[2,2]/tensor[3,3]"
238 :    
239 :     Traits: all things things that have boilerplate code (especially
240 :     volume rendering) should be expressed in terms of the unique
241 :     computational core. Different kinds of streamline/tractography
242 :     computation will be another example, as well as particle systems.
243 :    
244 : jhr 1115 Einstein summation notation
245 :    
246 :     "tensor comprehension" (like list comprehension)
247 :    
248 : glk 1204 Fields coming from different sources of data:
249 :     * triangular or tetrahedral meshes over 2D or 3D domains (of the
250 :     source produced by finite-element codes; these will come with their
251 :     own specialized kinds of reconstruction kernels, called "basis
252 :     functions" in this context)
253 :     * Large point clouds, with some radial basis function around each point,
254 :     which will be tuned by parameters of the point (at least one parameter
255 :     giving some notion of radius)
256 :    
257 : jhr 1115 ======================
258 :     BUGS =================
259 :     ======================
260 :    
261 :     test/zslice2.diderot:
262 :     // HEY (bug) bspln5 leads to problems ...
263 :     // uncaught exception Size [size]
264 :     // raised at c-target/c-target.sml:47.15-47.19
265 :     //field#4(3)[] F = img ⊛ bspln5;
266 : glk 1336

root@smlnj-gforge.cs.uchicago.edu
ViewVC Help
Powered by ViewVC 1.0.0