Home My Page Projects Code Snippets Project Openings diderot
 Summary Activity Tracker Tasks SCM

# SCM Repository

[diderot] Annotation of /trunk/TODO
 [diderot] / trunk / TODO

# Annotation of /trunk/TODO

Revision 1568 - (view) (download)

 1 : glk 1446 *************************************************** 2 : *************************************************** 3 : THIS TODO HAS BEEN MOVED TO THE DIDEROT WIKI: 4 : nseltzer 1445 5 : glk 1446 http://diderot-wiki.cs.uchicago.edu/index.php/Todo 6 : 7 : jhr 1568 PLEASE USE THAT PAGE TO UPDATE PROBLEMS AND PROGRESS 8 : glk 1446 *************************************************** 9 : *************************************************** 10 : 11 : glk 1162 NOTE: GLK's approximate ranking of 8 most important tagged with 12 : jhr 1115 [GLK:1], [GLK:2], ... 13 : 14 : glk 1156 ======================== 15 : SHORT TERM ============= (*needed* for streamlines & tractography) 16 : ======================== 17 : jhr 1115 18 : glk 1338 [GLK:2] Add sequence types (needed for evals & evecs) 19 : jhr 1115 syntax 20 : types: ty '{' INT '}' 21 : value construction: '{' e1 ',' … ',' en '}' 22 : indexing: e '{' e '}' 23 : 24 : glk 1338 [GLK:3] evals & evecs for symmetric tensor[2,2] and 25 : glk 1162 tensor[3,3] (requires sequences) 26 : 27 : glk 1156 ability to emit/track/record variables into dynamically re-sized 28 : glk 1389 runtime output buffer 29 : jhr 1115 30 : glk 1442 [GLK:4] tensor fields from tensor images: Initially need at least 31 : convolution on tensor[2,2] and tensor[3,3] (the same component-wise 32 : convolution as for vectors). 33 : jhr 1115 34 : glk 1156 ======================== 35 : glk 1162 SHORT-ISH TERM ========= (to make using Diderot less annoying to 36 : ======================== program in, and slow to execute) 37 : jhr 1115 38 : glk 1167 Allow ".ddro" file extensions in addition to ".diderot" 39 : 40 : glk 1204 Be able to output values of type tensor[2,2] and tensor[3,3]; 41 : (currently only scalars & vectors). Want to add some regression tests 42 : based on this and currently can't 43 : glk 1167 44 : glk 1338 [GLK:1] Proper handling of stabilize method 45 : jhr 1115 46 : glk 1442 Convolution on general tensor images (order > 2) 47 : 48 : glk 1162 allow "*" to represent "modulate": per-component multiplication of 49 : vectors, and vectors only (not tensors of order 2 or higher). Once 50 : sequences are implemented this should be removed: the operation is not 51 : invariant WRT basis so it is not a legit vector computation. 52 : 53 : implicit type promotion of integers to reals where reals are 54 : required (e.g. not exponentiation "^") 55 : 56 : glk 1442 [Nick working on this] Save Diderot output to nrrd, instead of "mip.txt" 57 : jhr 1115 For grid of strands, save to similarly-shaped array 58 : For list of strands, save to long 1-D (or 2-D for non-scalar output) list 59 : For ragged things (like tractography output), will need to save both 60 : complete list of values, as well as list of start indices and lengths 61 : to index into complete list 62 : 63 : glk 1442 [GLK:6] ability to declare a field so that probe positions are 64 : glk 1120 *always* "inside"; with various ways of mapping the known image values 65 : to non-existant index locations. One possible syntax emphasizes that 66 : there is a index mapping function that logically precedes convolution: 67 : glk 1162 F = bspln3 ⊛ (img ◦ clamp) 68 : glk 1120 F = bspln3 ⊛ (img ◦ repeat) 69 : F = bspln3 ⊛ (img ◦ mirror) 70 : where "◦" or "∘" is used to indicate function composition 71 : jhr 1115 72 : glk 1162 Level of differentiability in field type should be statement about how 73 : much differentiation the program *needs*, rather than what the kernel 74 : *provides*. The needed differentiability can be less than or equal to 75 : the provided differentiability. 76 : 77 : glk 1156 Use ∇⊗ etc. syntax 78 : syntax [DONE] 79 : typechecking 80 : IL and codegen 81 : jhr 1115 82 : glk 1156 Add type aliases for color types 83 : rgb = real{3} 84 : rgba = real{4} 85 : jhr 1115 86 : glk 1389 Revisit how images are created within the language. 87 : jhr 1568 The "load" operator should probably go away, and its strange 88 : glk 1389 that strings are there only as a way to refer to nrrd filenames 89 : 90 : jhr 1115 ============================== 91 : glk 1156 MEDIUM TERM ================== (*needed* for particles) 92 : jhr 1115 ============================== 93 : 94 : glk 1442 [Lamont working on this] run-time birth of strands 95 : jhr 1115 96 : "initially" supports lists 97 : 98 : glk 1254 "initially" supports lists of positions output from different 99 : initalization Diderot program (or output from the same program; 100 : e.g. using output of iso2d.diderot for one isovalue to seed the input 101 : to another invocation of the same program) 102 : jhr 1115 103 : glk 1442 [Lamont working on this] Communication between strands: they have to 104 : be able to learn each other's state (at the previous iteration). 105 : Early version of this can have the network of neighbors be completely 106 : static (for running one strand/pixel image computations). Later 107 : version with strands moving through the domain will require some 108 : spatial data structure to optimize discovery of neighbors. 109 : jhr 1115 110 : glk 1156 ============================ 111 : MEDIUM-ISH TERM ============ (to make Diderot more useful/effective) 112 : ============================ 113 : jhr 1115 114 : glk 1442 [GLK:5] Want code-generation working for tensors of order three. 115 : Order three matters for edge detection in scalar fields (to get 116 : second derivatives of gradient magnitude), second derivatives 117 : of vector fields (for some feature extraction), and first 118 : derivatives of diffusion tensor fields. 119 : 120 : glk 1156 Python/ctypes interface to run-time 121 : jhr 1115 122 : glk 1156 support for Python interop and GUI 123 : jhr 1115 124 : glk 1162 Allow integer exponentiation ("^2") to apply to square matrices, 125 : to represent repeated matrix multiplication 126 : 127 : glk 1156 Put small 1-D and 2-D fields, when reconstructed specifically by tent 128 : and when differentiation is not needed, into faster texture buffers. 129 : test/illust-vr.diderot is good example of program that uses multiple 130 : such 1-D fields basically as lookup-table-based function evaluation 131 : 132 : extend norm (|exp|) to all tensor types [DONE for vectors and matrices] 133 : 134 : determinant ("det") for tensor[3,3] 135 : 136 : jhr 1115 add ":" for tensor dot product (contracts out two indices 137 : instead of one like •), valid for all pairs of tensors with 138 : at least two indices 139 : 140 : glk 1156 test/uninit.diderot: 141 : documents need for better compiler error messages when output variables 142 : are not initialized; the current messages are very cryptic 143 : jhr 1115 144 : want: warnings when "D" (reserved for differentiation) is declared as 145 : a variable name (get confusing error messages now) 146 : 147 : glk 1156 ============================== 148 : LONG TERM ==================== (make Diderot more interesting/attractive from 149 : ============================== a research standpoint) 150 : jhr 1115 151 : glk 1156 IL support for higher-order tensor values (matrices, etc). 152 : tensor construction [DONE] 153 : tensor indexing [DONE] 154 : tensor slicing 155 : jhr 1115 156 : glk 1156 Better handling of variables that determines the scope of a variable 157 : based on its actual use, instead of where the user defined it. So, 158 : for example, we should lift strand-invariant variables to global 159 : scope. Also prune out useless variables, which should include field 160 : variables after the translation to mid-il. 161 : 162 : test/vr-kcomp2.diderot: Add support for code like 163 : (F1 if x else F2)@pos 164 : This will require duplication of the continuation of the conditional 165 : (but we should only duplicate over the live-range of the result of the 166 : conditional. 167 : 168 : glk 1338 [GLK:7] Want: non-trivial field expressions & functions. 169 : glk 1162 scalar fields from scalar fields F and G: 170 : field#0(2)[] X = (sin(F) + 1.0)/2; 171 : field#0(2)[] X = F*G; 172 : scalar field of vector field magnitude: 173 : glk 1156 image(2)[2] Vimg = load(...); 174 : field#0(2)[] Vlen = |Vimg ⊛ bspln3|; 175 : glk 1162 field of normalized vectors (for LIC and vector field feature extraction) 176 : field#2(2)[2] F = ... 177 : field#0(2)[2] V = normalize(F); 178 : scalar field of gradient magnitude (for edge detection)) 179 : glk 1156 field#2(2)[] F = Fimg ⊛ bspln3; 180 : field#0(2)[] Gmag = |∇F|; 181 : glk 1162 scalar field of squared gradient magnitude (simpler to differentiate): 182 : glk 1156 field#2(2)[] F = Fimg ⊛ bspln3; 183 : field#0(2)[] Gmsq = ∇F•∇F; 184 : glk 1162 There is value in having these, even if the differentiation of them is 185 : not supported (hence the indication of "field#0" for these above) 186 : glk 1156 187 : jhr 1195 Introduce region types (syntax region(d), where d is the dimension of the 188 : region. One useful operator would be 189 : dom : field#k(d)[s] -> region(d) 190 : Then the inside test could be written as 191 : pos ∈ dom(F) 192 : We could further extend this approach to allow geometric definitions of 193 : regions. It might also be useful to do inside tests in world space, 194 : instead of image space. 195 : 196 : glk 1162 co- vs contra- index distinction 197 : glk 1156 198 : glk 1162 Permit field composition: 199 : glk 1156 field#2(3)[3] warp = bspln3 ⊛ warpData; 200 : field#2(3)[] F = bspln3 ⊛ img; 201 : field#2(3)[] Fwarp = F ◦ warp; 202 : glk 1162 So Fwarp(x) = F(warp(X)). Chain rule can be used for differentation. 203 : This will be instrumental for expressing non-rigid registration 204 : methods (but those will require co-vs-contra index distinction) 205 : glk 1156 206 : glk 1155 Allow the convolution to be specified either as a single 1D kernel 207 : (as we have it now): 208 : field#2(3)[] F = bspln3 ⊛ img; 209 : or, as a tensor product of kernels, one for each axis, e.g. 210 : field#0(3)[] F = (bspln3 ⊗ bspln3 ⊗ tent) ⊛ img; 211 : glk 1212 This is especially important for things like time-varying fields 212 : and the use of scale-space in field visualization: one axis of the 213 : must be convolved with a different kernel during probing. 214 : What is very unclear is how, in such cases, we should notate the 215 : gradient, when we only want to differentiate with respect to some 216 : subset of the axes. One ambitious idea would be: 217 : glk 1162 field#0(3)[] Ft = (bspln3 ⊗ bspln3 ⊗ tent) ⊛ img; // 2D time-varying field 218 : glk 1204 field#0(2)[] F = lambda([x,y], Ft([x,y,42.0])) // restriction to time=42.0 219 : vec2 grad = ∇F([x,y]); // 2D gradient 220 : glk 1155 221 : glk 1162 representation of tensor symmetry 222 : jhr 1115 (have to identify the group of index permutations that are symmetries) 223 : 224 : dot works on all tensors 225 : 226 : outer works on all tensors 227 : 228 : glk 1204 Help for debugging Diderot programs: need to be able to uniquely 229 : identify strands, and for particular strands that are known to behave 230 : badly, do something like printf or other logging of their computations 231 : and updates. 232 : 233 : Permit writing dimensionally general code: Have some statement of the 234 : dimension of the world "W" (or have it be learned from one particular 235 : field of interest), and then able to write "vec" instead of 236 : "vec2/vec3", and perhaps "tensor[W,W]" instead of 237 : "tensor[2,2]/tensor[3,3]" 238 : 239 : Traits: all things things that have boilerplate code (especially 240 : volume rendering) should be expressed in terms of the unique 241 : computational core. Different kinds of streamline/tractography 242 : computation will be another example, as well as particle systems. 243 : 244 : jhr 1115 Einstein summation notation 245 : 246 : "tensor comprehension" (like list comprehension) 247 : 248 : glk 1204 Fields coming from different sources of data: 249 : * triangular or tetrahedral meshes over 2D or 3D domains (of the 250 : source produced by finite-element codes; these will come with their 251 : own specialized kinds of reconstruction kernels, called "basis 252 : functions" in this context) 253 : * Large point clouds, with some radial basis function around each point, 254 : which will be tuned by parameters of the point (at least one parameter 255 : giving some notion of radius) 256 : 257 : jhr 1115 ====================== 258 : BUGS ================= 259 : ====================== 260 : 261 : test/zslice2.diderot: 262 : // HEY (bug) bspln5 leads to problems ... 263 : // uncaught exception Size [size] 264 : // raised at c-target/c-target.sml:47.15-47.19 265 : //field#4(3)[] F = img ⊛ bspln5; 266 : glk 1336

 root@smlnj-gforge.cs.uchicago.edu ViewVC Help Powered by ViewVC 1.0.0