# SCM Repository

# Annotation of /branches/fem/TODO

Parent Directory | Revision Log

Revision **1336** -
(**view**)
(**download**)

Original Path: *trunk/TODO*

1 : | glk | 1162 | NOTE: GLK's approximate ranking of 8 most important tagged with |

2 : | jhr | 1115 | [GLK:1], [GLK:2], ... |

3 : | |||

4 : | glk | 1156 | ======================== |

5 : | SHORT TERM ============= (*needed* for streamlines & tractography) | ||

6 : | ======================== | ||

7 : | jhr | 1115 | |

8 : | jhr | 1257 | Remove CL from compiler |

9 : | |||

10 : | glk | 1162 | [GLK:3] Add sequence types (needed for evals & evecs) |

11 : | jhr | 1115 | syntax |

12 : | types: ty '{' INT '}' | ||

13 : | value construction: '{' e1 ',' … ',' en '}' | ||

14 : | indexing: e '{' e '}' | ||

15 : | |||

16 : | glk | 1162 | [GLK:4] evals & evecs for symmetric tensor[2,2] and |

17 : | tensor[3,3] (requires sequences) | ||

18 : | |||

19 : | glk | 1156 | ability to emit/track/record variables into dynamically re-sized |

20 : | runtime buffer | ||

21 : | jhr | 1115 | |

22 : | glk | 1156 | tensor fields: convolution on general tensor images |

23 : | jhr | 1115 | |

24 : | glk | 1156 | ======================== |

25 : | glk | 1162 | SHORT-ISH TERM ========= (to make using Diderot less annoying to |

26 : | ======================== program in, and slow to execute) | ||

27 : | jhr | 1115 | |

28 : | jhr | 1246 | value-numbering optimization [DONE] |

29 : | jhr | 1115 | |

30 : | glk | 1167 | Allow ".ddro" file extensions in addition to ".diderot" |

31 : | |||

32 : | glk | 1204 | Be able to output values of type tensor[2,2] and tensor[3,3]; |

33 : | (currently only scalars & vectors). Want to add some regression tests | ||

34 : | based on this and currently can't | ||

35 : | glk | 1167 | |

36 : | glk | 1162 | [GLK:1] Add a clamp function, which takes three arguments; either |

37 : | three scalars: | ||

38 : | clamp(lo, hi, x) = max(lo, min(hi, x)) | ||

39 : | or three vectors of the same size: | ||

40 : | clamp(lo, hi, [x,y]) = [max(lo[0], min(hi[0], x)), | ||

41 : | max(lo[1], min(hi[1], y))] | ||

42 : | This would be useful in many current Diderot programs. | ||

43 : | One question: clamp(x, lo, hi) is the argument order used in OpenCL | ||

44 : | and other places, but clamp(lo, hi, x) is much more consistent with | ||

45 : | lerp(lo, hi, x), hence GLK's preference | ||

46 : | jhr | 1295 | [DONE] |

47 : | jhr | 1115 | |

48 : | glk | 1162 | [GLK:2] Proper handling of stabilize method |

49 : | |||

50 : | allow "*" to represent "modulate": per-component multiplication of | ||

51 : | vectors, and vectors only (not tensors of order 2 or higher). Once | ||

52 : | sequences are implemented this should be removed: the operation is not | ||

53 : | invariant WRT basis so it is not a legit vector computation. | ||

54 : | |||

55 : | implicit type promotion of integers to reals where reals are | ||

56 : | required (e.g. not exponentiation "^") | ||

57 : | |||

58 : | [GLK:5] Save Diderot output to nrrd, instead of "mip.txt" | ||

59 : | jhr | 1115 | For grid of strands, save to similarly-shaped array |

60 : | For list of strands, save to long 1-D (or 2-D for non-scalar output) list | ||

61 : | For ragged things (like tractography output), will need to save both | ||

62 : | complete list of values, as well as list of start indices and lengths | ||

63 : | to index into complete list | ||

64 : | |||

65 : | glk | 1162 | [GLK:6] Use of Teem's "hest" command-line parser for getting |

66 : | glk | 1212 | any "input" variables that are not defined in the source file. |

67 : | jhr | 1115 | |

68 : | glk | 1162 | [GLK:7] ability to declare a field so that probe positions are |

69 : | glk | 1120 | *always* "inside"; with various ways of mapping the known image values |

70 : | to non-existant index locations. One possible syntax emphasizes that | ||

71 : | there is a index mapping function that logically precedes convolution: | ||

72 : | glk | 1162 | F = bspln3 ⊛ (img ◦ clamp) |

73 : | glk | 1120 | F = bspln3 ⊛ (img ◦ repeat) |

74 : | F = bspln3 ⊛ (img ◦ mirror) | ||

75 : | where "◦" or "∘" is used to indicate function composition | ||

76 : | jhr | 1115 | |

77 : | glk | 1162 | Level of differentiability in field type should be statement about how |

78 : | much differentiation the program *needs*, rather than what the kernel | ||

79 : | *provides*. The needed differentiability can be less than or equal to | ||

80 : | the provided differentiability. | ||

81 : | |||

82 : | glk | 1156 | Use ∇⊗ etc. syntax |

83 : | syntax [DONE] | ||

84 : | typechecking | ||

85 : | IL and codegen | ||

86 : | jhr | 1115 | |

87 : | glk | 1156 | Add type aliases for color types |

88 : | rgb = real{3} | ||

89 : | rgba = real{4} | ||

90 : | jhr | 1115 | |

91 : | ============================== | ||

92 : | glk | 1156 | MEDIUM TERM ================== (*needed* for particles) |

93 : | jhr | 1115 | ============================== |

94 : | |||

95 : | run-time birth of strands | ||

96 : | |||

97 : | "initially" supports lists | ||

98 : | |||

99 : | glk | 1254 | "initially" supports lists of positions output from different |

100 : | initalization Diderot program (or output from the same program; | ||

101 : | e.g. using output of iso2d.diderot for one isovalue to seed the input | ||

102 : | to another invocation of the same program) | ||

103 : | jhr | 1115 | |

104 : | glk | 1156 | Communication between strands: they have to be able to learn each |

105 : | other's state (at the previous iteration). Early version of this can | ||

106 : | have the network of neighbors be completely static (for running one | ||

107 : | strand/pixel image computations). Later version with strands moving | ||

108 : | through the domain will require some spatial data structure to | ||

109 : | optimize discovery of neighbors. | ||

110 : | jhr | 1115 | |

111 : | glk | 1156 | ============================ |

112 : | MEDIUM-ISH TERM ============ (to make Diderot more useful/effective) | ||

113 : | ============================ | ||

114 : | jhr | 1115 | |

115 : | glk | 1156 | Python/ctypes interface to run-time |

116 : | jhr | 1115 | |

117 : | glk | 1156 | support for Python interop and GUI |

118 : | jhr | 1115 | |

119 : | glk | 1162 | Allow integer exponentiation ("^2") to apply to square matrices, |

120 : | to represent repeated matrix multiplication | ||

121 : | |||

122 : | glk | 1156 | Alow X *= Y, X /= Y, X += Y, X -= Y to mean what they do in C, |

123 : | provided that X*Y, X/Y, X+Y, X-Y are already supported. | ||

124 : | Nearly every Diderot program would be simplified by this. | ||

125 : | jhr | 1298 | [DONE] |

126 : | jhr | 1115 | |

127 : | glk | 1156 | Put small 1-D and 2-D fields, when reconstructed specifically by tent |

128 : | and when differentiation is not needed, into faster texture buffers. | ||

129 : | test/illust-vr.diderot is good example of program that uses multiple | ||

130 : | such 1-D fields basically as lookup-table-based function evaluation | ||

131 : | |||

132 : | jhr | 1301 | expand trace in mid to low translation [DONE] |

133 : | glk | 1156 | |

134 : | extend norm (|exp|) to all tensor types [DONE for vectors and matrices] | ||

135 : | |||

136 : | determinant ("det") for tensor[3,3] | ||

137 : | |||

138 : | jhr | 1115 | add ":" for tensor dot product (contracts out two indices |

139 : | instead of one like •), valid for all pairs of tensors with | ||

140 : | at least two indices | ||

141 : | |||

142 : | glk | 1156 | test/uninit.diderot: |

143 : | documents need for better compiler error messages when output variables | ||

144 : | are not initialized; the current messages are very cryptic | ||

145 : | jhr | 1115 | |

146 : | want: warnings when "D" (reserved for differentiation) is declared as | ||

147 : | a variable name (get confusing error messages now) | ||

148 : | |||

149 : | glk | 1156 | ============================== |

150 : | LONG TERM ==================== (make Diderot more interesting/attractive from | ||

151 : | ============================== a research standpoint) | ||

152 : | jhr | 1115 | |

153 : | glk | 1156 | IL support for higher-order tensor values (matrices, etc). |

154 : | tensor construction [DONE] | ||

155 : | tensor indexing [DONE] | ||

156 : | tensor slicing | ||

157 : | verify that hessians work correctly [DONE] | ||

158 : | jhr | 1115 | |

159 : | glk | 1156 | Better handling of variables that determines the scope of a variable |

160 : | based on its actual use, instead of where the user defined it. So, | ||

161 : | for example, we should lift strand-invariant variables to global | ||

162 : | scope. Also prune out useless variables, which should include field | ||

163 : | variables after the translation to mid-il. | ||

164 : | |||

165 : | test/vr-kcomp2.diderot: Add support for code like | ||

166 : | (F1 if x else F2)@pos | ||

167 : | This will require duplication of the continuation of the conditional | ||

168 : | (but we should only duplicate over the live-range of the result of the | ||

169 : | conditional. | ||

170 : | |||

171 : | glk | 1162 | [GLK:8] Want: non-trivial field expressions & functions. |

172 : | scalar fields from scalar fields F and G: | ||

173 : | field#0(2)[] X = (sin(F) + 1.0)/2; | ||

174 : | field#0(2)[] X = F*G; | ||

175 : | scalar field of vector field magnitude: | ||

176 : | glk | 1156 | image(2)[2] Vimg = load(...); |

177 : | field#0(2)[] Vlen = |Vimg ⊛ bspln3|; | ||

178 : | glk | 1162 | field of normalized vectors (for LIC and vector field feature extraction) |

179 : | field#2(2)[2] F = ... | ||

180 : | field#0(2)[2] V = normalize(F); | ||

181 : | scalar field of gradient magnitude (for edge detection)) | ||

182 : | glk | 1156 | field#2(2)[] F = Fimg ⊛ bspln3; |

183 : | field#0(2)[] Gmag = |∇F|; | ||

184 : | glk | 1162 | scalar field of squared gradient magnitude (simpler to differentiate): |

185 : | glk | 1156 | field#2(2)[] F = Fimg ⊛ bspln3; |

186 : | field#0(2)[] Gmsq = ∇F•∇F; | ||

187 : | glk | 1162 | There is value in having these, even if the differentiation of them is |

188 : | not supported (hence the indication of "field#0" for these above) | ||

189 : | glk | 1156 | |

190 : | jhr | 1195 | Introduce region types (syntax region(d), where d is the dimension of the |

191 : | region. One useful operator would be | ||

192 : | dom : field#k(d)[s] -> region(d) | ||

193 : | Then the inside test could be written as | ||

194 : | pos ∈ dom(F) | ||

195 : | We could further extend this approach to allow geometric definitions of | ||

196 : | regions. It might also be useful to do inside tests in world space, | ||

197 : | instead of image space. | ||

198 : | |||

199 : | glk | 1162 | co- vs contra- index distinction |

200 : | glk | 1156 | |

201 : | glk | 1162 | Permit field composition: |

202 : | glk | 1156 | field#2(3)[3] warp = bspln3 ⊛ warpData; |

203 : | field#2(3)[] F = bspln3 ⊛ img; | ||

204 : | field#2(3)[] Fwarp = F ◦ warp; | ||

205 : | glk | 1162 | So Fwarp(x) = F(warp(X)). Chain rule can be used for differentation. |

206 : | This will be instrumental for expressing non-rigid registration | ||

207 : | methods (but those will require co-vs-contra index distinction) | ||

208 : | glk | 1156 | |

209 : | glk | 1155 | Allow the convolution to be specified either as a single 1D kernel |

210 : | (as we have it now): | ||

211 : | field#2(3)[] F = bspln3 ⊛ img; | ||

212 : | or, as a tensor product of kernels, one for each axis, e.g. | ||

213 : | field#0(3)[] F = (bspln3 ⊗ bspln3 ⊗ tent) ⊛ img; | ||

214 : | glk | 1212 | This is especially important for things like time-varying fields |

215 : | and the use of scale-space in field visualization: one axis of the | ||

216 : | must be convolved with a different kernel during probing. | ||

217 : | What is very unclear is how, in such cases, we should notate the | ||

218 : | gradient, when we only want to differentiate with respect to some | ||

219 : | subset of the axes. One ambitious idea would be: | ||

220 : | glk | 1162 | field#0(3)[] Ft = (bspln3 ⊗ bspln3 ⊗ tent) ⊛ img; // 2D time-varying field |

221 : | glk | 1204 | field#0(2)[] F = lambda([x,y], Ft([x,y,42.0])) // restriction to time=42.0 |

222 : | vec2 grad = ∇F([x,y]); // 2D gradient | ||

223 : | glk | 1155 | |

224 : | glk | 1204 | Tensors of order 3 (e.g. gradients of diffusion tensor fields, or |

225 : | hessians of vector fields) and order 4 (e.g. Hessians of diffusion | ||

226 : | tensor fields). | ||

227 : | |||

228 : | glk | 1162 | representation of tensor symmetry |

229 : | jhr | 1115 | (have to identify the group of index permutations that are symmetries) |

230 : | |||

231 : | dot works on all tensors | ||

232 : | |||

233 : | outer works on all tensors | ||

234 : | |||

235 : | glk | 1204 | Help for debugging Diderot programs: need to be able to uniquely |

236 : | identify strands, and for particular strands that are known to behave | ||

237 : | badly, do something like printf or other logging of their computations | ||

238 : | and updates. | ||

239 : | |||

240 : | Permit writing dimensionally general code: Have some statement of the | ||

241 : | dimension of the world "W" (or have it be learned from one particular | ||

242 : | field of interest), and then able to write "vec" instead of | ||

243 : | "vec2/vec3", and perhaps "tensor[W,W]" instead of | ||

244 : | "tensor[2,2]/tensor[3,3]" | ||

245 : | |||

246 : | Traits: all things things that have boilerplate code (especially | ||

247 : | volume rendering) should be expressed in terms of the unique | ||

248 : | computational core. Different kinds of streamline/tractography | ||

249 : | computation will be another example, as well as particle systems. | ||

250 : | |||

251 : | jhr | 1115 | Einstein summation notation |

252 : | |||

253 : | "tensor comprehension" (like list comprehension) | ||

254 : | |||

255 : | glk | 1204 | Fields coming from different sources of data: |

256 : | * triangular or tetrahedral meshes over 2D or 3D domains (of the | ||

257 : | source produced by finite-element codes; these will come with their | ||

258 : | own specialized kinds of reconstruction kernels, called "basis | ||

259 : | functions" in this context) | ||

260 : | * Large point clouds, with some radial basis function around each point, | ||

261 : | which will be tuned by parameters of the point (at least one parameter | ||

262 : | giving some notion of radius) | ||

263 : | |||

264 : | jhr | 1115 | ====================== |

265 : | BUGS ================= | ||

266 : | ====================== | ||

267 : | |||

268 : | test/zslice2.diderot: | ||

269 : | // HEY (bug) bspln5 leads to problems ... | ||

270 : | // uncaught exception Size [size] | ||

271 : | // raised at c-target/c-target.sml:47.15-47.19 | ||

272 : | //field#4(3)[] F = img ⊛ bspln5; | ||

273 : | glk | 1336 | |

274 : | test/lic2.diderot: | ||

275 : | error in addEdge(ENTRY01D8,NULL0006) | ||

276 : | uncaught exception Fail [Fail: setPred on NULL node NULL0006] | ||

277 : | raised at common/phase-timer.sml:76.50-76.52 | ||

278 : | raised at IL/ssa-fn.sml:308.7-308.9 | ||

279 : | raised at IL/ssa-fn.sml:245.26-245.70 |

root@smlnj-gforge.cs.uchicago.edu | ViewVC Help |

Powered by ViewVC 1.0.0 |