# SCM Repository

# Annotation of /trunk/TODO

Parent Directory | Revision Log

Revision **1212** -
(**view**)
(**download**)

1 : | glk | 1162 | NOTE: GLK's approximate ranking of 8 most important tagged with |

2 : | jhr | 1115 | [GLK:1], [GLK:2], ... |

3 : | |||

4 : | glk | 1156 | ======================== |

5 : | SHORT TERM ============= (*needed* for streamlines & tractography) | ||

6 : | ======================== | ||

7 : | jhr | 1115 | |

8 : | glk | 1162 | [GLK:3] Add sequence types (needed for evals & evecs) |

9 : | jhr | 1115 | syntax |

10 : | types: ty '{' INT '}' | ||

11 : | value construction: '{' e1 ',' … ',' en '}' | ||

12 : | indexing: e '{' e '}' | ||

13 : | |||

14 : | glk | 1162 | [GLK:4] evals & evecs for symmetric tensor[2,2] and |

15 : | tensor[3,3] (requires sequences) | ||

16 : | |||

17 : | glk | 1156 | ability to emit/track/record variables into dynamically re-sized |

18 : | runtime buffer | ||

19 : | jhr | 1115 | |

20 : | glk | 1156 | tensor fields: convolution on general tensor images |

21 : | jhr | 1115 | |

22 : | glk | 1156 | ======================== |

23 : | glk | 1162 | SHORT-ISH TERM ========= (to make using Diderot less annoying to |

24 : | ======================== program in, and slow to execute) | ||

25 : | jhr | 1115 | |

26 : | jhr | 1165 | value-numbering optimization [DONE, but needs more testing] |

27 : | jhr | 1115 | |

28 : | glk | 1167 | Allow ".ddro" file extensions in addition to ".diderot" |

29 : | |||

30 : | glk | 1204 | Be able to output values of type tensor[2,2] and tensor[3,3]; |

31 : | (currently only scalars & vectors). Want to add some regression tests | ||

32 : | based on this and currently can't | ||

33 : | glk | 1167 | |

34 : | glk | 1162 | [GLK:1] Add a clamp function, which takes three arguments; either |

35 : | three scalars: | ||

36 : | clamp(lo, hi, x) = max(lo, min(hi, x)) | ||

37 : | or three vectors of the same size: | ||

38 : | clamp(lo, hi, [x,y]) = [max(lo[0], min(hi[0], x)), | ||

39 : | max(lo[1], min(hi[1], y))] | ||

40 : | This would be useful in many current Diderot programs. | ||

41 : | One question: clamp(x, lo, hi) is the argument order used in OpenCL | ||

42 : | and other places, but clamp(lo, hi, x) is much more consistent with | ||

43 : | lerp(lo, hi, x), hence GLK's preference | ||

44 : | jhr | 1115 | |

45 : | glk | 1162 | [GLK:2] Proper handling of stabilize method |

46 : | |||

47 : | allow "*" to represent "modulate": per-component multiplication of | ||

48 : | vectors, and vectors only (not tensors of order 2 or higher). Once | ||

49 : | sequences are implemented this should be removed: the operation is not | ||

50 : | invariant WRT basis so it is not a legit vector computation. | ||

51 : | |||

52 : | implicit type promotion of integers to reals where reals are | ||

53 : | required (e.g. not exponentiation "^") | ||

54 : | |||

55 : | [GLK:5] Save Diderot output to nrrd, instead of "mip.txt" | ||

56 : | jhr | 1115 | For grid of strands, save to similarly-shaped array |

57 : | For list of strands, save to long 1-D (or 2-D for non-scalar output) list | ||

58 : | For ragged things (like tractography output), will need to save both | ||

59 : | complete list of values, as well as list of start indices and lengths | ||

60 : | to index into complete list | ||

61 : | |||

62 : | glk | 1162 | [GLK:6] Use of Teem's "hest" command-line parser for getting |

63 : | glk | 1212 | any "input" variables that are not defined in the source file. |

64 : | jhr | 1115 | |

65 : | glk | 1162 | [GLK:7] ability to declare a field so that probe positions are |

66 : | glk | 1120 | *always* "inside"; with various ways of mapping the known image values |

67 : | to non-existant index locations. One possible syntax emphasizes that | ||

68 : | there is a index mapping function that logically precedes convolution: | ||

69 : | glk | 1162 | F = bspln3 ⊛ (img ◦ clamp) |

70 : | glk | 1120 | F = bspln3 ⊛ (img ◦ repeat) |

71 : | F = bspln3 ⊛ (img ◦ mirror) | ||

72 : | where "◦" or "∘" is used to indicate function composition | ||

73 : | jhr | 1115 | |

74 : | glk | 1162 | Level of differentiability in field type should be statement about how |

75 : | much differentiation the program *needs*, rather than what the kernel | ||

76 : | *provides*. The needed differentiability can be less than or equal to | ||

77 : | the provided differentiability. | ||

78 : | |||

79 : | glk | 1156 | Use ∇⊗ etc. syntax |

80 : | syntax [DONE] | ||

81 : | typechecking | ||

82 : | IL and codegen | ||

83 : | jhr | 1115 | |

84 : | glk | 1156 | Add type aliases for color types |

85 : | rgb = real{3} | ||

86 : | rgba = real{4} | ||

87 : | jhr | 1115 | |

88 : | ============================== | ||

89 : | glk | 1156 | MEDIUM TERM ================== (*needed* for particles) |

90 : | jhr | 1115 | ============================== |

91 : | |||

92 : | run-time birth of strands | ||

93 : | |||

94 : | "initially" supports lists | ||

95 : | |||

96 : | "initially" supports lists of positions output from | ||

97 : | different initalization Diderot program | ||

98 : | |||

99 : | glk | 1156 | Communication between strands: they have to be able to learn each |

100 : | other's state (at the previous iteration). Early version of this can | ||

101 : | have the network of neighbors be completely static (for running one | ||

102 : | strand/pixel image computations). Later version with strands moving | ||

103 : | through the domain will require some spatial data structure to | ||

104 : | optimize discovery of neighbors. | ||

105 : | jhr | 1115 | |

106 : | glk | 1156 | ============================ |

107 : | MEDIUM-ISH TERM ============ (to make Diderot more useful/effective) | ||

108 : | ============================ | ||

109 : | jhr | 1115 | |

110 : | glk | 1156 | Python/ctypes interface to run-time |

111 : | jhr | 1115 | |

112 : | glk | 1156 | support for Python interop and GUI |

113 : | jhr | 1115 | |

114 : | glk | 1162 | Allow integer exponentiation ("^2") to apply to square matrices, |

115 : | to represent repeated matrix multiplication | ||

116 : | |||

117 : | glk | 1156 | Alow X *= Y, X /= Y, X += Y, X -= Y to mean what they do in C, |

118 : | provided that X*Y, X/Y, X+Y, X-Y are already supported. | ||

119 : | Nearly every Diderot program would be simplified by this. | ||

120 : | jhr | 1115 | |

121 : | glk | 1156 | Put small 1-D and 2-D fields, when reconstructed specifically by tent |

122 : | and when differentiation is not needed, into faster texture buffers. | ||

123 : | test/illust-vr.diderot is good example of program that uses multiple | ||

124 : | such 1-D fields basically as lookup-table-based function evaluation | ||

125 : | |||

126 : | expand trace in mid to low translation | ||

127 : | |||

128 : | extend norm (|exp|) to all tensor types [DONE for vectors and matrices] | ||

129 : | |||

130 : | determinant ("det") for tensor[3,3] | ||

131 : | |||

132 : | jhr | 1115 | add ":" for tensor dot product (contracts out two indices |

133 : | instead of one like •), valid for all pairs of tensors with | ||

134 : | at least two indices | ||

135 : | |||

136 : | glk | 1156 | test/uninit.diderot: |

137 : | documents need for better compiler error messages when output variables | ||

138 : | are not initialized; the current messages are very cryptic | ||

139 : | jhr | 1115 | |

140 : | want: warnings when "D" (reserved for differentiation) is declared as | ||

141 : | a variable name (get confusing error messages now) | ||

142 : | |||

143 : | glk | 1156 | ============================== |

144 : | LONG TERM ==================== (make Diderot more interesting/attractive from | ||

145 : | ============================== a research standpoint) | ||

146 : | jhr | 1115 | |

147 : | glk | 1156 | IL support for higher-order tensor values (matrices, etc). |

148 : | tensor construction [DONE] | ||

149 : | tensor indexing [DONE] | ||

150 : | tensor slicing | ||

151 : | verify that hessians work correctly [DONE] | ||

152 : | jhr | 1115 | |

153 : | glk | 1156 | Better handling of variables that determines the scope of a variable |

154 : | based on its actual use, instead of where the user defined it. So, | ||

155 : | for example, we should lift strand-invariant variables to global | ||

156 : | scope. Also prune out useless variables, which should include field | ||

157 : | variables after the translation to mid-il. | ||

158 : | |||

159 : | test/vr-kcomp2.diderot: Add support for code like | ||

160 : | (F1 if x else F2)@pos | ||

161 : | This will require duplication of the continuation of the conditional | ||

162 : | (but we should only duplicate over the live-range of the result of the | ||

163 : | conditional. | ||

164 : | |||

165 : | glk | 1162 | [GLK:8] Want: non-trivial field expressions & functions. |

166 : | scalar fields from scalar fields F and G: | ||

167 : | field#0(2)[] X = (sin(F) + 1.0)/2; | ||

168 : | field#0(2)[] X = F*G; | ||

169 : | scalar field of vector field magnitude: | ||

170 : | glk | 1156 | image(2)[2] Vimg = load(...); |

171 : | field#0(2)[] Vlen = |Vimg ⊛ bspln3|; | ||

172 : | glk | 1162 | field of normalized vectors (for LIC and vector field feature extraction) |

173 : | field#2(2)[2] F = ... | ||

174 : | field#0(2)[2] V = normalize(F); | ||

175 : | scalar field of gradient magnitude (for edge detection)) | ||

176 : | glk | 1156 | field#2(2)[] F = Fimg ⊛ bspln3; |

177 : | field#0(2)[] Gmag = |∇F|; | ||

178 : | glk | 1162 | scalar field of squared gradient magnitude (simpler to differentiate): |

179 : | glk | 1156 | field#2(2)[] F = Fimg ⊛ bspln3; |

180 : | field#0(2)[] Gmsq = ∇F•∇F; | ||

181 : | glk | 1162 | There is value in having these, even if the differentiation of them is |

182 : | not supported (hence the indication of "field#0" for these above) | ||

183 : | glk | 1156 | |

184 : | jhr | 1195 | Introduce region types (syntax region(d), where d is the dimension of the |

185 : | region. One useful operator would be | ||

186 : | dom : field#k(d)[s] -> region(d) | ||

187 : | Then the inside test could be written as | ||

188 : | pos ∈ dom(F) | ||

189 : | We could further extend this approach to allow geometric definitions of | ||

190 : | regions. It might also be useful to do inside tests in world space, | ||

191 : | instead of image space. | ||

192 : | |||

193 : | glk | 1162 | co- vs contra- index distinction |

194 : | glk | 1156 | |

195 : | glk | 1162 | Permit field composition: |

196 : | glk | 1156 | field#2(3)[3] warp = bspln3 ⊛ warpData; |

197 : | field#2(3)[] F = bspln3 ⊛ img; | ||

198 : | field#2(3)[] Fwarp = F ◦ warp; | ||

199 : | glk | 1162 | So Fwarp(x) = F(warp(X)). Chain rule can be used for differentation. |

200 : | This will be instrumental for expressing non-rigid registration | ||

201 : | methods (but those will require co-vs-contra index distinction) | ||

202 : | glk | 1156 | |

203 : | glk | 1155 | Allow the convolution to be specified either as a single 1D kernel |

204 : | (as we have it now): | ||

205 : | field#2(3)[] F = bspln3 ⊛ img; | ||

206 : | or, as a tensor product of kernels, one for each axis, e.g. | ||

207 : | field#0(3)[] F = (bspln3 ⊗ bspln3 ⊗ tent) ⊛ img; | ||

208 : | glk | 1212 | This is especially important for things like time-varying fields |

209 : | and the use of scale-space in field visualization: one axis of the | ||

210 : | must be convolved with a different kernel during probing. | ||

211 : | What is very unclear is how, in such cases, we should notate the | ||

212 : | gradient, when we only want to differentiate with respect to some | ||

213 : | subset of the axes. One ambitious idea would be: | ||

214 : | glk | 1162 | field#0(3)[] Ft = (bspln3 ⊗ bspln3 ⊗ tent) ⊛ img; // 2D time-varying field |

215 : | glk | 1204 | field#0(2)[] F = lambda([x,y], Ft([x,y,42.0])) // restriction to time=42.0 |

216 : | vec2 grad = ∇F([x,y]); // 2D gradient | ||

217 : | glk | 1155 | |

218 : | glk | 1204 | Tensors of order 3 (e.g. gradients of diffusion tensor fields, or |

219 : | hessians of vector fields) and order 4 (e.g. Hessians of diffusion | ||

220 : | tensor fields). | ||

221 : | |||

222 : | glk | 1162 | representation of tensor symmetry |

223 : | jhr | 1115 | (have to identify the group of index permutations that are symmetries) |

224 : | |||

225 : | dot works on all tensors | ||

226 : | |||

227 : | outer works on all tensors | ||

228 : | |||

229 : | glk | 1204 | Help for debugging Diderot programs: need to be able to uniquely |

230 : | identify strands, and for particular strands that are known to behave | ||

231 : | badly, do something like printf or other logging of their computations | ||

232 : | and updates. | ||

233 : | |||

234 : | Permit writing dimensionally general code: Have some statement of the | ||

235 : | dimension of the world "W" (or have it be learned from one particular | ||

236 : | field of interest), and then able to write "vec" instead of | ||

237 : | "vec2/vec3", and perhaps "tensor[W,W]" instead of | ||

238 : | "tensor[2,2]/tensor[3,3]" | ||

239 : | |||

240 : | Traits: all things things that have boilerplate code (especially | ||

241 : | volume rendering) should be expressed in terms of the unique | ||

242 : | computational core. Different kinds of streamline/tractography | ||

243 : | computation will be another example, as well as particle systems. | ||

244 : | |||

245 : | jhr | 1115 | Einstein summation notation |

246 : | |||

247 : | "tensor comprehension" (like list comprehension) | ||

248 : | |||

249 : | glk | 1204 | Fields coming from different sources of data: |

250 : | * triangular or tetrahedral meshes over 2D or 3D domains (of the | ||

251 : | source produced by finite-element codes; these will come with their | ||

252 : | own specialized kinds of reconstruction kernels, called "basis | ||

253 : | functions" in this context) | ||

254 : | * Large point clouds, with some radial basis function around each point, | ||

255 : | which will be tuned by parameters of the point (at least one parameter | ||

256 : | giving some notion of radius) | ||

257 : | |||

258 : | jhr | 1115 | ====================== |

259 : | BUGS ================= | ||

260 : | ====================== | ||

261 : | |||

262 : | test/zslice2.diderot: | ||

263 : | // HEY (bug) bspln5 leads to problems ... | ||

264 : | // uncaught exception Size [size] | ||

265 : | // raised at c-target/c-target.sml:47.15-47.19 | ||

266 : | //field#4(3)[] F = img ⊛ bspln5; |

root@smlnj-gforge.cs.uchicago.edu | ViewVC Help |

Powered by ViewVC 1.0.0 |