# SCM Repository

# Annotation of /branches/vis12/TODO

Parent Directory | Revision Log

Revision **1388** -
(**view**)
(**download**)

Original Path: *trunk/TODO*

1 : | glk | 1162 | NOTE: GLK's approximate ranking of 8 most important tagged with |

2 : | jhr | 1115 | [GLK:1], [GLK:2], ... |

3 : | |||

4 : | glk | 1156 | ======================== |

5 : | SHORT TERM ============= (*needed* for streamlines & tractography) | ||

6 : | ======================== | ||

7 : | jhr | 1115 | |

8 : | jhr | 1388 | Remove CL from compiler [DONE] |

9 : | jhr | 1257 | |

10 : | glk | 1338 | [GLK:2] Add sequence types (needed for evals & evecs) |

11 : | jhr | 1115 | syntax |

12 : | types: ty '{' INT '}' | ||

13 : | value construction: '{' e1 ',' … ',' en '}' | ||

14 : | indexing: e '{' e '}' | ||

15 : | |||

16 : | glk | 1338 | [GLK:3] evals & evecs for symmetric tensor[2,2] and |

17 : | glk | 1162 | tensor[3,3] (requires sequences) |

18 : | |||

19 : | glk | 1156 | ability to emit/track/record variables into dynamically re-sized |

20 : | runtime buffer | ||

21 : | jhr | 1115 | |

22 : | jhr | 1388 | tensor fields: convolution on general tensor images (order > 1) |

23 : | jhr | 1115 | |

24 : | glk | 1156 | ======================== |

25 : | glk | 1162 | SHORT-ISH TERM ========= (to make using Diderot less annoying to |

26 : | ======================== program in, and slow to execute) | ||

27 : | jhr | 1115 | |

28 : | jhr | 1246 | value-numbering optimization [DONE] |

29 : | jhr | 1115 | |

30 : | glk | 1167 | Allow ".ddro" file extensions in addition to ".diderot" |

31 : | |||

32 : | glk | 1204 | Be able to output values of type tensor[2,2] and tensor[3,3]; |

33 : | (currently only scalars & vectors). Want to add some regression tests | ||

34 : | based on this and currently can't | ||

35 : | glk | 1167 | |

36 : | glk | 1338 | [GLK:1] Proper handling of stabilize method |

37 : | jhr | 1115 | |

38 : | glk | 1162 | allow "*" to represent "modulate": per-component multiplication of |

39 : | vectors, and vectors only (not tensors of order 2 or higher). Once | ||

40 : | sequences are implemented this should be removed: the operation is not | ||

41 : | invariant WRT basis so it is not a legit vector computation. | ||

42 : | |||

43 : | implicit type promotion of integers to reals where reals are | ||

44 : | required (e.g. not exponentiation "^") | ||

45 : | |||

46 : | glk | 1338 | [GLK:4] Save Diderot output to nrrd, instead of "mip.txt" |

47 : | jhr | 1115 | For grid of strands, save to similarly-shaped array |

48 : | For list of strands, save to long 1-D (or 2-D for non-scalar output) list | ||

49 : | For ragged things (like tractography output), will need to save both | ||

50 : | complete list of values, as well as list of start indices and lengths | ||

51 : | to index into complete list | ||

52 : | |||

53 : | glk | 1338 | [GLK:5] Use of Teem's "hest" command-line parser for getting |

54 : | jhr | 1388 | any "input" variables that are not defined in the source file. [DONE] |

55 : | jhr | 1115 | |

56 : | glk | 1338 | [GLK:6] ability to declare a field so that probe positions are |

57 : | glk | 1120 | *always* "inside"; with various ways of mapping the known image values |

58 : | to non-existant index locations. One possible syntax emphasizes that | ||

59 : | there is a index mapping function that logically precedes convolution: | ||

60 : | glk | 1162 | F = bspln3 ⊛ (img ◦ clamp) |

61 : | glk | 1120 | F = bspln3 ⊛ (img ◦ repeat) |

62 : | F = bspln3 ⊛ (img ◦ mirror) | ||

63 : | where "◦" or "∘" is used to indicate function composition | ||

64 : | jhr | 1115 | |

65 : | glk | 1162 | Level of differentiability in field type should be statement about how |

66 : | much differentiation the program *needs*, rather than what the kernel | ||

67 : | *provides*. The needed differentiability can be less than or equal to | ||

68 : | the provided differentiability. | ||

69 : | |||

70 : | glk | 1156 | Use ∇⊗ etc. syntax |

71 : | syntax [DONE] | ||

72 : | typechecking | ||

73 : | IL and codegen | ||

74 : | jhr | 1115 | |

75 : | glk | 1156 | Add type aliases for color types |

76 : | rgb = real{3} | ||

77 : | rgba = real{4} | ||

78 : | jhr | 1115 | |

79 : | ============================== | ||

80 : | glk | 1156 | MEDIUM TERM ================== (*needed* for particles) |

81 : | jhr | 1115 | ============================== |

82 : | |||

83 : | run-time birth of strands | ||

84 : | |||

85 : | "initially" supports lists | ||

86 : | |||

87 : | glk | 1254 | "initially" supports lists of positions output from different |

88 : | initalization Diderot program (or output from the same program; | ||

89 : | e.g. using output of iso2d.diderot for one isovalue to seed the input | ||

90 : | to another invocation of the same program) | ||

91 : | jhr | 1115 | |

92 : | glk | 1156 | Communication between strands: they have to be able to learn each |

93 : | other's state (at the previous iteration). Early version of this can | ||

94 : | have the network of neighbors be completely static (for running one | ||

95 : | strand/pixel image computations). Later version with strands moving | ||

96 : | through the domain will require some spatial data structure to | ||

97 : | optimize discovery of neighbors. | ||

98 : | jhr | 1115 | |

99 : | glk | 1156 | ============================ |

100 : | MEDIUM-ISH TERM ============ (to make Diderot more useful/effective) | ||

101 : | ============================ | ||

102 : | jhr | 1115 | |

103 : | glk | 1156 | Python/ctypes interface to run-time |

104 : | jhr | 1115 | |

105 : | glk | 1156 | support for Python interop and GUI |

106 : | jhr | 1115 | |

107 : | glk | 1162 | Allow integer exponentiation ("^2") to apply to square matrices, |

108 : | to represent repeated matrix multiplication | ||

109 : | |||

110 : | glk | 1156 | Put small 1-D and 2-D fields, when reconstructed specifically by tent |

111 : | and when differentiation is not needed, into faster texture buffers. | ||

112 : | test/illust-vr.diderot is good example of program that uses multiple | ||

113 : | such 1-D fields basically as lookup-table-based function evaluation | ||

114 : | |||

115 : | jhr | 1301 | expand trace in mid to low translation [DONE] |

116 : | glk | 1156 | |

117 : | extend norm (|exp|) to all tensor types [DONE for vectors and matrices] | ||

118 : | |||

119 : | determinant ("det") for tensor[3,3] | ||

120 : | |||

121 : | jhr | 1115 | add ":" for tensor dot product (contracts out two indices |

122 : | instead of one like •), valid for all pairs of tensors with | ||

123 : | at least two indices | ||

124 : | |||

125 : | glk | 1156 | test/uninit.diderot: |

126 : | documents need for better compiler error messages when output variables | ||

127 : | are not initialized; the current messages are very cryptic | ||

128 : | jhr | 1115 | |

129 : | want: warnings when "D" (reserved for differentiation) is declared as | ||

130 : | a variable name (get confusing error messages now) | ||

131 : | |||

132 : | glk | 1156 | ============================== |

133 : | LONG TERM ==================== (make Diderot more interesting/attractive from | ||

134 : | ============================== a research standpoint) | ||

135 : | jhr | 1115 | |

136 : | glk | 1156 | IL support for higher-order tensor values (matrices, etc). |

137 : | tensor construction [DONE] | ||

138 : | tensor indexing [DONE] | ||

139 : | tensor slicing | ||

140 : | verify that hessians work correctly [DONE] | ||

141 : | jhr | 1115 | |

142 : | glk | 1156 | Better handling of variables that determines the scope of a variable |

143 : | based on its actual use, instead of where the user defined it. So, | ||

144 : | for example, we should lift strand-invariant variables to global | ||

145 : | scope. Also prune out useless variables, which should include field | ||

146 : | variables after the translation to mid-il. | ||

147 : | |||

148 : | test/vr-kcomp2.diderot: Add support for code like | ||

149 : | (F1 if x else F2)@pos | ||

150 : | This will require duplication of the continuation of the conditional | ||

151 : | (but we should only duplicate over the live-range of the result of the | ||

152 : | conditional. | ||

153 : | |||

154 : | glk | 1338 | [GLK:7] Want: non-trivial field expressions & functions. |

155 : | glk | 1162 | scalar fields from scalar fields F and G: |

156 : | field#0(2)[] X = (sin(F) + 1.0)/2; | ||

157 : | field#0(2)[] X = F*G; | ||

158 : | scalar field of vector field magnitude: | ||

159 : | glk | 1156 | image(2)[2] Vimg = load(...); |

160 : | field#0(2)[] Vlen = |Vimg ⊛ bspln3|; | ||

161 : | glk | 1162 | field of normalized vectors (for LIC and vector field feature extraction) |

162 : | field#2(2)[2] F = ... | ||

163 : | field#0(2)[2] V = normalize(F); | ||

164 : | scalar field of gradient magnitude (for edge detection)) | ||

165 : | glk | 1156 | field#2(2)[] F = Fimg ⊛ bspln3; |

166 : | field#0(2)[] Gmag = |∇F|; | ||

167 : | glk | 1162 | scalar field of squared gradient magnitude (simpler to differentiate): |

168 : | glk | 1156 | field#2(2)[] F = Fimg ⊛ bspln3; |

169 : | field#0(2)[] Gmsq = ∇F•∇F; | ||

170 : | glk | 1162 | There is value in having these, even if the differentiation of them is |

171 : | not supported (hence the indication of "field#0" for these above) | ||

172 : | glk | 1156 | |

173 : | jhr | 1195 | Introduce region types (syntax region(d), where d is the dimension of the |

174 : | region. One useful operator would be | ||

175 : | dom : field#k(d)[s] -> region(d) | ||

176 : | Then the inside test could be written as | ||

177 : | pos ∈ dom(F) | ||

178 : | We could further extend this approach to allow geometric definitions of | ||

179 : | regions. It might also be useful to do inside tests in world space, | ||

180 : | instead of image space. | ||

181 : | |||

182 : | glk | 1162 | co- vs contra- index distinction |

183 : | glk | 1156 | |

184 : | glk | 1162 | Permit field composition: |

185 : | glk | 1156 | field#2(3)[3] warp = bspln3 ⊛ warpData; |

186 : | field#2(3)[] F = bspln3 ⊛ img; | ||

187 : | field#2(3)[] Fwarp = F ◦ warp; | ||

188 : | glk | 1162 | So Fwarp(x) = F(warp(X)). Chain rule can be used for differentation. |

189 : | This will be instrumental for expressing non-rigid registration | ||

190 : | methods (but those will require co-vs-contra index distinction) | ||

191 : | glk | 1156 | |

192 : | glk | 1155 | Allow the convolution to be specified either as a single 1D kernel |

193 : | (as we have it now): | ||

194 : | field#2(3)[] F = bspln3 ⊛ img; | ||

195 : | or, as a tensor product of kernels, one for each axis, e.g. | ||

196 : | field#0(3)[] F = (bspln3 ⊗ bspln3 ⊗ tent) ⊛ img; | ||

197 : | glk | 1212 | This is especially important for things like time-varying fields |

198 : | and the use of scale-space in field visualization: one axis of the | ||

199 : | must be convolved with a different kernel during probing. | ||

200 : | What is very unclear is how, in such cases, we should notate the | ||

201 : | gradient, when we only want to differentiate with respect to some | ||

202 : | subset of the axes. One ambitious idea would be: | ||

203 : | glk | 1162 | field#0(3)[] Ft = (bspln3 ⊗ bspln3 ⊗ tent) ⊛ img; // 2D time-varying field |

204 : | glk | 1204 | field#0(2)[] F = lambda([x,y], Ft([x,y,42.0])) // restriction to time=42.0 |

205 : | vec2 grad = ∇F([x,y]); // 2D gradient | ||

206 : | glk | 1155 | |

207 : | glk | 1204 | Tensors of order 3 (e.g. gradients of diffusion tensor fields, or |

208 : | hessians of vector fields) and order 4 (e.g. Hessians of diffusion | ||

209 : | tensor fields). | ||

210 : | |||

211 : | glk | 1162 | representation of tensor symmetry |

212 : | jhr | 1115 | (have to identify the group of index permutations that are symmetries) |

213 : | |||

214 : | dot works on all tensors | ||

215 : | |||

216 : | outer works on all tensors | ||

217 : | |||

218 : | glk | 1204 | Help for debugging Diderot programs: need to be able to uniquely |

219 : | identify strands, and for particular strands that are known to behave | ||

220 : | badly, do something like printf or other logging of their computations | ||

221 : | and updates. | ||

222 : | |||

223 : | Permit writing dimensionally general code: Have some statement of the | ||

224 : | dimension of the world "W" (or have it be learned from one particular | ||

225 : | field of interest), and then able to write "vec" instead of | ||

226 : | "vec2/vec3", and perhaps "tensor[W,W]" instead of | ||

227 : | "tensor[2,2]/tensor[3,3]" | ||

228 : | |||

229 : | Traits: all things things that have boilerplate code (especially | ||

230 : | volume rendering) should be expressed in terms of the unique | ||

231 : | computational core. Different kinds of streamline/tractography | ||

232 : | computation will be another example, as well as particle systems. | ||

233 : | |||

234 : | jhr | 1115 | Einstein summation notation |

235 : | |||

236 : | "tensor comprehension" (like list comprehension) | ||

237 : | |||

238 : | glk | 1204 | Fields coming from different sources of data: |

239 : | * triangular or tetrahedral meshes over 2D or 3D domains (of the | ||

240 : | source produced by finite-element codes; these will come with their | ||

241 : | own specialized kinds of reconstruction kernels, called "basis | ||

242 : | functions" in this context) | ||

243 : | * Large point clouds, with some radial basis function around each point, | ||

244 : | which will be tuned by parameters of the point (at least one parameter | ||

245 : | giving some notion of radius) | ||

246 : | |||

247 : | jhr | 1115 | ====================== |

248 : | BUGS ================= | ||

249 : | ====================== | ||

250 : | |||

251 : | test/zslice2.diderot: | ||

252 : | // HEY (bug) bspln5 leads to problems ... | ||

253 : | // uncaught exception Size [size] | ||

254 : | // raised at c-target/c-target.sml:47.15-47.19 | ||

255 : | //field#4(3)[] F = img ⊛ bspln5; | ||

256 : | glk | 1336 |

root@smlnj-gforge.cs.uchicago.edu | ViewVC Help |

Powered by ViewVC 1.0.0 |