| 1 |
*************************************************** |
This to do for the Vis 2012 submission |
|
*************************************************** |
|
|
THIS TODO HAS BEEN MOVED TO THE DIDEROT WIKI: |
|
| 2 |
|
|
| 3 |
http://diderot-wiki.cs.uchicago.edu/index.php/Todo |
3rd order tensors: 2x2x2 and 3x3x3 |
| 4 |
|
|
| 5 |
PLEASE USE THAT PAGE TO UPDATE PROBLEMS AND PROGRESS |
dynamic-length sequences |
|
*************************************************** |
|
|
*************************************************** |
|
| 6 |
|
|
| 7 |
NOTE: GLK's approximate ranking of 8 most important tagged with |
curl 2D and 3D |
|
[GLK:1], [GLK:2], ... |
|
| 8 |
|
|
| 9 |
======================== |
support iteration over image indices |
|
SHORT TERM ============= (*needed* for streamlines & tractography) |
|
|
======================== |
|
|
|
|
|
[GLK:2] Add sequence types (needed for evals & evecs) |
|
|
syntax |
|
|
types: ty '{' INT '}' |
|
|
value construction: '{' e1 ',' … ',' en '}' |
|
|
indexing: e '{' e '}' |
|
|
|
|
|
[GLK:3] evals & evecs for symmetric tensor[2,2] and |
|
|
tensor[3,3] (requires sequences) |
|
|
|
|
|
ability to emit/track/record variables into dynamically re-sized |
|
|
runtime output buffer |
|
|
|
|
|
[GLK:4] tensor fields from tensor images: Initially need at least |
|
|
convolution on tensor[2,2] and tensor[3,3] (the same component-wise |
|
|
convolution as for vectors). |
|
|
|
|
|
======================== |
|
|
SHORT-ISH TERM ========= (to make using Diderot less annoying to |
|
|
======================== program in, and slow to execute) |
|
|
|
|
|
Allow ".ddro" file extensions in addition to ".diderot" |
|
|
|
|
|
Be able to output values of type tensor[2,2] and tensor[3,3]; |
|
|
(currently only scalars & vectors). Want to add some regression tests |
|
|
based on this and currently can't |
|
|
|
|
|
[GLK:1] Proper handling of stabilize method |
|
|
|
|
|
Convolution on general tensor images (order > 2) |
|
|
|
|
|
allow "*" to represent "modulate": per-component multiplication of |
|
|
vectors, and vectors only (not tensors of order 2 or higher). Once |
|
|
sequences are implemented this should be removed: the operation is not |
|
|
invariant WRT basis so it is not a legit vector computation. |
|
|
|
|
|
implicit type promotion of integers to reals where reals are |
|
|
required (e.g. not exponentiation "^") |
|
|
|
|
|
[Nick working on this] Save Diderot output to nrrd, instead of "mip.txt" |
|
|
For grid of strands, save to similarly-shaped array |
|
|
For list of strands, save to long 1-D (or 2-D for non-scalar output) list |
|
|
For ragged things (like tractography output), will need to save both |
|
|
complete list of values, as well as list of start indices and lengths |
|
|
to index into complete list |
|
|
|
|
|
[GLK:6] ability to declare a field so that probe positions are |
|
|
*always* "inside"; with various ways of mapping the known image values |
|
|
to non-existant index locations. One possible syntax emphasizes that |
|
|
there is a index mapping function that logically precedes convolution: |
|
|
F = bspln3 ⊛ (img ◦ clamp) |
|
|
F = bspln3 ⊛ (img ◦ repeat) |
|
|
F = bspln3 ⊛ (img ◦ mirror) |
|
|
where "◦" or "∘" is used to indicate function composition |
|
|
|
|
|
Level of differentiability in field type should be statement about how |
|
|
much differentiation the program *needs*, rather than what the kernel |
|
|
*provides*. The needed differentiability can be less than or equal to |
|
|
the provided differentiability. |
|
|
|
|
|
Use ∇⊗ etc. syntax |
|
|
syntax [DONE] |
|
|
typechecking |
|
|
IL and codegen |
|
|
|
|
|
Add type aliases for color types |
|
|
rgb = real{3} |
|
|
rgba = real{4} |
|
|
|
|
|
Revisit how images are created within the language. |
|
|
The "load" operator should probably go away, and its strange |
|
|
that strings are there only as a way to refer to nrrd filenames |
|
|
|
|
|
============================== |
|
|
MEDIUM TERM ================== (*needed* for particles) |
|
|
============================== |
|
|
|
|
|
[Lamont working on this] run-time birth of strands |
|
|
|
|
|
"initially" supports lists |
|
|
|
|
|
"initially" supports lists of positions output from different |
|
|
initalization Diderot program (or output from the same program; |
|
|
e.g. using output of iso2d.diderot for one isovalue to seed the input |
|
|
to another invocation of the same program) |
|
|
|
|
|
[Lamont working on this] Communication between strands: they have to |
|
|
be able to learn each other's state (at the previous iteration). |
|
|
Early version of this can have the network of neighbors be completely |
|
|
static (for running one strand/pixel image computations). Later |
|
|
version with strands moving through the domain will require some |
|
|
spatial data structure to optimize discovery of neighbors. |
|
|
|
|
|
============================ |
|
|
MEDIUM-ISH TERM ============ (to make Diderot more useful/effective) |
|
|
============================ |
|
|
|
|
|
[GLK:5] Want code-generation working for tensors of order three. |
|
|
Order three matters for edge detection in scalar fields (to get |
|
|
second derivatives of gradient magnitude), second derivatives |
|
|
of vector fields (for some feature extraction), and first |
|
|
derivatives of diffusion tensor fields. |
|
|
|
|
|
Python/ctypes interface to run-time |
|
|
|
|
|
support for Python interop and GUI |
|
|
|
|
|
Allow integer exponentiation ("^2") to apply to square matrices, |
|
|
to represent repeated matrix multiplication |
|
|
|
|
|
Put small 1-D and 2-D fields, when reconstructed specifically by tent |
|
|
and when differentiation is not needed, into faster texture buffers. |
|
|
test/illust-vr.diderot is good example of program that uses multiple |
|
|
such 1-D fields basically as lookup-table-based function evaluation |
|
|
|
|
|
extend norm (|exp|) to all tensor types [DONE for vectors and matrices] |
|
|
|
|
|
determinant ("det") for tensor[3,3] |
|
|
|
|
|
add ":" for tensor dot product (contracts out two indices |
|
|
instead of one like •), valid for all pairs of tensors with |
|
|
at least two indices |
|
|
|
|
|
test/uninit.diderot: |
|
|
documents need for better compiler error messages when output variables |
|
|
are not initialized; the current messages are very cryptic |
|
|
|
|
|
want: warnings when "D" (reserved for differentiation) is declared as |
|
|
a variable name (get confusing error messages now) |
|
|
|
|
|
============================== |
|
|
LONG TERM ==================== (make Diderot more interesting/attractive from |
|
|
============================== a research standpoint) |
|
|
|
|
|
IL support for higher-order tensor values (matrices, etc). |
|
|
tensor construction [DONE] |
|
|
tensor indexing [DONE] |
|
|
tensor slicing |
|
|
|
|
|
Better handling of variables that determines the scope of a variable |
|
|
based on its actual use, instead of where the user defined it. So, |
|
|
for example, we should lift strand-invariant variables to global |
|
|
scope. Also prune out useless variables, which should include field |
|
|
variables after the translation to mid-il. |
|
|
|
|
|
test/vr-kcomp2.diderot: Add support for code like |
|
|
(F1 if x else F2)@pos |
|
|
This will require duplication of the continuation of the conditional |
|
|
(but we should only duplicate over the live-range of the result of the |
|
|
conditional. |
|
|
|
|
|
[GLK:7] Want: non-trivial field expressions & functions. |
|
|
scalar fields from scalar fields F and G: |
|
|
field#0(2)[] X = (sin(F) + 1.0)/2; |
|
|
field#0(2)[] X = F*G; |
|
|
scalar field of vector field magnitude: |
|
|
image(2)[2] Vimg = load(...); |
|
|
field#0(2)[] Vlen = |Vimg ⊛ bspln3|; |
|
|
field of normalized vectors (for LIC and vector field feature extraction) |
|
|
field#2(2)[2] F = ... |
|
|
field#0(2)[2] V = normalize(F); |
|
|
scalar field of gradient magnitude (for edge detection)) |
|
|
field#2(2)[] F = Fimg ⊛ bspln3; |
|
|
field#0(2)[] Gmag = |∇F|; |
|
|
scalar field of squared gradient magnitude (simpler to differentiate): |
|
|
field#2(2)[] F = Fimg ⊛ bspln3; |
|
|
field#0(2)[] Gmsq = ∇F•∇F; |
|
|
There is value in having these, even if the differentiation of them is |
|
|
not supported (hence the indication of "field#0" for these above) |
|
|
|
|
|
Introduce region types (syntax region(d), where d is the dimension of the |
|
|
region. One useful operator would be |
|
|
dom : field#k(d)[s] -> region(d) |
|
|
Then the inside test could be written as |
|
|
pos ∈ dom(F) |
|
|
We could further extend this approach to allow geometric definitions of |
|
|
regions. It might also be useful to do inside tests in world space, |
|
|
instead of image space. |
|
|
|
|
|
co- vs contra- index distinction |
|
|
|
|
|
Permit field composition: |
|
|
field#2(3)[3] warp = bspln3 ⊛ warpData; |
|
|
field#2(3)[] F = bspln3 ⊛ img; |
|
|
field#2(3)[] Fwarp = F ◦ warp; |
|
|
So Fwarp(x) = F(warp(X)). Chain rule can be used for differentation. |
|
|
This will be instrumental for expressing non-rigid registration |
|
|
methods (but those will require co-vs-contra index distinction) |
|
|
|
|
|
Allow the convolution to be specified either as a single 1D kernel |
|
|
(as we have it now): |
|
|
field#2(3)[] F = bspln3 ⊛ img; |
|
|
or, as a tensor product of kernels, one for each axis, e.g. |
|
|
field#0(3)[] F = (bspln3 ⊗ bspln3 ⊗ tent) ⊛ img; |
|
|
This is especially important for things like time-varying fields |
|
|
and the use of scale-space in field visualization: one axis of the |
|
|
must be convolved with a different kernel during probing. |
|
|
What is very unclear is how, in such cases, we should notate the |
|
|
gradient, when we only want to differentiate with respect to some |
|
|
subset of the axes. One ambitious idea would be: |
|
|
field#0(3)[] Ft = (bspln3 ⊗ bspln3 ⊗ tent) ⊛ img; // 2D time-varying field |
|
|
field#0(2)[] F = lambda([x,y], Ft([x,y,42.0])) // restriction to time=42.0 |
|
|
vec2 grad = ∇F([x,y]); // 2D gradient |
|
|
|
|
|
representation of tensor symmetry |
|
|
(have to identify the group of index permutations that are symmetries) |
|
|
|
|
|
dot works on all tensors |
|
|
|
|
|
outer works on all tensors |
|
|
|
|
|
Help for debugging Diderot programs: need to be able to uniquely |
|
|
identify strands, and for particular strands that are known to behave |
|
|
badly, do something like printf or other logging of their computations |
|
|
and updates. |
|
|
|
|
|
Permit writing dimensionally general code: Have some statement of the |
|
|
dimension of the world "W" (or have it be learned from one particular |
|
|
field of interest), and then able to write "vec" instead of |
|
|
"vec2/vec3", and perhaps "tensor[W,W]" instead of |
|
|
"tensor[2,2]/tensor[3,3]" |
|
|
|
|
|
Traits: all things things that have boilerplate code (especially |
|
|
volume rendering) should be expressed in terms of the unique |
|
|
computational core. Different kinds of streamline/tractography |
|
|
computation will be another example, as well as particle systems. |
|
|
|
|
|
Einstein summation notation |
|
|
|
|
|
"tensor comprehension" (like list comprehension) |
|
|
|
|
|
Fields coming from different sources of data: |
|
|
* triangular or tetrahedral meshes over 2D or 3D domains (of the |
|
|
source produced by finite-element codes; these will come with their |
|
|
own specialized kinds of reconstruction kernels, called "basis |
|
|
functions" in this context) |
|
|
* Large point clouds, with some radial basis function around each point, |
|
|
which will be tuned by parameters of the point (at least one parameter |
|
|
giving some notion of radius) |
|
|
|
|
|
====================== |
|
|
BUGS ================= |
|
|
====================== |
|
|
|
|
|
test/zslice2.diderot: |
|
|
// HEY (bug) bspln5 leads to problems ... |
|
|
// uncaught exception Size [size] |
|
|
// raised at c-target/c-target.sml:47.15-47.19 |
|
|
//field#4(3)[] F = img ⊛ bspln5; |
|
| 10 |
|
|
Parent Directory
| 
Revision Log
| 
Patch
