--- branches/vis12/TODO 2012/01/22 15:23:36 1685
+++ branches/vis12/TODO 2012/01/23 18:49:17 1686
@@ -1,266 +1,10 @@
-***************************************************
-***************************************************
-THIS TODO HAS BEEN MOVED TO THE DIDEROT WIKI:
+This to do for the Vis 2012 submission
-http://diderot-wiki.cs.uchicago.edu/index.php/Todo
+3rd order tensors: 2x2x2 and 3x3x3
-PLEASE USE THAT PAGE TO UPDATE PROBLEMS AND PROGRESS
-***************************************************
-***************************************************
+dynamic-length sequences
-NOTE: GLK's approximate ranking of 8 most important tagged with
-[GLK:1], [GLK:2], ...
+curl 2D and 3D
-========================
-SHORT TERM ============= (*needed* for streamlines & tractography)
-========================
-
-[GLK:2] Add sequence types (needed for evals & evecs)
- syntax
- types: ty '{' INT '}'
- value construction: '{' e1 ',' … ',' en '}'
- indexing: e '{' e '}'
-
-[GLK:3] evals & evecs for symmetric tensor[2,2] and
-tensor[3,3] (requires sequences)
-
-ability to emit/track/record variables into dynamically re-sized
-runtime output buffer
-
-[GLK:4] tensor fields from tensor images: Initially need at least
-convolution on tensor[2,2] and tensor[3,3] (the same component-wise
-convolution as for vectors).
-
-========================
-SHORT-ISH TERM ========= (to make using Diderot less annoying to
-======================== program in, and slow to execute)
-
-Allow ".ddro" file extensions in addition to ".diderot"
-
-Be able to output values of type tensor[2,2] and tensor[3,3];
-(currently only scalars & vectors). Want to add some regression tests
-based on this and currently can't
-
-[GLK:1] Proper handling of stabilize method
-
-Convolution on general tensor images (order > 2)
-
-allow "*" to represent "modulate": per-component multiplication of
-vectors, and vectors only (not tensors of order 2 or higher). Once
-sequences are implemented this should be removed: the operation is not
-invariant WRT basis so it is not a legit vector computation.
-
-implicit type promotion of integers to reals where reals are
-required (e.g. not exponentiation "^")
-
-[Nick working on this] Save Diderot output to nrrd, instead of "mip.txt"
- For grid of strands, save to similarly-shaped array
- For list of strands, save to long 1-D (or 2-D for non-scalar output) list
- For ragged things (like tractography output), will need to save both
- complete list of values, as well as list of start indices and lengths
- to index into complete list
-
-[GLK:6] ability to declare a field so that probe positions are
-*always* "inside"; with various ways of mapping the known image values
-to non-existant index locations. One possible syntax emphasizes that
-there is a index mapping function that logically precedes convolution:
- F = bspln3 ⊛ (img ◦ clamp)
- F = bspln3 ⊛ (img ◦ repeat)
- F = bspln3 ⊛ (img ◦ mirror)
-where "◦" or "∘" is used to indicate function composition
-
-Level of differentiability in field type should be statement about how
-much differentiation the program *needs*, rather than what the kernel
-*provides*. The needed differentiability can be less than or equal to
-the provided differentiability.
-
-Use ∇⊗ etc. syntax
- syntax [DONE]
- typechecking
- IL and codegen
-
-Add type aliases for color types
- rgb = real{3}
- rgba = real{4}
-
-Revisit how images are created within the language.
-The "load" operator should probably go away, and its strange
-that strings are there only as a way to refer to nrrd filenames
-
-==============================
-MEDIUM TERM ================== (*needed* for particles)
-==============================
-
-[Lamont working on this] run-time birth of strands
-
-"initially" supports lists
-
-"initially" supports lists of positions output from different
-initalization Diderot program (or output from the same program;
-e.g. using output of iso2d.diderot for one isovalue to seed the input
-to another invocation of the same program)
-
-[Lamont working on this] Communication between strands: they have to
-be able to learn each other's state (at the previous iteration).
-Early version of this can have the network of neighbors be completely
-static (for running one strand/pixel image computations). Later
-version with strands moving through the domain will require some
-spatial data structure to optimize discovery of neighbors.
-
-============================
-MEDIUM-ISH TERM ============ (to make Diderot more useful/effective)
-============================
-
-[GLK:5] Want code-generation working for tensors of order three.
-Order three matters for edge detection in scalar fields (to get
-second derivatives of gradient magnitude), second derivatives
-of vector fields (for some feature extraction), and first
-derivatives of diffusion tensor fields.
-
-Python/ctypes interface to run-time
-
-support for Python interop and GUI
-
-Allow integer exponentiation ("^2") to apply to square matrices,
-to represent repeated matrix multiplication
-
-Put small 1-D and 2-D fields, when reconstructed specifically by tent
-and when differentiation is not needed, into faster texture buffers.
-test/illust-vr.diderot is good example of program that uses multiple
-such 1-D fields basically as lookup-table-based function evaluation
-
-extend norm (|exp|) to all tensor types [DONE for vectors and matrices]
-
-determinant ("det") for tensor[3,3]
-
-add ":" for tensor dot product (contracts out two indices
-instead of one like •), valid for all pairs of tensors with
-at least two indices
-
-test/uninit.diderot:
-documents need for better compiler error messages when output variables
-are not initialized; the current messages are very cryptic
-
-want: warnings when "D" (reserved for differentiation) is declared as
-a variable name (get confusing error messages now)
-
-==============================
-LONG TERM ==================== (make Diderot more interesting/attractive from
-============================== a research standpoint)
-
-IL support for higher-order tensor values (matrices, etc).
- tensor construction [DONE]
- tensor indexing [DONE]
- tensor slicing
-
-Better handling of variables that determines the scope of a variable
-based on its actual use, instead of where the user defined it. So,
-for example, we should lift strand-invariant variables to global
-scope. Also prune out useless variables, which should include field
-variables after the translation to mid-il.
-
-test/vr-kcomp2.diderot: Add support for code like
- (F1 if x else F2)@pos
-This will require duplication of the continuation of the conditional
-(but we should only duplicate over the live-range of the result of the
-conditional.
-
-[GLK:7] Want: non-trivial field expressions & functions.
-scalar fields from scalar fields F and G:
- field#0(2)[] X = (sin(F) + 1.0)/2;
- field#0(2)[] X = F*G;
-scalar field of vector field magnitude:
- image(2)[2] Vimg = load(...);
- field#0(2)[] Vlen = |Vimg ⊛ bspln3|;
-field of normalized vectors (for LIC and vector field feature extraction)
- field#2(2)[2] F = ...
- field#0(2)[2] V = normalize(F);
-scalar field of gradient magnitude (for edge detection))
- field#2(2)[] F = Fimg ⊛ bspln3;
- field#0(2)[] Gmag = |∇F|;
-scalar field of squared gradient magnitude (simpler to differentiate):
- field#2(2)[] F = Fimg ⊛ bspln3;
- field#0(2)[] Gmsq = ∇F•∇F;
-There is value in having these, even if the differentiation of them is
-not supported (hence the indication of "field#0" for these above)
-
-Introduce region types (syntax region(d), where d is the dimension of the
-region. One useful operator would be
- dom : field#k(d)[s] -> region(d)
-Then the inside test could be written as
- pos ∈ dom(F)
-We could further extend this approach to allow geometric definitions of
-regions. It might also be useful to do inside tests in world space,
-instead of image space.
-
-co- vs contra- index distinction
-
-Permit field composition:
- field#2(3)[3] warp = bspln3 ⊛ warpData;
- field#2(3)[] F = bspln3 ⊛ img;
- field#2(3)[] Fwarp = F ◦ warp;
-So Fwarp(x) = F(warp(X)). Chain rule can be used for differentation.
-This will be instrumental for expressing non-rigid registration
-methods (but those will require co-vs-contra index distinction)
-
-Allow the convolution to be specified either as a single 1D kernel
-(as we have it now):
- field#2(3)[] F = bspln3 ⊛ img;
-or, as a tensor product of kernels, one for each axis, e.g.
- field#0(3)[] F = (bspln3 ⊗ bspln3 ⊗ tent) ⊛ img;
-This is especially important for things like time-varying fields
-and the use of scale-space in field visualization: one axis of the
-must be convolved with a different kernel during probing.
-What is very unclear is how, in such cases, we should notate the
-gradient, when we only want to differentiate with respect to some
-subset of the axes. One ambitious idea would be:
- field#0(3)[] Ft = (bspln3 ⊗ bspln3 ⊗ tent) ⊛ img; // 2D time-varying field
- field#0(2)[] F = lambda([x,y], Ft([x,y,42.0])) // restriction to time=42.0
- vec2 grad = ∇F([x,y]); // 2D gradient
-
-representation of tensor symmetry
-(have to identify the group of index permutations that are symmetries)
-
-dot works on all tensors
-
-outer works on all tensors
-
-Help for debugging Diderot programs: need to be able to uniquely
-identify strands, and for particular strands that are known to behave
-badly, do something like printf or other logging of their computations
-and updates.
-
-Permit writing dimensionally general code: Have some statement of the
-dimension of the world "W" (or have it be learned from one particular
-field of interest), and then able to write "vec" instead of
-"vec2/vec3", and perhaps "tensor[W,W]" instead of
-"tensor[2,2]/tensor[3,3]"
-
-Traits: all things things that have boilerplate code (especially
-volume rendering) should be expressed in terms of the unique
-computational core. Different kinds of streamline/tractography
-computation will be another example, as well as particle systems.
-
-Einstein summation notation
-
-"tensor comprehension" (like list comprehension)
-
-Fields coming from different sources of data:
-* triangular or tetrahedral meshes over 2D or 3D domains (of the
- source produced by finite-element codes; these will come with their
- own specialized kinds of reconstruction kernels, called "basis
- functions" in this context)
-* Large point clouds, with some radial basis function around each point,
- which will be tuned by parameters of the point (at least one parameter
- giving some notion of radius)
-
-======================
-BUGS =================
-======================
-
-test/zslice2.diderot:
-// HEY (bug) bspln5 leads to problems ...
-// uncaught exception Size [size]
-// raised at c-target/c-target.sml:47.15-47.19
-//field#4(3)[] F = img ⊛ bspln5;
+support iteration over image indices