Interactive Computational Imaging for Deformable Object Analysis

Donald G. Dansereau1, Surya P. N. Singh2, Jürgen Leitner1

      [Paper here] [Click red things for details]

Soft is Hard

  • “Is it deformable?”
  • “Is it stiff enough to pick up?”
  • “Is it okay to deform?”
  • “How much deflection to apply?”
  • “Where should I grip it?”
  • “Is it elastic? Fragile? Both?”
  • No specialized sensors
  • Previously unseen objects

Computational Imaging


  • Treat the camera like an optical computer
  • Bayer masks, HDR, focal sweep, flutter shutter, light fields
  • We introduce interactivity: the robot can affect the scene

System Overview


Optical Flow & Stats

Shape of dense optical flow: ratios of parallel, perpendicular components


Interaction: Stimulating the Scene

Periodic, gentle squeezing

  • Problems: Fragile, stiff, low-texture objects

Matched Filter

Amplify motion, reject noise

   EXAMPLE s,u [x4]

  • Treat the video as a 3D block
  • Single-stage linear filter

Results: Discriminating Objects

  • PCA on flow shape statistics
  • No dependence on texture, colour, brightness


Results: Cup Fill Level

  • Side view of paper cup
  • Filled to 5 levels
  • Visual compressibility
  • Similar to Poisson's ratio
  • Preliminary results only