Direct time-of-flight for active 3D imaging

Direct time-of-flight (D-ToF) imagers, which detect the time-of-arrival of reflected light pulses from a synchronized laser, are one of the key enablers for active 3D imaging, which has a wide range of applications from manufacturing to defense. A device called a single photon avalanche diode (SPAD), which is affordable, low-noise, and effective, is generally used to detect the photons' arrival. Finally, a histogram of ToF data is generated using a technique called time correlated single photon counting (TCSPC). Several challenges—background noise, bulky in-pixel circuits, and large volumes of data—must  be overcome to make the combined use of TCSPC and SPAD a viable solution.

Overcoming the challenges of TCSPC and SPAD

To address the issues preventing the widespread adoption of TCSPC and SPAD for 3D imaging, our first objective was to thoroughly review all working modes for TCSPC, utilizing a physically-plausible SPAD sensor model. Secondly, we explored alternatives to the conventional ToF histogram acquisition scheme, focusing on compressive sensing (CS), which is data-agnostic by design, and expectation-maximization (EM), which is intrinsically data-driven. The idea was to arrive at a balanced view of the advantages and potential drawbacks of both techniques and their capacity to enable concrete and practical sensors design with improved characteristics. Finally, we also investigated the application of today's deep learning models for depth map rendering.

Findings promising for tomorrow's depth sensing technologies

We established that ToF histograms from high photon counts in synchronous modes could be accurately modeled using a mixture of truncated-shifted Erlang distributions. A customized EM algorithm was proposed for precise pixel parameter estimation. A novel 2-stage online EM algorithm was developed for efficient peak detection, coupled with a pixel-level hardware implementation. This approach demonstrated a substantial compression ratio of 0.4% and a 40% reduction in pixel pitch, all without sacrificing performance. A pixel-wise CS approach was also designed, with its hardware implementation. Beyond showcasing the effectiveness of information retrieval via a LASSO type of algorithm and enabling a practical pixel pitch reduction, the CS data can also feed deep generative models (DGM) for reconstruction.

​From photons in the wild to meaningful data, a study of TCSPC physical models and processing variants.