Henderson, Robert

Underwood, Ian

Dutton, Neale

Mattioli Della Rocca, Francescopaolo

2021-10-06T10:46:18Z

2021-10-06T10:46:18Z

2021-07-31

Time of flight (ToF) imaging has become a prime 3D sensing technology in consumer and industrial applications, favoured for its low mechanical and computational complexity amongst ranging techniques. Despite its advantages, ToF sensors exhibit high data rate and power due to their temporal resolution and fast frame rates. In Internet of Things (IoT) networks, energy and output data efficiency are crucial for sensors to prevent bandwidth saturation of the processing nodes, while scaling the number of connected devices and extending the life of systems on limited energy supplies. 3D imaging applications, such as surveillance, object tracking and classification in industrial robotic automation, feature conditions where the event interval and frequency of events can be far lower than the time spent by the sensor capturing static scenes. ToF cameras lack the embedded intelligence to recognise frames of interest. Temporal contrast vision sensors, on the other hand, efficiently limit the expense of energy and data bandwidth to detect motion by outputting low data rate binary frames. However, while these sensors integrate well with standard intensity imaging, they do not provide 3D image sensing functionality. The goal of this research is to investigate the feasibility of combining vision algorithms with ToF imaging topologies to design a motion-triggered 3D image sensor. A frame differencing vision architecture is leveraged to limit ToF acquisition to short event-driven duty-cycles, reducing the high consumption and data traffic to external processing and shortening the active time of the ToF emitter. The design strategy is founded on hardware resource sharing in the implementation of both motion detection and ranging functions to minimise pixel area scaling. The shot noise-limited light detection provided by single photon avalanche diodes (SPAD), further reinforced by their recent integration in advanced digital CMOS technologies, makes this detector the enabling factor for the design of the first fully digital motion-triggered 3D image sensor, superior in motion contrast sensitivity and noise performance to analogue-based vision sensors. A 128×128 SPAD motion-triggered 3D image sensor was designed in STMicroelectronics 40 nm CMOS process. 40 µm×20 µm pixels integrate two 16-bit time-gated counters to acquire a 6-bin ToF histogram. The counters are repurposed for acquisition of consecutive frames for motion detection by a frame comparison algorithm. An embedded column-parallel processor performs relative frame differencing with an adaptive per pixel threshold. The sensor achieves a 78% power saving and a data rate reduction of 10 times when operating the sensor in motion-triggered 3D imaging at 30 fps, compared to continuous ToF acquisition. High motion sensitivity below 4% contrast is achieved with a 1.39% pixel-to-pixel contrast FPN by shot noise-limited integration and spatial noise filtering over a dynamic range exceeding 60 dB. ToF 3D imaging is accomplished by a 100 fps histogram acquisition triggered by motion in the scene with 1.5 cm depth accuracy over a measured 3 m range.

https://hdl.handle.net/1842/38122

http://dx.doi.org/10.7488/era/1391

en

The University of Edinburgh

F. Mattioli Della Rocca, J. Nedbal, D. Tyndall, N. Krstajic, D. D.-U. Li, S. M. Ameer-Beg, ´ and R. K. Henderson, "Real-time fluorescence lifetime actuation for cell sorting using a CMOS SPAD silicon photomultiplier," Optics Letters 41, 673-676, 2016

F. Mattioli Della Rocca, T. A. Abbas, N. A. W. Dutton and R. K. Henderson, "A high dynamic range SPAD pixel for time of flight imaging," 2017 IEEE SENSORS, Glasgow, 2017, pp. 1-3

F. Mattioli Della Rocca, H. Mai, S. W. Hutchings, T. Al Abbas, A. Tsiamis, P. Lomax, I. Gyongy, N. A. W. Dutton, and R. K. Henderson, "A 128×128 SPAD Dynamic Vision Triggered Time of Flight Imager," ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference (ESSCIRC), Cracow, Poland, 2019, pp. 93-96.

F. Mattioli Della Rocca, H. Mai, Sam W. Hutchings, T. Al Abbas, K. Buckbee, A. Tsiamis, P. Lomax, I. Gyongy, N. A. W. Dutton, and R. K. Henderson, "A 128×128 SPAD Motion Triggered Time of Flight Image Sensor with In-Pixel Histogram and Column-Parallel Vision Processor", in IEEE Journal of Solid-State Circuits, vol. 55, no. 7, pp. 1762-1775, July 2020.

N. Finlayson, T. Al Abbas, F. Mattioli Della Rocca, O. Almer, S. Gnecchi, N. A. W. Dutton, and R. K. Henderson, "Hypervelocity time-of-flight characterisation of a 14GS/s histogram ming CMOS SPAD sensor," Proc. SPIE 10111, Quantum Sensing and Nano Electronics and Photonics XIV, 101112Z, 2017.

T. Al Abbas, N. A. W. Dutton, O. Almer, F. Mattioli Della Rocca, S. Pellegrini, B. Rae, D. Golanski, and R. K. Henderson, "8.25 mm pitch 66% fill factor global shared well SPAD image sensor in 40nm CMOS FSI technology”, in Proc. IISW, pp. 238–241, 2017.

T. Al Abbas, N. A. W. Dutton, O. Almer, N. Finlayson, F. Mattioli Della Rocca, and R. K. Henderson, "A CMOS SPAD Sensor With a Multi-Event Folded Flash Time-to-Digital Converter for Ultra-Fast Optical Transient Capture," in IEEE Sensors Journal, vol. 18, no. 8, pp. 3163-3173, 15 April15, 2018.

N. Dutton, T. Al Abbas, I. Gyongy, F. Mattioli Della Rocca, and R. Henderson, "High Dynamic Range Imaging at the Quantum Limit with Single Photon Avalanche Diode-Based Image Sensors," Sensors, vol. 18, no. 4, p. 1166, Apr. 2018.

N. A. W. Dutton, T. Al Abbas, F. Mattioli Della Rocca, N. Finlayson, B. Rae, and R. K. Henderson "Time of Flight Imaging and Sensing for Mobile Applications", In: Makinwa K., Baschirotto A., Harpe P. (eds) Low-Power Analog Techniques, Sensors for Mobile Devices, and Energy Efficient Amplifiers. Springer, Cham, 2019.

T. Al Abbas, D. Chitnis, F. Mattioli Della Rocca, and R. K. Henderson. "Dual layer 3D-stacked high dynamic range SPAD pixel." In Proc. IISW, pp. 206-209, 2019.

R. K. Henderson, N. Johnston, F. Mattioli Della Rocca, H. Chen, D. D.-U. Li, G. Hungerford, R. Hirsch, D. Mcloskey, P. Yip, and D. J. S. Birch, "A 192×128 Time Correlated SPAD Image Sensor in 40-nm CMOS Technology," in IEEE Journal of Solid-State Circuits, vol. 54, no. 7, pp. 1907-1916, July 2019.

A. Maccarone, F. Mattioli Della Rocca, A. McCarthy, R. K. Henderson, and G. S. Buller, "Three-dimensional imaging of stationary and moving targets in turbid underwater environments using a single-photon detector array," Opt. Express 27, 28437-28456 (2019).

A. Maccarone, A. McCarthy, J. Tachella, D. A. Garcia, F. Mattioli Della Rocca, Y. Altmann, S. McLaughlin, R. K. Henderson, G. S. Buller, "Underwater depth imaging using time-correlated single-photon counting at video frame rates," Proc. SPIE 11160, Electro-Optical Remote Sensing XIII, 111600I, 10 October, 2019.

F. Mattioli Della Rocca and N. A. W. Dutton. "Method and apparatus for measuring time of flight." United States patent application US 15/343,427. 2018 May 10.

F. Mattioli Della Rocca and J. K. Moore. "Apparatus for use in luminescence applications." U.S. Patent No. 10,119,912. 6 Nov. 2018.

S.Lee, J. K. Moore, and F. Mattioli Della Rocca. "Ranging apparatus." U.S. Patent No. 10,436,581. 8 Oct. 2019

2026-07-31

Time of flight imaging

ToF imaging

output data efficiency

vision algorithms

ToF imaging topologies

motion-triggered 3D image sensor

vision architecture

Motion-triggered 3D imaging with a Time of Flight CMOS SPAD image sensor

Thesis or Dissertation

Doctoral

PhD Doctor of Philosophy

Restricted Access