Advancing pulmonary health: image processing and machine learning in fibre-bundle endo-microscopy systems

Abstract

This thesis presents a comprehensive investigation into advanced image processing and machine learning techniques aimed at enhancing the utility of real-time fluorescence lifetime imaging (FLIm) for in vivo pulmonary optical endomicroscopy (OEM). Addressing the inherent challenges of clinical FLIm applications—such as motion artefacts, uninformative frames, and registration difficulties—this work introduces significant advancements across several key areas.

A novel temporal reliability and accuracy via correlation enhanced registration (TRACER) pipeline is developed to integrate a sequence of pre-processing steps, including uninformative frame removal and motion characterisation via dense optical flow (OF), alongside a tracking-based normalised cross correlation (NCC) image registration method. TRACER yields marked improvements in registration performance, achieving a 20% to 30% enhancement across quality of alignment (QA), structural similarity index measure (SSIM), and normalised root mean squared error (NRMSE) metrics, while operating an order of magnitude faster than the next best registration method. This improved registration efficiency is critical for the real-time clinical applicability of FLIm.

A multi-objective image registration framework is also introduced, leveraging both intensity and lifetime information inherent to FLIm sequences to address complex spatio-temporal dynamics. A novel multi-objective optimisation (MOO) strategy, dynamic multi-objective search (DYNMOS), is proposed, which effectively balances intensity and lifetime contrast for improved registration robustness, particularly in scenarios involving significant displacements or repetitive structures.

The use of Pareto fronts further clarifies the trade-offs between competing registration objectives.

Building on these image processing advances, the thesis explores the impact of pipeline robustness on machine learning (ML) applications for OEM-FLIm data. It addresses uninformative frame detection and automated analysis of biologically relevant probes. A transfer learning (TL) strategy employing a pre-trained VGG16-based convolutional neural network (CNN) is proposed to accurately identify uninformative frames, enhancing data quality and processing efficiency. Additionally, the effects of different pipeline elements are examined in the context of a U-Net-based deep learning framework for the automatic detection of neutrophil activation probe (NAP) signals within FLIm images. These results underscore the importance of robust processing in enabling clinically meaningful, automated disease characterisation at the alveolar level.

In conclusion, this thesis delivers a systematic suite of image processing and ML methodologies that significantly enhance the reliability, accuracy, and automation of FLIm in pulmonary imaging. The combined developments in registration (TRACER, DYNMOS) and ML-driven analysis of frame quality and NAP activity pave the way for clinically translatable applications of fibre-bundle OEM systems.

Future directions should focus on refining these techniques, integrating them into real-time workflows, and broadening their applicability to a wider array of pulmonary diseases and imaging probes. The findings presented here hold strong potential for improving diagnostic precision and advancing our understanding of lung pathology at the microscopic scale.