Deep learning based 2D-3D registration system for augmented visualization in Image Guided Endovascular Surgery

Image guided surgery, such as endovascular stenting for abdominal aortic aneurysms (AAA), requires visualization of critical information in real time displayed on top of intra-operative radiological images, which often offer poor contrast and not always reliable anatomical position. Augmentation of intra-operative 2D images requires fusion with pre-operative 3D data, in which structures of interest have higher resolution and are easier to segment, by the matching process of 2D-3D registration. Although this problem has been studied extensively and well established solutions exist, these suffer from considerable drawbacks: insufficient frame rate, limited capture range, a lot of iterative process and complex setup, and involving lots of manual input. To overcome these problems, the emerging techniques of neural networks - successfully applied to other areas of medical image processing - offer the opportunity to explore ways to obtain fully automatic and fast computation with controllable capture ranges, set during the network’s training phase. This thesis proposes a framework for 2D-3D registration based on deep-learning, that aims to align 3D axial computed tomography (CT) scans of the lower spine to 2D fluoroscopic images. The proposed Convolutional Neural Network (CNN) computes the alignment transformation by means of regression of its parameters. In addition, this work proposes a solution for the lack of intra-operative images during training by application of an intensive data augmentation scheme on which training data is generated from the 3D scan by simulation of the radiographic projection process.

Image guided surgery, such as endovascular stenting for abdominal aortic aneurysms (AAA), requires visualisation of critical information in real time displayed on top of intra-operative radiological images, which often offer poor contrast and not always reliable anatomical position. Augmentation of intra-operative 2D images requires fusion with pre-operative 3D data, in which structures of interest have higher resolution and are easier to segment, by the matching process of 2D-3D registration. Although this problem has been studied extensively and well established solutions exist, these suffer from considerable drawbacks: insufficient frame rate, limited capture range, a lot of iterative process and complex setup, and involving lots of manual input. To overcome these problems, the emerging techniques of neural networks - successfully applied to other areas of medical image processing - offer the opportunity to explore ways to obtain fully automatic and fast computation with controllable capture ranges, set during the network’s training phase. This thesis proposes a framework for 2D-3D registration based on deep-learning, that aims to align 3D axial computed tomography (CT) scans of the lower spine to 2D fluoroscopic images. The proposed Convolutional Neural Network (CNN) computes the alignment transformation by means of regression of its parameters. In addition, this work proposes a solution for the lack of intra-operative images during training by application of an intensive data augmentation scheme on which training data is generated from the 3D scan by simulation of the radiographic projection process.