Background and goal of study: Puncture of central vessels is a frequently used therapeutic and diagnostic procedure. The use of ultrasound (US) during needle insertion has become the gold standard. Handling the US probe and needle is challenging, especially in dif ficult anatomic conditions. Our long-term vision is a deep learning based and augmented reality (AR) assisted needle puncture. We aim to visualize the vessel structures in 3D based on 2D US image segmentation. While punctuating, the relative needle tip position and relevant vessels can be highlighted via AR lenses to optimize the image guidance process.
Materials and methods: Our experimental setup (Fig. 1) allows to record robot poses for 3D reconstruction1 and US images simultaneously while moving the probe manually over the vessel structures. We record a pre-clinical dataset consisting of 3445 US images of the v. jugularis and art. carotis from seven dif ferent probands. The data is split into individual subsets for training and testing of a neural network, LinkNet2, for segmentation. Figure 1. Our experimental setup with the US probe mounted to the robot (Panda, Franka Emika) and positioned at the v. jugularis with a visualization of exemplar y US images along with the segmentation label and the segmentation mask predicted by the neural network. Results and discussion: We obtain the best segmentation results for the LinkNet pretrained with a ResNet101 backbone, resulting in a DICE score of 0.915 and a Jaccard Index of 0.847. The segmentation masks of the vessels show a high amount of overlap to the labels (Fig. 1) and capture the form of the vessels. Minor errors occur in areas where the two vessels are too close in the underlying US images.
Conclusion: Our results show that the LinkNet is capable of segmenting the area of interest with high quality. It is a small network, suf ficient for fast data processing. Future work can improve our results using more data. Peripheral nerve block or puncture of groin vessels are further possible applications, as well as training of US inexperienced users.
References: 1. Virga, S., et al. “Automatic force-compliant robotic ultrasound screening of abdominal aortic aneurysms.”IEEE/RSJ IROS, 2016. 2. Chaurasia, A., and E. Culurciello. “Linknet: Exploiting encoder representations for ef ficient semantic segmentation.”IEEE VCIP, 2017.
Acknowledgements: This work was partially funded by FMTHH (grant: 03FMTHH20).
Background and Goal of Study: Puncture of central vessels is a frequently used and important therapeutic and diagnostic procedure. The use of ultrasound (US) during needle insertion has become the standard. Handling the US-probe and needle is challenging, especially in difficult or unclear anatomic conditions. Our long-term vision is a deep learning based and augmented reality (AR) assisted needle puncture. We aim to visualize the vessel structures in 3D based on 2D US image segmentation. While punctuating, the relative needle tip position and relevant vessels can be highlighted via AR lenses to optimize the image guidance process. Materials and Methods: In our experimental setup, we mount the US probe to a collaborative robot and the physician manually moves the US probe along the relevant vessel structures. Robot pose and US images were allocated simultaneously during motion and can be used for 3D vessel reconstructions based on the positions1. We record a pre-clinical dataset consisting of 3445 US images of the v. jugularis int. and art. carotis int. from seven different probands. Part of the data set is used to train a convolutional neural network, LinkNet2, for segmentation of the vessels. The remaining data is used for validation and testing, whereas the separation is made according to the different probands.
Results: We obtain the best segmentation results for the LinkNet pretrained with a ResNet101 backbone, resulting in a DICE score of 0.915 and a Jaccard Index of 0.847. The segmentation masks of the vessels show a high amount of overlap to the labels (Figure 1) and capture the form of the vessels. Minor errors occur in areas where the two vessels are too close in the underlying US images. Discussion and Conclusion: Our results show that the LinkNet is capable of segmenting the area of interest with high quality. It is a small network, sufficient for fast data processing. Furthermore, our data set proves that the network can generalize on unseen data. Future work with further data sets can even improve our current results. Peripheral nerve block or puncture of groin vessels are further possible applications, as well as training of US inexperienced users.
References: 1 S. Virga et al., "Automatic force-compliant robotic ultrasound screening of abdominal aortic aneurysms," 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016, pp. 508-513, doi: 10.1109/IROS.2016.7759101. 2 Chaurasia, A., Culurciello, E.: Linknet: Exploiting encoder representations for efficient semantic segmentation. CoRR abs/1707.03718 (2017),http://arxiv.org/abs/1707.03718