by Anderson Maciel (Técnico Lisboa), Catarina Moreira (University of Technology Sydney), and Joaquim Jorge (Técnico Lisboa)
Desktop-based virtual colonoscopy has been proven to be an asset in identifying colon anomalies. The process is accurate, although time consuming. In this article, we present a new design exploring elements of the VR paradigm to make the immersive analysis more efficient while still effective.
Colorectal cancer (CRC) is the third-leading cause of cancer-related mortality both in the United States and worldwide. In 2023 alone, it is projected that around 153,020 individuals will receive a CRC diagnosis, and approximately 52,550 individuals will sadly lose their lives to this disease [1]. Thus, prevention and treatment of colorectal cancer require the early detection of adenomas, polyps, and CRC. While traditional colonoscopy (TC) is the gold standard. However, it is invasive and requires uncomfortable preparation since it requires a scope to be inserted through the rectum.
Virtual Colonoscopy (VC) was created to solve some TC disadvantages [2]. Through CT scans and desktop programs, radiologists navigate through a reconstructed colon in a two-way path: from the rectum to the cecum and from the cecum to the rectum to avoid missing anomalies in folds. Despite solving invasiveness, it brings its disadvantages. Navigation is restricted to a single view, causing uncertainty regarding the types of polyps, leading to a higher referral rate to colonoscopy and superior cost. While VC can detect most colon anomalies, it is highly time-consuming, taking much longer than a conventional total colonoscopy. Moreover, the exam only considers the inside of the colon; extracolonic lesions may still exist. Thus, the need for further exams increases the cost, and on top of that, false positives can occur.
VC proved effective and started to be used in clinical practice. Researchers focused on finding new means to explore and analyse the available data, proposing Immersive Virtual Colonoscopy(IVC) [3]. In IVC, a 3D model of the colon is reconstructed from CT scans and viewed through VR. IVC has unique advantages when compared to desktop VC. It allows radiologists to freely explore the inside and outside of the colon using natural head and body motion to better analyse volumetric data with stereopsis and parallax cues. Indeed, 3D visualisation positively influences search tasks. Navigation is more flexible since radiologists are not bound to a single path, and measures can be taken directly in 3D. A major challenge remains in designing an efficient VR interface for IVC. Our work focuses on reducing exam duration while increasing precision and recall.
Our Immersive Virtual Colonoscopy Design
Our IVC design supports the same tasks as VC but with VR elements and metaphors. The system was developed on Unity, as it offers a favourable environment to integrate the colonoscopy procedure simulation with the VR equipment. The radiologist starts with a view from outside of the colon. They can rotate the colon for external inspection, point to the colon to teleport to the pointed position, or simply switch to the inside view. Once inside, they can move using the “fly-through over the centre line” mechanism, which allows doctors to navigate seamlessly through the colon by following its central path using the controllers. In addition to navigation, the interface incorporates measurement capabilities directly in 3D. Users can utilise measurement tools to assess various aspects of the colon, such as the diameter of polyps or the length of potential abnormalities. These measurements are recorded and stored, enabling radiologists to review and compare previous measurements during subsequent sessions. This feature proves particularly valuable for monitoring the progression of conditions or tracking changes over time.
Another noteworthy feature of virtual colonoscopy is the inclusion of a coverage visualisation using a heatmap, shown in Figure 1b, which depicts via a colour gradient (red for the not-seen areas and blue for the most-seen areas) to highlight areas of the colon that have yet to be thoroughly examined or remain unseen during the procedure. This feature assists users in identifying potentially overlooked regions, prompting them to redirect their attention to those areas and ensuring comprehensive coverage during the virtual colonoscopy examination.
Figure 1. Highlights of the immersive colonoscopy viewer. In (a), a view from inside the colon in VR shows a medial axis line in white. (b) colourmap view of eye-tracked data, indicating the areas already examined in blue and not yet seen in red. (c) CT data plane visible in situ, where the tissue density can be inspected to help detect disease.
Final Remarks
We designed a viable VR interface for the VC diagnostic procedure. We explored further some of the elements previously addressed in the literature, such as the navigation and measurement tools. Yet, we innovated with eye tracking to estimate the coverage and with an in situ CT slice viewer. Our preliminary findings suggest that these technologies enhance IVC’s effectiveness by providing valuable insights into the user’s decision-making process and improving the accuracy of lesion detection and diagnosis. This prompts new research to measure the impact of our techniques.
This work was partially supported by FCT - Portuguese National Science & Technology Foundation grant 2022.09212.PTDC (Xavier) under the auspices of the UNESCO Chair on AI & VR.
References:
[1] R.L. Siegel, et al., “Colorectal cancer statistics 2023,” CA Cancer J Clin., vol 73, no. 3, pp. 233–254, 2023. doi: https://doi.org/10.3322/caac.21772
[2] S. Mirhosseini, et al., “Immersive Virtual Colonoscopy,” in IEEE Transactions on Visualization and Computer Graphics, vol. 25, no. 5, pp. 2011–2021, May 2019. doi: https://doi.org/10.1109/TVCG.2019.2898763.
[3] J. Serras, et al., “Development of an Immersive Virtual Colonoscopy Viewer for Colon Growths Diagnosis”, in 2023 IEEE Conf. on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) (pp. 152-155), IEEE., 2023.
Please contact:
Joaquim Jorge, Técnico Lisboa, Portugal
