Automated sizing of colorectal polyps using computer vision

Colorectal polyp size is an important biomarker that influences management decisions, but currently used subjective methods are flawed. We explored two computer vision (CV) techniques for binary classification of polyp size as either ≤5 mm or >5 mm. First, we used premeasured phantom polyps (22 such polyps’ videos) fixed on a pig colon model to explore the concept of automated sizing using structure from motion (SfM) approach and compared it with the sizing by 10 independent endoscopists: overall, average diagnostic accuracy of the SfM system (85.2%) was superior to endoscopists judgement (59.5%). Second, we developed a deep learning model based on convolutional neural networks (CNN) and found 80% accuracy in 10 videos of human polyps. Real-time automated polyp sizing when combined with artificial intelligence (AI) assissted polyp characterisation could improve polyp management strategies. ### CV techniques CV can be defined as the ability of machines to process and understand visual data, automating the type of tasks the human eye would normally be required to do. In order to perform automated polyp size classification, we employed two types of CV techniques, SfM and deep learning (DL). SfM is a photogrammetric imaging technique that algorithmically recovers three-dimensional (3D) structure of an object from multiple two-dimensional (2D) images and is commonly used in topographic studies. SfM finds matching points in input images and recovers the 3D structure by solving the epipolar constraint equation derived from these matching points, as briefly illustrated in figure 1. The algorithm calculates a camera’s pose as a rotation matrix and a translation vector using matching points. Finally, we apply mathematical formulas to compute the distance between the polyp and endoscope, and that distance is used to compute polyp size in real time. Compared with DL, this SfM technique uses less data making it relatively easier and quicker to convert into a clinical …