Key additional metrics for evaluating Artificial Intelligence enabled Computer Aided Detection system for Colon Polyps

The traditional metrics for artificial intelligence (AI) enabled computer aided detection (CAD) systems for colorectal polyps are sensitivity and specificity and have been studied in various publications before. Those publications have not focused as much on the time and speed factor though i.e., early and quick detection of polyps in colonoscopy videos. While it is desirable for AI enabled CAD system for polyps to accurately localize polyps present, we believe that it is also highly critical to measure how “quickly” and how “early” the first instance of the polyp is detected by the system. This will insure that real-time feedback can be provided in a useable manner to the physician.

We propose 2 metrics Tq and Te to address this. Tq measures how long it takes the CAD system to detect the polyp. Te measures how much earlier the CAD system detects relative to a physician with larger negative numbers indicating a faster CAD system or a slower physician. Tq= td-ta where ta is the arrival frame of the polyp and td is the frame in which the polyp is detected by CAD system. Te = td-tp where tp is the frame in which the polyp is first detected by physician. We believe, designing a CAD algorithm based on Tq ,Te measures, in addition to the traditional performance metrics will result in a more useful product for the physicians. Our deep learning based CAD algorithm is designed in this manner and trained on 26000 images extracted from procedure recordings.

In a recently concluded retrospective study performed with assistance of gastroenterologists with varying experience, we measured Te and Tq for our algorithm on 25 video clips with hard to see polyps. Our CAD algorithm is capable of reliably detecting polyps quickly and earlier A detailed analysis also revealed that our CAD system detected much earlier when physicians missed brief visibility (< 500ms) of the polyps.

Can you detect faster than our CAD system? Try it out on the clips below.

Please contact us at info@endovigilant.com to learn more.