On-the-fly Raman microscopy guaranteeing the accuracy of diagnosis by reinforcement learning

We present our recent study combined multi-armed bandits algorithm in reinforcement learning with spontaneous Raman measurements for the acceleration of the measurements by designing and generating optimal illumination pattern “on the fly” during the measurements while keeping the accuracy of the diagnosis. Here accurate diagnosis means that a user can determine an allowance error rate δ a priori to ensure that the diagnosis can be accurately accomplished with probability greater than (1 −δ)×100%. We present our algorithm and our simulation studies using Raman images in the diagnosis of follicular thyroid carcinoma, and show that this protocol can accelerate in speedy and accurate diagnoses faster than the point scanning Raman microscopy that requires the full detailed scanning over all pixels. The on-the-fly Raman image microscopy is the first Raman microscope design to accelerate measurements by combining one of reinforcement learning techniques, multi-armed bandit algorithm utilized in the Monte Carlo tree search in alpha-GO. Given a descriptor based on Raman signals to quantify the degree of the predefined quantity to be evaluated, e.g., the degree of cancers, anomaly or defects of materials, the on-the-fly Raman image microscopy evaluates the upper and lower confidence bounds in addition to the sample average of that quantity based on finite point illuminations, and then the bandit algorithm feedbacks the desired illumination pattern to accelerate the detection of the anomaly, during the measurement to the microscope. The realization of the programmable illumination microscope using a spatial light modulator will be presented.