Online Large-Scale Hypothesis Tesng with Corrupted Data

Abstract

Modern statistical inference involves processing extensive datasets, with multiple hypothesis testing being one methodology to draw conclusions on many features at once. Control of the false discovery rate (FDR) is essential. Target classification via satellite imagery and acoustic signal processing are examples in military applications where false detections can be costly for a Command and Control framework. These contexts also showcase another layer of complexity: the volume of data is often processed online, with decisions having to be made sequentially on evolving, incomplete datasets. This underscores the need for FDR control in an online environment. Current methods for online FDR control are successful in this regard; however, they are not designed with data error or, worse, data corruption in mind. This research will explore the level of robustness of the Levels Based On Recent Discovery (LORD) algorithm. The fundamental objective is to learn how to corrupt data and make it robust against such corruption efficiently. This work will draw insights from studying corruption-robust bandit algorithms and aim to advance the adversarial online multiple-hypothesis testing field.