An In-Depth Investigation into the Performance of State-of-the-Art Zero-Shot, Single-Shot, and Few-Shot Learning Approaches on an Out-of-Distribution Zero-Day Malware Attack Detection

N-shot learning has emerge in recent year as potential learning approach to solve the problem of data scarcity by learning underlying pattern from a few training sample. Despite recent state-of-the-art research on model-agnostic metal learning, transfer learning, and optimization strategy to rapidly learn valid information from few sample, there remains a big challenge on an actual out-of-distribution zero-day without any similarity to previously known malware family or new variant of an existing malware family. This ultimately questions the effectiveness of current state-of-the-art few-shot learning approach. In this research, we did an in-depth investigation into the performance of state-of-the-art Zero-shot, Single-shot, and few-shot learning approaches on zero-day out-of-distribution malware attack detection based on their static properties using Malimg and Malevis malware dataset. We ensure our model was aware of an out-of-distribution class during training while varying the number of samples in the out-of-distribution class accordingly zero-shot(no sample), single-shot (1 sample), few-shot(5 samples) while using confusion matrix to get the actual number of correct prediction on out-of-distribution malware validation samples. we assert that the model should be smart enough to detect and classify previously unseen data into an empty family as an out-of-distribution considering that the model was made to be aware of the existence of such distribution during training. Result shows 0, 0, and 3 correct out-of-distribution predictions on Zero-shot, single-shot, and few-shot experiments respectively, thereby showing limitation of the current state-of-the-art N-shot approaches on out-of-distribution attack.