RIVOX: Real-Time Crowd Behavior and Violence Detection Using Deep Learning

Bridging the gap between high-accuracy deep learning and the physical limits of surveillance hardware remains a significant hurdle for real-world security deployments. Most existing threat detection models are developed in isolation, focusing on a single category like violence or weapon spotting, but combining these into a unified "multi-threat" system often leads to severe CPU-GPU resource contention and bottlenecks. In this paper, we introduce RIVOX, a deployment-oriented framework designed to detect violence, weapons, and fire in real-time without requiring dedicated server infrastructure. The core of our system is a dual-mode architecture that prevents hardware "resource contention". For the behavior-based mode, we implemented a hybrid two-stage pipeline where a lightweight motion-based prefilter (Stage 1) acts as a gatekeeper, effectively screening out non-suspicious frames. This logic allowed us to bypass 48.5% of unnecessary deep inference calls during testing, significantly reducing the thermal load on the processor. Frames that trigger the filter are then processed by a deep spatio-temporal network combining MobileNetV2 and Bidirectional LSTMs (BiLSTM) for high-confidence classification. For object-based threats, RIVOX utilizes GPU-accelerated YOLOv8 models optimized with a 1:5 frame-skipping strategy to maintain a stable duty cycle helping maintain thermal stability. Evaluated on a mobile workstation featuring a Ryzen 9 CPU and an RTX 4060 GPU, RIVOX maintains a consistent 16 FPS pipeline, proving that resource-aware engineering can make advanced, multi-threat surveillance viable on portable edge hardware.