Fault-tolerant scheduling of graph-based loads on fog/cloud environments with multi-level queues and LSTM-based workload prediction

Workflow scheduling in the fog/cloud environment is a complex challenge, particularly with the increasing number of IoT devices. Many workflow applications have time constraints. The duplication approach is one of the methods that can help tasks to meet their deadline. This paper presents an architecture that uses a multi-criteria scheduling algorithm and multi-level queues. The proposed architecture determines a duplication coefficient for each queue, and tasks are placed in these queues using a triple-tuple approach composed of the tasks' priority, time pressure, and workload prediction of required resources. The proposed architecture uses various methods for calculating desired parameters. CPE is used to calculate the priority of tasks. Workload prediction is performed by Long-Short Term Memory (LSTM) neural network which is known for its forecasting capabilities. To evaluate the effectiveness of our proposed method, we conducted a software simulation study comparing it with four recent algorithms. The simulation results confirmed the superiority of our approach in different areas. The throughput is improved by 48.6%, communication cost was reduced by 56%. Also, make-span, waiting time, and the parallelism degree are improved between 15 to 62%.