Task scheduling in a Reconfigurable Active SSD Distributed System -

Abstract

In massive computing applications that process large amounts of data, I-O operations consume a significant portion of the overall execution time. Such applications belong to the class of Big Data Analytics (BDA). High-end computational capabilities and active storage solutions focus on pushing as much of the computation as possible closer to where the data resides. This decreases the time taken to transfer the data from the stor-age to the computing nodes, thus reducing the overall time taken by I-O operations. However, other components of the computations, such as data aggregation and subse-quent processing of aggregated data, are frequently serial or exhibit low levels of paral-lelism, thus requiring these components to execute on high-performance general-purpose processing nodes. Although the resulting distributed computing systems required for executing BDA applications feature massive parallel processing of huge amount of data on specialized computing nodes, they also feature significant complexity in scheduling multiple application components with various computational requirements for optimal execution on heterogeneous computing nodes. The heterogeneity in the system is due to both having different types of computing nodes and using multiple types of connecting networks between the computing nodes. This paper considers cloud environments integrated with high-end active storage systems, and aims to develop tools and methods for optimal distribution of computational tasks of BDA applications in these heterogeneous computing systems. We evaluate mapping of big data applications repre-sented in Directed Acyclic Graphs (DAG) into the heterogeneous computational nodes. This mapping is done through an optimization algorithm that minimizes the overall exe-cution time and data communication delay. We compare the algorithm with the Genetic Algorithm (GA) and Heterogeneous Earliest Finish Time (HEFT) algorithms. We com-pare the performance between a standard system and the system with high-performance nodes by apply