A preliminary study of incorporating GPUs in the Hadoop framework

A preliminary study of incorporating GPUs in the Hadoop framework

Abstract—Fine-grained parallel processors can be employed asaccelerators in MapReduce clusters to improve the completiontime of MapReduce jobs or to substantially reduce the size of theclusters required to achieve a desired parallel speedup. However,significant architectural differences between conventional CPUsand accelerators pose new challenges for effective scheduling ofMapReduce tasks on individual cluster nodes. In this paper, wepresent a Hadoop-based framework that allows employing bothCPUs and GPUs for MapReduce-type applications. We base anovel framework, called Surena, on existing work that allowswriting MapReduce applications for GPUs and they incorporateit in the overall Hadoop framework. In particular, we showthat by using simple scheduling optimizations, Surena can fullyutilize GPUs during the map phase of MapReduce jobs whichis often the dominant component in the total execution timeof MapReduce applications. Our performance results showsspeedups of up to 21x for our framework compare to Hadoop.