GeePS: Scalable deep learning on distributed GPUs with a GPU-specialized parameter server

http://www.pdl.cmu.edu/PDL-FTP/CloudComputing/GeePS-cui-eurosys16.pdf Deep learning workloads are highly parallelizable. However, distributing deep neural network on GPUs can be inefficient due to data movement overheads, GPU stalls, limited GPU memory etc. This paper describes GeePS, a parameter server system specialized for...