Blog

0

InfiniBand

http://www.mellanox.com/pdf/whitepapers/Intro_to_IB_for_End_Users.pdf https://www.mellanox.com/pdf/whitepapers/IB_Intro_WP_190.pdf This is an overview of InfiniBand architecture that I gathered from the IB white paper and user manual. InfiniBand (IB) is an alternative proposal for the widely popular TCP/IP/Ethernet networking stack. IB mainly targets High Performance...

2

Beyond Data and Model Parallelism for Deep Neural Networks

https://arxiv.org/abs/1807.05358 Existing deep learning frameworks either use data or model parallelism as distribution strategy. However, these strategies often produce sub-optimal performance for many models. In data parallel distribution, the computation graph is partitioned on the data sample dimension....