Autoplay Next video from Max length

Training Distributed Deep Recurrent Neural Networks with Mixed Precision on GPU Clusters

651 views · Published 25 September 2018 · 30:59 · Indexed 8 November 2026

Channel: Databricks · 2018 · Science & Technology

Alexey Svyatkovskiy is a Data Scientist at Microsoft. In this talk, we evaluate training of deep recurrent neural networks with half-precision floats on Pascal and Volta GPUs. We implement a distributed, data-parallel, synchronous training algorithm by integrating TensorFlow and CUDA-aware MPI to enable execution across multiple GPU nodes and making use of high-speed interconnects. We introduce a learning rate schedule facilitating neural network convergence at up to O(100) workers.

To learn more: https://databricks.com/product/getting-started-guide
About: Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.
Read more here: https://databricks.com/product/unified-data-analytics-platform

Connect with us:
Website: https://databricks.com
Facebook: https://www.facebook.com/databricksinc
Twitter: https://twitter.com/databricks
LinkedIn: https://www.linkedin.com/company/databricks
Instagram: https://www.instagram.com/databricksinc/ Databricks is proud to announce that Gartner has named us a Leader in both the 2021 Magic Quadrant for Cloud Database Management Systems and the 2021 Magic Quadrant for Data Science and Machine Learning Platforms. Download the reports here. https://databricks.com/databricks-named-leader-by-gartner

Training Distributed Deep Recurrent Neural Networks with Mixed Precision on GPU Clusters

More from this channel