From left: Sambit Das, Phani Motamarri and Vikram Gavini
A team led by Prof. Vikram Gavini (Professor of Mechanical Engineering and MICDE affiliate) and including Dr. Sambit Das (MICDE Fellow) and Dr. Phani Motamarri (Assistant Research Scientist and MICDE affiliate), is one of two finalists nominated for this year’s Gordon Bell Prize. The award, generally considered to be the highest honor of its kind, worldwide, recognizes outstanding achievement in high-performance computing. Gavini’s team has developed a methodology that combines advanced finite-element discretization methods for Density Functional Theory (DFT)1 with efficient computational methodologies and mixed precision strategies to achieve a 46 Peta-FLOPS2 sustained performance on 3,800 GPU nodes of the Summit supercomputer. Their work titled “Fast, scalable and accurate finite-element based ab initio calculations using mixed precision computing: 46 PFLOPS simulation of a metallic dislocation3 system” also involved Dr. Bruno Turcksin and Dr. Ying Wai Li from Oak Ridge National Laboratory, and Los Alamos National Laboratory, and Mr. Brent Leback from NVIDIA Corporation.
Electron density contour of pyramidal II screw dislocation system in Mg with 61,640 electrons (6,164 Mg atoms).
First principle calculation methods4 have been immensely successful in predicting a variety of material properties. These calculations are prohibitively expensive as the computational complexity scales with the number of electrons in the system. Prof. Gavini’s research work is focussed on developing fast and accurate algorithms for Kohn-Sham5 density functional theory, a workhorse of first principle approaches that occupies a significant fraction of the world’s supercomputing resources. In the current work, Dr. Das, Dr. Motamarri and Prof. Gavini used recent developments in the computational framework for real-space DFT calculations using higher-order adaptive finite elements, and pioneered algorithmic advances in the solution of the governing equations, along with a clever parallel implementation that reduced the data access costs and communication bottlenecks. This resulted in fast, accurate and scalable large-scale DFT calculations that are an order of magnitude faster than existing widely used DFT codes. They demonstrated an unprecedented sustained performance of 46 Peta-FLOPS on a dislocation system containing ~100,000 electrons, which is the subject of the Gordon Bell nomination.
Past winners of the Gordon Bell Prize have typically been large teams working on grand challenge problems in astrophysics, climate science, natural hazard modeling, quantum physics, materials science and public health. The purpose of the award is to track the progress over time of parallel computing, with particular emphasis on rewarding innovation in applying high-performance computing to applications in science, engineering, and large-scale data analytics. If you are attending the SuperComputing’19 conference this year in Denver, you can learn more about Dr. Das, Dr. Motamarri and Dr. Gavini’s achievement at the Gordon Bell Prize finalists’ presentations on Wednesday, November 20, 2019, at 4:15 pm in rooms 205-207.
Related Publication: S. Das, P. Motamarri, V. Gavini, B. Turcksin, Y. W. Li, and B. Leback. “Fast, Scalable and Accurate Finite-Element Based Ab initio Calculations Using Mixed Precision Computing: 46 PFLOPS Simulation of a Metallic Dislocation System.” To appear in SC’19 Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, Denver, CO, November 17–22, 2019.
[1] Density functional theory (DFT) is a computational quantum mechanical modeling method used in physics, chemistry and materials science to investigate the electronic structure (or nuclear structure) (principally the ground state) of many-body systems, in particular atoms, molecules, and the condensed phases. https://en.wikipedia.org/wiki/Density_functional_theory.
[2] A PETAFLOP is a unit of computing speed equal to one thousand million million (1015) floating-point operations per second.
[3] In materials science, dislocations are line defects that exist in crystalline solids.
[4] First principle calculation methods use the principle of quantum mechanics to compute properties directly from basic physical quantities such as, e.g., mass and charge.
[5] W. Kohn, L. J. Sham, Self-consistent equations including exchange and correlation effects, Phys. Rev. 140(4A) (1965) A1133.