Event-Triggered Communication in High Performance Computing

High Performance Computing has enabled breakthroughs in computational science and engineering by allowing simulations on parallel computing platforms. One of the major challenges in developing parallel algorithms for these platforms is that the arithmetic performance, measured in terms of the number of floating point operations per second (flops), is typically better than the communication performance, measured in terms of how fast data can be communicated between processors. As a result, the performance of these parallel algorithms is often limited by the communication between processors. This dissertation aims to reduce the overhead of communication by developing algorithms that communicate only when necessary. In particular, we apply the concept of event-triggered communication from networked control systems to design faster, energy-efficient and scalable parallel algorithms for high performance computing. We analyze and implement parallel numerical solvers with event-triggered communication for a class of partial differential equations. We also study and implement event-triggered communication for distributed training of artificial neural networks. We show that such communication strategies can reduce the number of messages to be communicated while converging to the correct solution. Reducing the number of messages is expected to decrease the time and energy associated with communication as well as alleviate congestion in interconnects. The developed algorithms can be useful for future exascale computing platforms.