๐”– Scriptorium
โœฆ   LIBER   โœฆ

๐Ÿ“

Parallel C++: Efficient and Scalable High-Performance Parallel Programming Using HPX

โœ Scribed by Patrick Diehl, Steven R. Brandt, Hartmut Kaiser


Publisher
Springer
Year
2024
Tongue
English
Leaves
233
Edition
1
Category
Library

โฌ‡  Acquire This Volume

No coin nor oath required. For personal study only.

โœฆ Synopsis


This textbook focuses on practical parallel C++ programming at the graduate student level. In particular, it shows the APIs and related language features in the C++ 17 and C++ 20 standards, covering both single node and distributed systems. It shows that with the parallel features in the C++ 17 and C++ 20 standards, learning meta-languages like OpenMP is no longer necessary. Using the C++ standard library for parallelism and concurrency (HPX), the same language features can be extended to distributed codes, providing a higher-level C++ interface to distributed programming than the Message Passing Interface (MPI).

The book starts with the single-threaded implementation of the fractal sets, e.g. Julia set, and Mandelbrot set, using the C++ Standard Library (SL)โ€™s container and algorithms. This code base is used for parallel implementation using low-level threads, asynchronous programming, parallel algorithms, and coroutines. The asynchronous programming examples are then extended to distributed programming using the C++ standard library for parallelism and concurrency (HPX). Octo-Tiger, an astrophysics code for stellar merger, is used as a showcase for a portable, efficient, and scalable high-performance application using HPX.

The bookโ€™s core audience is advanced undergraduate and graduate students who want to learn the basics of parallel and distributed C++ programming but are not computer science majors. Basic C++ knowledge, like functions, classes, loops, and conditional statements, is assumed as a requirement, while C++ advanced topics, like generic programming, lambda functions, smart pointers, and move semantics, are briefly summarized in the appendix.


โœฆ Table of Contents


Foreword
Preface
Acknowledgments
Contents
Acronyms
Part I Preliminaries
1 Compiling and Running the Code and Examples in This Book
1.1 Using the C++ Explorer
1.2 Using CMake and C++ Compiler
Part II Introduction to C++ and C++ Standard Library
2 About C++, C++ Standard, and the C++ Standard Library
2.1 Brief History of C++, the C++ Standard, and Parallel Programming
2.2 Standard Template Library (STL) and C++ Standard Library
2.3 C++ Compilers
3 C++ Standard Library
3.1 Overview of the C++ Standard Library
3.2 Containers
3.2.1 Vector
3.2.2 List
3.2.3 List vs. Vector
3.2.4 Array
3.2.5 Iterators
3.3 Algorithms
4 Example Mandelbrot Set and Julia Set
4.1 Mandelbrot Set
4.2 Julia Set
4.3 Single Threaded Implementation of the Mandelbrot Set
Part III The C++ Standard Library for Concurrency and Parallelism (HPX)
5 Why HPX?
5.1 Governing Principles
5.1.1 Focus on Latency Hiding Insteadof Latency Avoidance
5.1.2 Embrace Fine-Grained Parallelism Instead of Heavyweight Threads
5.1.3 Rediscover Constraint-Based Synchronization to Replace Global Barriers
5.1.4 Adaptive Locality Control Instead of Static Data Distribution
5.1.5 Prefer Moving Work to the Data Over Moving Data to the Work
5.1.6 Favor Message Driven Computation Over Message Passing
6 The C++ Standard Library for Parallelismand Concurrency (HPX)
6.1 HPX's Architecture
6.2 Applications
Part IV Parallel Programming
7 Parallel Programming
7.1 An Overview of Parallel Programming
7.2 Race Conditions
7.2.1 Mutexes and Deadlocks
7.2.2 Atomic Operation
7.3 Performance Measurements
7.3.1 Amdahl's Law
7.3.2 Gustafson's Law
7.3.3 Speedup and Parallel Efficiency
7.3.4 Weak Scaling and Strong Scaling
7.4 Memory Access
7.5 Parallelism Computer Architectures
7.5.1 Pipelined SIMD
7.5.2 Single Instruction Multiple Data (SIMD)
8 Programming with Low Level Threads
8.1 Implementation of the Fractal Sets
9 Asynchronous Programming
9.1 Advanced Synchronization in HPX
9.2 Implementation of the Fractal Sets
10 Parallel Algorithms
10.1 Parallel Algorithms in HPX
10.1.1 Combining Parallel Algorithms and Asynchronous Programming
10.1.2 Single Instruction Multiple Data
10.2 Additional Parameters for the Execution Policies
10.3 Implementation of the Fractal Sets
11 Coroutines
11.1 Implementation of the Fractal Sets
12 Benchmarking the Fractal Set Codes
Part V Distributed Programming
13 Distributed Computing and Programming
13.1 Overview of Distributed Programming in C++ and Asynchronous Many Task Systems
13.2 Data Distribution
13.3 Distributed Input and Output
13.4 Serialization
13.5 Message Passing
13.5.1 Implementation of the Fractal Set Using MPI
13.5.2 Implementation of the Fractal SetUsing MPI+OpenMP
13.6 Benchmark of MPI and MPI+OpenMP
14 Distributed Programming Using HPX
14.1 Active Messaging
14.1.1 Plain Action
14.1.2 Components and Actions
14.1.2.1 Components and Component Actions
14.1.2.2 Client
14.1.2.3 Using Components and Components Actions
14.1.2.4 Recap
14.1.3 Receiving Topology Information
14.2 Serialization
15 Examples of Distributed Programming
15.1 Distributed Implementation of the Taylor Series of the Natural Logarithm
15.2 Distributed Implementation of the Fractal Set
15.3 Improved Distributed Implementation of the Fractal Set
15.4 Benchmark of the Distributed Implementation of the Fractal Sets
16 Some Remarks on MPI+OpenMP and HPX
Part VI A Showcase for a Portable High Performance Application Using HPX
17 Accelerator Cards
18 Octo-Tiger, a Showcase for a Portable High Performance Application
18.1 Synchronous Communication vs. AsynchronousCommunication
18.2 Acceleration Support
18.3 HPX-Kokkos and Vectorization
Part VII Conclusion and Outlook
19 Conclusion and Outlook
19.1 Distributed Programming
19.2 Outlook
Appendix
A Advanced Topics in C++
A.1 Generic Programming
A.2 Lambda Functions
A.3 Move Semantics
A.4 Placement New
A.5 Smart Pointers
A.6 Ranges
B Supplementary Header Files
C Software and Hardware Documentation
References
Glossary
Index

โœฆ Subjects


Parallel Programming; High-Performance Computing; Scientific Computing; Distributed Programming; C++; C++ Programming; Asynchronous Programming; HPX Library; Parallel Algorithms; Octo-Tiger; Mandelbrot Sets


๐Ÿ“œ SIMILAR VOLUMES


Parallel C++: Efficient and Scalable Hig
โœ Patrick Diehl, Steven R. Brandt, Hartmut Kaiser ๐Ÿ“‚ Library ๐Ÿ“… 2024 ๐Ÿ› Springer ๐ŸŒ English

<p><span>This textbook focuses on practical parallel C++ programming at the graduate student level. In particular, it shows the APIs and related language features in the C++ 17 and C++ 20 standards, covering both single node and distributed systems. It shows that with the parallel features in the C+

Parallel programming using C++
โœ Gregory V. Wilson, Paul Lu ๐Ÿ“‚ Library ๐Ÿ“… 1996 ๐Ÿ› MIT Press ๐ŸŒ English

Foreword by Bjarne Stroustrup Software is generally acknowledged to be the single greatest obstacle preventing mainstream adoption of massively-parallel computing. While sequential applications are routinely ported to platforms ranging from PCs to mainframes, most parallel programs only ever run

Parallel and High Performance Programmin
โœ Fabio Nelli ๐Ÿ“‚ Library ๐ŸŒ English

<p><span>Unleash the capabilities of Python and its libraries for solving high performance computational problems.</span></p><p><span>Key Features </span></p><p><span>โ— Explores parallel programming concepts and techniques for high-performance computing.</span></p><p><span>โ— Covers parallel algorith

Parallel and Distributed Programming Usi
โœ Cameron Hughes, Tracey Hughes ๐Ÿ“‚ Library ๐Ÿ“… 2003 ๐Ÿ› Addison-Wesley Professional ๐ŸŒ English

Intended for programmers familiar with C++, this book explains how the C++ standard library, algorithms, and container classes behave in distributed and parallel environments, and offers methods for extending the C++ language through class libraries and function libraries to accomplish distributed a

Python parallel programming cookbook: ma
โœ Zaccone, Giancarlo ๐Ÿ“‚ Library ๐Ÿ“… 2015 ๐Ÿ› Packt Publishing Limited ๐ŸŒ English

Master efficient parallel programming to build powerful applications using Python About This Book * Design and implement efficient parallel software * Master new programming techniques to address and solve complex programming problems * Explore the world of parallel programming with this book, which

Python parallel programming cookbook: ma
โœ Zaccone, Giancarlo ๐Ÿ“‚ Library ๐Ÿ“… 2015 ๐Ÿ› Packt Publishing Limited ๐ŸŒ English

Master efficient parallel programming to build powerful applications using Python About This Book * Design and implement efficient parallel software * Master new programming techniques to address and solve complex programming problems * Explore the world of parallel programming with this book, which