Data parallelism: independent evaluation of different pieces of data Task parallelism: decomposition of independent tasks
Data parallelism is the main source of scalability in parallel programming, but task parallelism can also helps.
Examples of Data Parallelism
- GPU kernels
- parallel for loop, parallel STL algorithms
Examples of Task Parallelism
- threads, thread pools, actors, executors
- CUDA stream and CUDA task graph