Skip to content

Commit a4cf6da

Browse files
authored
Merge pull request #155 from UCL/mm_updates2024
Fix bandwidth expression
2 parents 1f9942c + 8ee6f29 commit a4cf6da

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

09distributed_computing/sec01DistributedMemoryModels.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,7 @@ Message passing naturally incurs a performance overhead. Data communication chan
9595
- The frequency of message passing should be kept down where possible.
9696
- The size of messages should be kept down where possible.
9797
- In general, a smaller number of large messages is better than a large number of small messages _for a given amount of data_.
98-
- This is true in general of data movement, whether through message passing or memory reads to RAM or hard disks. Loosely speaking, data movement general involves a latency ($L$) and bandwidth ($B$), such that the time for $N$ bytes of data to be transferred is $\sim BN + L$. If we send this data in $k$ separate messages, we will incur a $kL$ latency penalty instead of just $L$.
98+
- This is true in general of data movement, whether through message passing or memory reads to RAM or hard disks. Loosely speaking, data movement general involves a latency ($L$) and bandwidth ($B$), such that the time for $N$ bytes of data to be transferred is $\sim N/B + L$. If we send this data in $k$ separate messages, we will incur a $kL$ latency penalty instead of just $L$.
9999
- If you have to choose between sending a smaller amount of total data in a larger number of messages, or a larger amount of data using a smaller number of messages, then which you should pick will depend on which term in this expression becomes dominant!
100100

101101

0 commit comments

Comments
 (0)