Network Latency: how latency, bandwidth & packet drops impact your speed

Network Latency: how latency, bandwidth & packet drops impact your speed Build • Pavel Lunin • 21/07/22 • 14 min read

In this blog post, we attempt to demystify the topic of network latency. We take a look at the relationship between latency , bandwidth and packet drops and the impact of all this on "the speed of the internet". Next, we examine factors that contribute to latency including propagation , serialization , queuing and switching delays . Finally, we consider the more advanced topic of the relationship between latency and common bandwidth limitation techniques .

While the concept of bandwidth is easily understood by most IT engineers, latency has always been something of a mystery. At first glance, it might seem that it doesn't matter how long it takes for a chunk of data to cross the Atlantic ocean, as long as we have enough bandwidth between the source and destination. What's the big deal if our 100GE link is one kilometer or one thousand kilometers long, it's still 100GE isn't it?

This would be correct if we never experienced any packet loss in the network. Latency impact is all about flow and congestion control (which these days is mainly part of TCP). When one end sends a chunk of data to the other, it usually needs an acknowledgment that the data has been received by the other end. If no acknowledgment is received, the source end must resend the data. The greater the latency, the longer it takes to detect a data loss.

In fact, packet drops are part of the very nature of IP networks. Packets may be dropped even in a perfectly-operating network that has no issues with congestion or anything else.

Latency and the Unknown

We can regard network latency as a buffer for the data "in fly" between the source and the destination, where packets could possibly disappear. The greater the latency, the larger this buffer is, and the more data is in fly. But being in fly, the data is like Schrodinger's cat : neither sender nor receiver know if it's safely traveling towards its destination or already dropped somewhere along the way. The bigger the amount of data in this unknown state, the longer it takes to detect a loss event and recover from it. Consequently, the lower the effective bandwidth between the source and destination.

The maximum theoretical bandwidth that can be achieved is determined as follows:

As the R ound T rip T ime (RTT) can not be changed for a given source/destination pair, if we want to increase the available bandwidth we have to increase the amount of data in fly. Technically speaking, to achieve a given bandwidth over a route with a given RTT ( Delay ), the TCP transmission window (amount of data in fly) must be equal to the so-called B andwidth- D elay P roduct ( BDP ):

BDP_formula_resized.webp

So the bandwidth in packet networks comes at the cost of entropy (the amount of data in the unknown state being transported between the source and the destination) and is a linear function of network latency.

As noted above, packet drops are normal on the Internet due to the nature of packet networks. Modern, sophisticated congestion control schemes can increase the amount of data in fly (TCP transmission window) to large values quickly, in order to reach maximum bandwidth. However, when a drop is experienced, the time needed to detect that loss is still a function of latency.

Finally, all this translates into "the speed of the Internet". We can see the difference by downloading a 1GB file from a test server in Paris and then the same file in a version with 90 ms of additional simulated latency:

http://ping.online.net/1000Mo.dat

http://ping-90ms.online.net/1000Mo.dat

(You will need a good internet connection to carry out this test)

Latency User Experience Guidelines

The following guidelines can be used to roughly estimate the impact of latency:

RTT User Experience 150 ms "it works", but is not acceptable for most commercial applications these days

These numbers are subjective and may vary for different applications and user groups. They also depend on the user's habitual location. For example, people living in New Zealand or North-East Russia are somewhat used to higher latency for the majority of internet resources. They can tolerate "slower Internet", while US West Coast users who are more used to "fast internet" are often unhappy with the RTT to the European resources.

Where Latency Comes From

Propagation Delay

In modern networks, the primary source of latency is distance. This factor is also called propagation delay . The speed of light in a fiber is roughly 200,000 km per second, which gives us 5 ms per 1000 km single-direction and the mnemonic rule of 1 ms of round-trip time per 100 km.

However, fibers rarely follow as-the-crow-flies lines on the map, so the true distance is not always easy to estimate. While the routes of submarine cables are more or less straightforward, metro fiber paths in highly urbanized areas are anything but. And so, real-world RTT values can be roughly determined by coupling the above rule with the following considerations:

For multi-thousand kilometers long, transcontinental routes: as-the-crow-flies distance multiplied by a factor of 1.5

For shorter, terrestrial urban routes: as-the-crow-flies distance multiplied by a factor of 2.

See the examples below.

Paris—New York

5,800 km, 58×1.5 = 87 ms of RTT

par1 - instance$ PING 157.230 .229 .24 ( 157.230 .229 .24 ) 56 ( 84 ) bytes of data . 64 bytes from 157.230 .229 .24 : icmp_seq = 1 ttl = 52 time = 84.0 ms 64 bytes from 157.230 .229 .24 : icmp_seq = 2 ttl = 52 time = 83.9 ms 64 bytes from 157.230 .229 .24 : icmp_seq = 3 ttl = 52 time = 84.0 ms -- - 157.230 .229 .24 ping statistics -- - 3 packets transmitted , 3 received , 0 % packet loss , time 2002ms rtt min / avg / max / mdev = 83.992 / 84.015 / 84.040 / 0.237 ms CopyContentIcon Copy code Paris—Singapore

11,000 km, 110×1.5 = 165 ms

par1 - instance$ ping 188.166 .213 .141 - c 3 PING 188.166 .213 .141 ( 188.166 .213 .141 ) 56 ( 84 ) bytes of data . 64 bytes from 188.166 .213 .141 : icmp_seq = 1 ttl = 49 time = 158 ms 64 bytes from 188.166 .213 .141 : icmp_seq = 2 ttl = 49 time = 158 ms 64 bytes from 188.166 .213 .141 : icmp_seq = 3 ttl = 49 time = 158 ms -- - 188.166 .213 .141 ping statistics -- - 3 packets transmitted , 3 received , 0 % packet loss , time 2002ms rtt min / avg / max / mdev = 158.845 / 158.912 / 158.991 / 0.330 ms CopyContentIcon Copy code…