Errors and Audio/Video Transport
The way that bit errors occur in the creation or transmission of packets and how they affect the transport of media is a broadly misunderstood concept related to IP networks. This is probably because media originally was transmitted in an analog format or as a non-packetized digital stream. We won’t examine those formats. Rather, let’s investigate how errors impact IP transport of audio and video.
Sources of Errors
Errored packets generally result from one of three causes. Either there is a physical signaling error, a queue (buffer) discard, or corruption caused by a device.
In the first instance, a wired or wireless signal that represents one or more bits might be distorted. The more complex the waveform that is being used, the greater the number of bits that might be misinterpreted by the receiver. For example, if a QAM-64 signal becomes malformed, it could affect up to six bits.
Buffers or queues are used throughout the send-receive process. The application queues a message as it is sent to either the TCP or the UDP function. The Ethernet network interface driver queues the packet until the hardware is able to store and forward the packet. And, every modern network switching or routing device has buffers for input and for output as they decide how to manage and relay packets. Any of these buffers can be overfilled. When that happens, the default policy is not to expand the buffer. Rather, it is to drop the data that caused the buffer to overfill. This might seem like a bad idea. However the assumption by devices and software operating at such a low level, is that some other system component will detect the loss and mitigate the mistake.
In the third case, devices can make errors in how they write packets. They may not be programmed correctly or not be updates with the latest software.
Error check codes are often called checksums. So, we’ll use that term even though it originally referred to a specific technique.
The idea is this: the sender computes a code which is derived from the sequence of bits that are to be sent. The code is attached to the message. The receiver computes the code again using the same algorithm. If the code is different from the one which was transmitted, the receiver knows that something in the message was corrupted. Most modern checksums are very reliable. Ethernet does an error check called the Frame Check Sequence (FCS). It’s the last field in every Ethernet frame. IP, UDP, and TCP also run separate error checks.
When bits are corrupted and the frames they are in are dropped, the impact can vary considerably. The main factor determining what the effect will be is which layer four protocol is being used. For example, if video is transported using UDP, such as in video conferencing, IPTV, or digital signage, the result is usually tilling. We see this when small blocks or strips on the screen appear black, the wrong color, or are otherwise distorted. These derive from missing packets which would have allowed the receiver to rebuild those blocks correctly. On the other hand, if TCP is being used as the L4 protocol, a completely different result can occur. The result of packet loss will be increased delay in the delivery of all the remaining packets. If the receiver play out buffer doesn’t empty, it may have no affect at all. Otherwise, if the play out buffer empties, there will be a pause in the video. Adaptive Bit Rate (ABR) video, such as Apple HLS, Microsoft Smooth Streaming or DASH, use TCP.
Mitigating errors can also be differentiated by the L4 protocol in use. With UDP, there generally isn’t time for a retransmission. So, the receiver must try to compensate. Some receiving software or encoders will try estimating what the missing data should have been. For example, if a block is distorted in the middle of a blue sky, the receiver can estimate that the block should look the same as the surrounding video blocks. Sometimes forward error correction (FEC) can be used. Somewhat like a pre-emptive error check, the sender calculates a correction code and attaches it to the data that is sent. The receiver uses this code to repair the corrupted bits. FEC is used in many implementations of video conferencing. It is also often used in audio. In addition, it’s a standard feature in SRT (Secure Reliable Transport), which was introduced recently by Wowza and Haivision.
Obviously, errors in network transmissions are not a good thing. But, techniques of mitigating those errors have become so robust that we consider transmission over the Internet a routine matter. Twenty five years ago, we would have thought this impossible.