FISP1 ECE 7750 Distributed Control Systems FISP1 Topic: Network Delay Mitigation Prepared by: Nathan Erni Dept. of Electrical and Computer Engineering.

ECE 7750 Distributed Control SystemsFISP1

Topic: Network Delay Mitigation

Prepared by: Nathan Erni

Dept. of Electrical and Computer Engineering Utah State University

E: [email protected]; T: (435)232-7526;

2/18/2010

Overview• Problem Statement• UDP vs. TCP• Things to Consider• UDP

– What causes Delay• TCP

– How are packets lost– What causes Delay

• Measurement Resources• Summary• Further Work• References• Questions

Problem Statement

Network Delay

UDP vs. TCP• User Datagram Protocol (UDP)

– Connectionless• Header information is reduced, thus packet size is lighter• Reduction in processing time

– Assumes packet arrival. Does not rebroadcast– Mainly used in short communication links (LAN)– Latency can be as low as packet Round Trip Time (RTT)

• Best type of protocol to reduce latency. Most common are Real Time Protocol (RTP) and Real Time Control Protocol (RTCP) run under the UDP protocol.

– Can have Missing packets, duplicate packets, or packet in wrong order• Transmission Control Protocol (TCP)

– Connection-Oriented • 3-way communication before data transfer

– Rebroadcasts packets if lost• Insures that packets arrive• Processing time increased

– Keeps track of order– Mainly used in long distance communication links (WAN)

Things to Consider

• Understand your system that you want to control and pick what protocol works best for you.– Where is your controller located compared to the

plant?– How will your system respond to packet loss and

delay?– How much bandwidth does your controller have

on the network? Is it shared?

User Datagram Protocol

• What Causes Delay?– Hub• All network data it receives on one port will be

immediately transmitted out all the other ports, so each computer must take it's turn before sending data.

– Switch• Acts like a smart hub. All network data it receives will

be directed to only the destination port. This frees up bandwidth for all other lines.

Two LANS Joined by a SubnetApplication

Presentation

Session

Transport

Network

Datalink

Physical

Network

Datalink

Physical

Network

Datalink

Physical

Network

Datalink

Physical

Network

Datalink

Physical

Application

Presentation

Session

Transport

Network

Datalink

Physical

LAN 1 LAN 2Transmission Line

Router 1 Router 2

~http://www.neng.usu.edu/classes/ece/6600/slides.html

Transmission Control Protocol

• How are Packets Lost?– Data Collision• Two packets are sent on the same line

– Buffer Overflow• Packets are transmitted too fast for the receiving end

– Noise• Noise corrupts data causing packet to be disregarded

Transmission Control Protocol

• How to Avoid Packets Lost?

– Data Collision• No avoidance

– Buffer Overflow• Receiver’s buffer size can be increased or sender’s

buffer size can be decreased

– Noise• No avoidance

(assuming no access to physical layer)

Transmission Control Protocol

• Sending and Resending Packets– Window Size• Allocation of how many packets can be sent during a

time duration

– Sending Protocol• Stop and Wait• Go Back N• Selective Repeat

Transmission Control Protocol

• Window Size

Sender

Receiver

0123

1234

2345

3456

4567

5678

0123

1234

2345

3456

4567

5678

Sender’s Window

Receiver’s Window~http://www.neng.usu.edu/classes/ece/6600/slides.html

Transmission Control Protocol

• Stop and Wait

• Go Back N

• Selective Repeat

Time

Station 1

Station 2

~http://www.neng.usu.edu/classes/ece/6600/slides.html

Transmission Control Protocol

• What Causes Delay?– Address Resolution Protocol (ARP) Cache Timeout• No association with MAC address and IP address of

receiver

– Unused Bandwidth• Free bandwidth on the line

– Buffer Overflow• Packets are transmitted too fast for the receiving end

Transmission Control Protocol

• How to Mitigate Delay?– Address Resolution Protocol (ARP) Cache Timeout• ARP associates the receivers MAC address with the

current IP address. If this address is not know a broadcast packet is sent out requesting the receivers information. The association only stays in ARP cache for 4 seconds. Each time a packet is sent the ARP cache is updated, thus transmission delay can me mitigated if packets are sent less than every 4 seconds. This elevates the need for sending ARP packets.• To view ARP cache type “arp –a” in command window

Transmission Control Protocol

• How to Mitigate Delay?– Unused Bandwidth• Find the Narrow Link and push bandwidth to that limit.

– Example: Consider a narrow link of 10 Mbps and a Round Trip Time (RTT) of 50 ms. The in flight data should be (10 Mbps) * (0.025 ms) = 250 Kbits.

~http://fasterdata.es.net/TCP-tuning/TCP-Tuning-Tutorial.pdf

Transmission Control Protocol

• How to Mitigate Delay?– Buffer Overflow

• Maximize Buffer Size– Default TCP send/receive buffers are 64 KB, and default

maximum size is 256 KB. This is 10 to 150 times too small– Good buffer size

» Buffer size = Bandwidth*RTT– Example: Consider a narrow link of 500 Mbps (62 Mbytes/sec)

and a Round Trip Time (RTT) of 50 ms. The buffer size should be (62 Mbytes/sec) * (0.05 seconds) = 3.1 Mbytes

• Caution: Do not set the buffer size too big or too many packets will be sent at once and data will be lost

Transmission Control Protocol

• Comparison

~http://fasterdata.es.net/TCP-tuning/TCP-Tuning-Tutorial.pdf

Transmission Control Protocol

• One more thought. Parallel Streaming

~http://fasterdata.es.net/TCP-tuning/TCP-Tuning-Tutorial.pdf

Measurement Resources• Tracert – measures delay to each node

– Command Window “tracert google.com”• Iperf – measures Interval, Transfer, and Bandwidth

– http://dast.nlanr.net/Projects/Iperf/• Pathrate – measures the capacity of the narrow link

– http://www.pathrate.org• Pathload – measures the available bandwidth

– http://www.pathrate.org• Pipechar – measures hop-by-hop available bandwidth

– http://dsd.lbl.gov/NCS/• Tcpdump – TCP header information for source/destination address

– ftp://ftp.ee.lbl.gov/• Tcptrace – plots output analysis

– http://www.ncne.nlanr.net?TCP/testrig/

Summary• UDP

– Latency as low as RTT– Risk packet loss, duplication, or wrong order

• TCP– Latency increase from UDP due to processing time– Insures packet transfer is correct

• Latency Mitigation– Avoid Hubs and Routers in LAN– Reduce unnecessary Nodes– Keep MAC/IP Correlation in ARP Cache– Utilize the Narrow Link Bandwidth– Maximize Buffer Settings– Use Parallel Streams

Further Work

• If time permitted, I would like to complete the below tasks. – Implementation of the tuning process in a real control system

and measure the actual speed increase in both a LAN and WAN system for the UDP and TCP protocols.

– Research other protocols to see if their RTT is faster or characteristics are better then UDP/TCP.

– Research topics on measurement of how much delay a system can handle before becoming unstable.

– Create a quick reference tuning guide/software that will allow you to measure desired information and change parameters within a network.

References

• TCP Tuning Techniques for High-Speed wide-Area Networks– http://fasterdata.es.net/TCP-tuning/TCP-Tuning-Tutorial.pdf

• Computer Networking Course Website ECE6600– http://www.neng.usu.edu/classes/ece/6600/slides.html

• Minimizing Average Network Delay for Ultra-Wideband Wireless Networks– http://www4.ncsu.edu/~mlsichit/Research/Publications/wcncUWB.

pdf• Stability Analysis of Networked Control Systems with Data

Dropout and Transmission Delays– http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=459417

8&isnumber=4592780• Topics in High-Performance Messaging

– http://www.29west.com/docs/THPM/packet-loss-myths.html

References• Stability of Networked Control Systems in the Presence of Packet Loss

– http://www.ecse.rpi.edu/~babak/cdc2003_camera_ready.pdf• Packet Loss Myths

– http://www.29west.com/docs/THPM/packet-loss-myths.html• Network Latency

– http://smutz.us/techtips/NetworkLatency.html• Using UDP socket connections for low-latency and loss-tolerant

scenarios in AIR 2 (Part 1)– http://www.insideria.com/2009/11/udp-socket-connections-for-los-1.html

• Comparative analysis – TCP – UDP– http://www.laynetworks.com/Comparative%20analysis_TCP%20Vs%20UDP.h

tm• Switches vs. Hubs

– http://www.networkclue.com/hardware/network/switches-vs-hubs.aspx

Questions?

Extra Material