summary of the problem
we are having a setup wherein a lot(800 to 2400 per second) of incoming connections to a linux box and we have a NAT device between the client and server. so there are so many TIME_WAIT sockets left in the system. To overcome that we had set tcp_tw_recycle to 1, but that led to drop of in comming connections. after browsing through the net we did find the references for why the dropping of frames with tcp_tw_recycle and NAT device happens.
resolution tried
we then tried by setting tcp_tw_reuse to 1 it worked fine without any issues with the same setup and configuration.
But the documentation says that tcp_tw_recycle and tcp_tw_reuse should not be used when the Connections that go through TCP state aware nodes, such as firewalls, NAT devices or load balancers may see dropped frames. The more connections there are, the more likely you will see this issue.
Queries
- can tcp_tw_reuse be used in this type of scenarios?
- if not, which part of the linux code is preventing tcp_tw_reuse being used for such scenario?
- generally what is the difference between tcp_tw_recycle and tcp_tw_reuse?