TCP listen queue full

Some clients started to have TCP connection issues and it ended up due to TCP listen queue full on server side. Server was written in perl and running on AIX

Troubleshooting the issue with tcpdump, I could see TCP SYN packets received on server side from client, but server is not responding - that led me to look at TCP statistics and  'netstat -s' did show below

$netstat -s|grep 'queue'
                933122 discarded due to listener's queue full

This counter was increasing over time. Looking at the perl code, listen backlog was set to 100.

$socket = new IO::Socket::INET (

LocalHost => $listen,

LocalPort => $port,

Proto => 'tcp',

Listen => 100,

Reuse => 1

) 

However, more clients were added to this application recently and were receving more than 100 connections at a time - which was causing packets to be droped. Simple fix was to increase listen backlog parameter in perl code and restart server

At OS layer the backlog is set via 'somaxconn' parameter. You could use below command to check the same on AIX

$no -o somaxconn

somaxconn = 16384