I would like to limit the number of connections to my server to X. This can be done by
iptables -A INPUT -p tcp --dport ABCD -m connlimit --connlimit-above X --connlimit-mask 0 -j DROP
Also, I would like to redirect the dropped requests to another port, say PQRS. How do I do this?
You can redirect them to other chain and then handle it on other chain.
Create a custom chain:
iptables -N custom_chain
Then redirect the traffic to the new chain:
iptables -A INPUT -p tcp --dport ABCD -m connlimit --connlimit-above X --connlimit-mask 0 -j custom_chain
Finally handle the traffic on your new custom chain to redirect traffic to desired XXXX port:
iptables -t nat -A custom_chain -p tcp --dport ABCD -j REDIRECT --to-port XXXX