Ok found the issue.
While the VM had a high net.ipv4.netfilter.ip_conntrack_max value, the dom0 xenserver didn't
All the connections of the guests count against the max value of the dom0 server.
Increased the limit by putting the following into /etc/sysctl.conf:
net.ipv4.netfilter.ip_conntrack_max = 131072
All works perfectly now. Probably will re-enable smart keepalives.
Consider this issue solved. Thanks for your help, enterprise license bought.
PS with Apache was getting about 300 req/s on a php script, with litespeed i get 20851 req/s - almost 70 times the performance!