No changes to httpd.conf as far as I know. Removing the UserDir public_html fixed the 500 on the machines. After doing so one machine I was able to use ~user to access the users folder but others I am not as I get a 404 error. The issue now is http://ip/~user on other machines it does not send it to the /home/user/www folder instead to /usr/local/apache/htdocs
It had to go through the actual shared IP the user was on. Now what's interesting here is I have a machine where that is not the case. What we've always done is given the user the servers main ip as the location to get to ~user. The reason being most systems for setting up accounts don't have the smarts to determine the IP the user was actually put on. So they just give the user http://mainIP/~username .