Nevermind. I found the problem.
Apparently, I had two VH configs, one for port 80 and one for port 443 (ssl). They both used the same php socket but the SetUID config for 443 was set to Server instead of Doc Root which overrode the setting for port 80.
I just changed it to Doc Root and now it works fine again.