After configuring a virtual host, the nginx configuration (/opt/zextras/conf/nginx/include/*) doesn't have any sign of the new virtual host, so the correct certificate isn't used.
Restarting the proxy didn't resolve this either.
Looking through the old Zimbra documentation, it seems that zmproxyconfgen is responsible for generating the configuration. Running this manually did solve that issue, but I'd imagine this should be run automatically either when the virtual host configuration is changed, or perhaps when the proxy is restarted/reloaded.
Another issue that's related, is that ${web.http.enabled} does not seem follow what zimbraReverseProxyHttpEnabled is set to. This prevents nginx from running on port 80 without manually editing the configuration.
Debugging this is a little tricky though, as zmproxyconfgen -D throws an exception:
Exception in thread "main" java.lang.NullPointerException
at com.zimbra.cs.util.proxyconfgen.ProxyConfGen.displayVariables(ProxyConfGen.java:782)
at com.zimbra.cs.util.proxyconfgen.ProxyConfGen.createConf(ProxyConfGen.java:2173)
at com.zimbra.cs.util.proxyconfgen.ProxyConfGen.main(ProxyConfGen.java:2676)