From the ZooKeeper FAQ:
Reliability:
A single ZooKeeper server (standalone) is essentially a coordinator with
no reliability (a single serving node failure
Basically, Zookeeper will work just fine as long as Active Zookeepers are in MAJORITY compared to failed Zookeepers. Also, in case of even quorum size i.e 2,4,6 etc. Failed = Active, because of that its not recommended.
Both 3 and 4 will handle only 1 faliures then why whould we want to used 4 Zookeepers instead of 3.