I successfully installed PHP 5.3.29 on Ubuntu 14 with Apache 2 separately.
I installed PHP with the following method:
sudo -i
wget http://in1.php.net
Wittich's answer is fabulous! I used it in order to replace php 5.1 with php 5.6 on my unix website. Unlike Wittich, I wasn't trying to get two versions running simultaneously; I was simply trying to do an upgrade from an older version of php that was integrated with apache to a newer version that would run from cgi.
For some reason, several of Wittich's commands (such as the SetHandler command) prevented Apache from loading after I tried them. I ended up having to simplify his answer so that Apache would run correctly. I made 6 changes in Wittich's procedures:
I used yum instead of apt-get because apt-get wouldn't run on my Unix system.
Since I was installing php 5.6 not php 5.3, I changed all instances of "php53" to "php56" in the configure command's switches. This was not necessary for everything to work, but it will help you understand my paths below:
In one of my httpd configuration folders (/etc/httpd/conf.d), I found a file that was already set up for a cgi implementation of php (php_cgi.conf). Its contents were:
scriptAlias /phppath/ "/var/www/cgi-bin/cgi_wrapper/"
Action php-script /phppath/cgi_wrapper
The contents and existence of this file let me skip or change several of Wittich's steps. I neither created php53-cgi, nor did I create php53.conf.
\#!/bin/sh
exec /usr/bin/php56/php-cgi
AddHandler php-script .php
So, in summary, it is easy to convert Wittich's answer into a way to upgrade from an older apache based php version to a newer cgi-based version if you have an existing php_cgi.conf file that you can hijack and use as a guide.
I found a posting by David Brogdon to be a good complement to Wittich's answer for those who are new to the configure, make, and make install commands:
http://www.onlamp.com/pub/a/php/2000/11/17/php_admin.html
I searched the Internet for hours and never found anything as helpful as Brogdon's post and Wittich's answer. Wittich's answer must have worked perfectly on his Unix system, but I had to modify it a bit in order to get it to work on mine.