I have upgraded to PHP 5.3 on a development machine (Windows 7 box).
php-v shows
PHP 5.3.3 (cli) (built: Jul 21 2010 20:36:55)
Copyright (c) 1997-20
Hello
You should also update apache php module
@soju - that fixed it, thanks, but I thought I'd make this a little clearer for others.
Apache requires the php module to serve php files. On Windows it comes as a .dll file. This would be called: php5apache2_2.dll - located within your php folder. Strangely though, during upgrading the php folder with a newer version, you may find out that apache still uses the old php module file. So while php -v correctly reports the new version, Apache still looks at the old php module (reported by phpinfo). To find out what module version you have, you could: right-click on the .dll file > select properties -> view the details tab. Or just hover on the file.

If this version is not what you want, you might want to hunt for a newer version and replace it. Good idea to back up first. Also, the php5apache.dll file works with and requires the file: php5ts.dll - so ensure these files are the same versions.
With your httpd.conf file with the directive:
LoadModule php5_module "\path to\php5apache2_2.dll"
You may restart apache and smile!