I\'ve tried everything to change the max_execution_time of a php crawler script so that it can run an infinite amount of time.
max_execution_time
I have changed the php.in
You could also try setting ignore_user_abort(TRUE); in your script as it might be the browser timing out rather than the script.
ignore_user_abort(TRUE);
From the php.net manual
See here for more info
http://www.php.net/manual/en/function.ignore-user-abort.php