I\'ve built a crawler that had to run on about 5M pages (by increasing the url ID) and then parses the pages which contain the info\' I need.
after using an algorith
Instead of doing recursion, the parts of the code with checkNextID(ID + 18) and similar could be replaced with ID+=18, and then if you remove all instances of return 0, then it should do the same thing but as a simple loop. You should then put a return 0 at the end and make your variables non-global.