I\'m working on a project with the World Bank analyzing their procurement processes.
The WB maintains websites for each of their projects, containing links and data
I am not sure I have understood every details of your problem. But what I know for sure is that casperJS works great for javascript generated content.
You can have a look at it here: http://casperjs.org/
It's written in Javascript and has a bunch of useful functions very well documented on the link I provided.
I have used it myself lately for a personal project and can be set up easily with a few lines of code.
Give it a go! Hope, that helps..