I am having this annoying problem which takes 10-20 seconds and sometimes more every time my android project is refreshed or rebuilt.
this helped me so you can try too:
Go to your Workspace root/.metadata/.plugins/org.eclipse.jdt.core/ and remove .org.eclipse.jdt.core.external.folders folder
it contains .project file with links to external folders which takes too long to refresh