问题
I got my SPA website (based on Node/Express/Mongo/Angular X) up and running. I created a sitemap.xml and submitted to Microsoft Bing, and from the server log, I see they started crawling. However, I noticed the page URL is called, but not the associated API for that page. So, basically it's just indexing the static skeleton of each page, not the dynamic real content.
I googled and see people saying "google can't index dynamic content" as suggested in this article. However, I also see other people saying crawler is just a person browsing and it should get its dynamic content.
I'm confused. Can somebody clarify? How to fix it?
回答1:
Web crawlers don't execute JavaScript. Google probably has a more sophisticated crawler that does, but most don't. It's just too compute intensive for trillions of pages. Add a static link for crawlers to follow. See https://stackoverflow.com/a/28075506/148844
Bing doesn't index JavaScript generated content.
https://www.bing.com/webmaster/help/webmaster-guidelines-30fba23a
Site technology The technology used on your website can sometimes prevent Bingbot from being able to find your content. Rich media (Flash, JavaScript, etc.) can lead to Bing not being able to crawl through navigation, or not see content embedded in a webpage. To avoid any issue, you should consider implementing a down-level experience which includes the same content elements and links as your rich version does. This will allow anyone (Bingbot) without rich media enabled to see and interact with your website.
Rich media cautions – don’t bury links to content inside JavaScript
Rich media warnings – don’t bury links in Javascript/flash/Silverlight;keep content out of these as well
Down-level experience enhances discoverability – avoid housing content inside Flash or JavaScript – these block crawlers form finding the content
来源：https://stackoverflow.com/questions/50299587/why-is-bing-crawler-not-fetching-the-dynamic-content-of-my-angular-web-page