问题
Currently we were working on a cloud migration project where we made the following changes:
SQL Server 2014 to Azure SQL PaaS
Redis cache[Windows porting] to Azure Redis PaaS
Static files from shared drives to Azure File Service
Implemented Transient Fault Handling for database interactions.
HttpSessionState changed from SQL Server to Custom[Redis PaaS]
The Application had two web applications that use the same database:
One built in Classic model of dot net coding with web-forms.
The other application built using dot net MVC4.
After we moved the applications from existing Rackspace environment[2 servers each with 4GB RAM] to Azure and ran a load test and received the following results:
The MVC4 application is fractionally faster.
The Web-Form application started performing poorly, with same load, response time increased from 0.46 seconds to 45.8 seconds.
The memory usage is same, database utilization is around 30%-40% and the CPU utilization in nearly 100%(all the web-servers) at 1100 concurrent users(at Rackspace, it served 4500 concurrent users).
We tested the application 2 D5 azure server VMs with RAM being higher and CPU being faster.
Can anyone highlight how such drastic performance drop(one application performing almost same, other one performing almost 100 times slower) is possible?
NB: One observation, the CPU utilization stays at 100% even after 30mins of stopping the load-test. Then it drops quickly.
回答1:
I will second the notion (emphatically) that you invest as much time and energy as you can in profiling your application to identify bottlenecks. Run profiles on-premises and in Azure and compare, if possible.
Your application clearly has many moving parts and a reasonably large surface area... that's no crime, but it does mean that its hard to pinpoint the issue(s) you're having without some visibility into runtime behavior. The issue could lie in your Redis caching, in the static file handling, or in the session state loading/unloading/interaction cycle. Or it could be elsewhere. There's no magic answer here... you need data.
That said... I've consulted on several Azure migration projects and my experience tells me that one area to look closer at is the interaction between your ASP.NET Web Forms code and SQL. Overly-chatty applications (ones that average multiple SQL calls per HTTP request) and/or ones that issue expensive queries that perform lots of logic on the database or return large result sets, tend to exhibit poor performance in public clouds like Azure, where code and data may not be co-located, "noisy neighbor" problems can impact database performance, etc. These issues aren't unique to Web Forms applications or Azure, but they tend to be exacerbated in older, legacy applications that were written with an assumption of code and data being physically close. Since you don't control (completely) where your code and data live in Azure relative to each other, issues that may be masked in an on-prem scenario can surface when moving to the cloud.
Some specifics to consider:
take a close look at the use of data binding in your Web Forms app... in practice it tends to encourage expensive queries and transfer of large result sets from database to application, something you might sometimes get away with on-premises but not in the cloud
take another look at your SQL configuration... you don't mention what tier you're using (Basic, Standard, Premium) but this choice can have a big impact on your overall app performance (and budget!). If it turns out (for example) that your Web Forms app does issue expensive queries, then use of a higher tier may help
Azure SQL DB tiers
- familiarize yourself with the notion of "cloud native" vs. "cloud capable" applications... generally speaking, just because you can find a way to run an application in the cloud doesn't mean its ideally suited to do so. From your description it sounds like you've made some effort to utilize some cloud-native services, so that's a good start. But if I had to guess (since we can't see your code) I'd think that you might need some additional refactoring in your Web Forms app to make it more efficient and better able to run in an environment you don't have 100% control over.
More on cloud-native
dated but still relevant advice on Azure migration
If you can give us more details on where you see bottlenecks, we can offer more specific advice.
Best of luck!
回答2:
There is some loop in the code that cause 100% CPU.
When the problem occurs, take a dump from (the kudu). Analyze it in windbg by 1) list threads cpu time with !runaway 2) check the callstack of the threads, specifically the greatest cpu consumer with ~*e!clrstack and with ~*kb
来源：https://stackoverflow.com/questions/36477242/huge-performance-drop-after-moving-to-azure