Console output missing in Azure DevOps release when enabling “Specify a batch size”

Question

We have a large test set running (based on NUnit) within Azure DevOps. Recently we enabled the \"Rerun failed tests\" option within the Visual Studio Test task. At first, th

Answer 1

I posted the question on developercommunity.visualstudio.com as well. The discussion took a bit of time, but summarized, the following response was given:

"The supported behavior is to have the Trace information from the tests be present as part of the test results file (trx)/Standard console logs and not part of the build logs (as you have mentioned in your question). Do note this is also dependent on the test framework being used. There’s no plan to have the trace information flow to the build logs from the test. The different behavior you are seeing when you turn on Batching option is due to the fact the internal flow of execution changes a bit (and eventually all flows will converge to same as batch option in coming days). we recommend not to take dependency on build logs. Instead the test tab is the place you will get in better logs in context to test case/test run."

A full test run output should be available in the trx files found in the test run. Upon check, this is indeed the case:

When opening the file in Notepad++, I finally see the logs of my complete run:

- All these lines are written by using Console.WriteLine().

- If there are more trx files than one, then the largest file holds the logs of the complete run: the small file only has the logs for the selected test.

NOTE: We found that during a test run timeout, this file is not generated. This has been reported to Microsoft and is acknowledged as an issue:

"got the point here. we are working on Advanced diagnostics in vstest task. for instance we will abort the test if a test is taking more time to complete. in this case a dump of the test process will be created as well and ofcourse we will have trx also getting uploaded. the second scenario where not a perticular test is taking time but overall run ends time out we will take a dump of test process and abort the run. Dumps will help you debug the issue."

Somewhere in the future, this dump should be available to use.