Why does UWP continuous speech recognition stop

Question

I have a Windows 10 UWP app that I am enabling voice recognition for a text box. Yes, I know that I can also leverage Cortana for this. However, Cortana comes with some cons as well, mainly that you have little to no control over Cortana from within the app.

This is where the Continuous Recognition of the SpeechRecognizer namespace comes in. I like the amount of control I have. However, it seems to randomly stop listening after some seconds.

Here is how I have it implemented. Note that I also tried to set every possible timeout to 0 which should mean no timeout.

Properties on page:

private SpeechRecognizer speechRecognizer;
private CoreDispatcher dispatcher;

OnLoaded for the page:

speechRecognizer = new SpeechRecognizer();
speechRecognizer.Timeouts.BabbleTimeout = TimeSpan.FromSeconds(0);
speechRecognizer.Timeouts.InitialSilenceTimeout = TimeSpan.FromSeconds(0);
speechRecognizer.Timeouts.EndSilenceTimeout = TimeSpan.FromSeconds(0);
speechRecognizer.ContinuousRecognitionSession.AutoStopSilenceTimeout = TimeSpan.FromSeconds(0);

SpeechRecognitionCompilationResult result = await speechRecognizer.CompileConstraintsAsync();
speechRecognizer.ContinuousRecognitionSession.ResultGenerated += ContinuousRecognitionSession_ResultGenerated;
speechRecognizer.StateChanged += SpeechRecognizer_StateChanged;

Then, when I click a button to start listening, I do this to start:

if (speechRecognizer.State == SpeechRecognizerState.Idle)
{
     await speechRecognizer.ContinuousRecognitionSession.StartAsync();
}

Finally, I listen to the two event handlers above, for ResultGenerated and StateChanged. I have breakpoints set in those two events. When the page loads, everything is instantiated just fine. When I click the button to start listening, it does start just fine as well and I see the StateChanged event handler fire to show it is listening. However, if I let the app sit idle (no speaking) for a few seconds (and the amount of seconds seems random, can be anywhere between 2-5 seconds), the StateChanged event will fire and say it is idle again. After that, I cannot get the ResultGenerated event to fire when I try speaking which further shows it is not listening anymore.

I can click the button to start listening again and it will, but with the same random stopping again.

Also, if I do speak right away, after I click the button, the speech recognition does work just fine.

What I want to happen is when you click the button, I want it to listen indefinitely, until I call StopAsync and tell it to stop. Anybody know why it just stops on its own???

UPDATE- I added the event handler for completed:

speechRecognizer.ContinuousRecognitionSession.Completed += ContinuousRecognitionSession_Completed;

Because this would give me a status in args.Status and I put a breakpoint there. The funny thing is, this breakpoint will hit in the 2-5 seconds when continuous recognition stops and it gives a status of "SUCCESS" even though I didn't speak anything and the ResultGenerated event never fired with a result. So, how is it getting a success with no result? And why is this causing it to stop?

Thanks!

Answer 1

So I had the same problem and came across this question. I think I figured it out eventually. The problem is that when the UWP app goes from the foreground (like switching to another app) the Speech Recognizer will stop (without any event).

When debugging this will, of course, happen when you set breakpoints. I think the problem is fixed by restarting the SpeechRecognizer when it goes into the foreground again.