Unit Testing with functions that return random results

Question

I don\'t think that this is specific to a language or framework, but I am using xUnit.net and C#.

I have a function that returns a random date in a certain range. I

Answer 1

Sure, using a fixed seed random number generator will work just fine, but even then you're simply trying to test for that which you cannot predict. Which is ok. It's equivalent to having a bunch of fixed tests. However, remember--test what is important, but don't try to test everything. I believe random tests are a way to try to test everything, and it's not efficient (or fast). You could potentially have to run a great many randomized tests before hitting a bug.

What I'm trying to get at here is that you should simply write a test for each bug you find in your system. You test out edge cases to make sure your function is running even in the extreme conditions, but really that's the best you can do without either spending too much time or making the unit tests slow to run, or simply wasting processor cycles.

Answer 2

If you want to check the quality of the random numbers (in terms of independance) there are several ways to do this. One good way is the Chi square test.

Answer 3

You don't need to control the system to make the results deterministic. You're on the right approach: decide what is important about the output of the function and test for that. In this case, it is important that the result be in a range of 40 days, and you are testing for that. It's also important that it not always return the same result, so test for that too. If you want to be fancier, you can test that the results pass some kind of randomness test..

Answer 4

In addition to testing that the function returns a date in the desired range, you want to ensure that the result is well-distributed. The test you describe would pass a function that simply returned the date you sent in!

So in addition to calling the function multiple times and testing that the result stays in the desired range, I would also try to assess the distribution, perhaps by putting the results in buckets and checking that the buckets have roughly equal numbers of results after you are done. You may need more than 100 calls to get stable results, but this doesn't sound like an expensive (run-time wise) function, so you can easily run it for a few K iterations.

I've had a problem before with non-uniform "random" functions.. they can be a real pain, it's worth testing for early.

Answer 5

Mock or fake out the random number generator

Do something like this... I didn't compile it so there might be a few syntax errors.

public interface IRandomGenerator
{
    double Generate(double max);
}

public class SomethingThatUsesRandom
{
    private readonly IRandomGenerator _generator;

    private class DefaultRandom : IRandomGenerator
    {
        public double Generate(double max)
        {
            return (new Random()).Next(max);
        }
    }

    public SomethingThatUsesRandom(IRandomGenerator generator)
    {
        _generator = generator;
    }

    public SomethingThatUsesRandom() : this(new DefaultRandom())
    {}

    public double MethodThatUsesRandom()
    {
        return _generator.Generate(40.0);
    }
}

In your test, just fake or mock out the IRandomGenerator to return something canned.