Unit testing code with a file system dependency

Question

I am writing a component that, given a ZIP file, needs to:

1. Unzip the file.
2. Find a specific dll among the unzipped files.
3. Load that dll thro

Answer 1

There's really nothing wrong with this, it's just a question of whether you call it a unit test or an integration test. You just have to make sure that if you do interact with the file system, there are no unintended side effects. Specifically, make sure that you clean up after youself -- delete any temporary files you created -- and that you don't accidentally overwrite an existing file that happened to have the same filename as a temporary file you were using. Always use relative paths and not absolute paths.

It would also be a good idea to chdir() into a temporary directory before running your test, and chdir() back afterwards.

Answer 2

I am reticent to pollute my code with types and concepts that exist only to facilitate unit testing. Sure, if it makes the design cleaner and better then great, but I think that is often not the case.

My take on this is that your unit tests would do as much as they can which may not be 100% coverage. In fact, it may only be 10%. The point is, your unit tests should be fast and have no external dependencies. They might test cases like "this method throws an ArgumentNullException when you pass in null for this parameter".

I would then add integration tests (also automated and probably using the same unit testing framework) that can have external dependencies and test end-to-end scenarios such as these.

When measuring code coverage, I measure both unit and integration tests.

Answer 3

For unit test I would suggest that you include the test file in your project(EAR file or equivalent) then use a relative path in the unit tests i.e. "../testdata/testfile".

As long as your project is correctly exported/imported than your unit test should work.

Answer 4

This seems to be more of an integration test as you are depending on a specific detail (the file system) that could change, in theory.

I would abstract the code that deals with the OS into it's own module (class, assembly, jar, whatever). In your case you want to load a specific DLL if found, so make an IDllLoader interface and DllLoader class. Have your app acquire the DLL from the DllLoader using the interface and test that .. you're not responsible for the unzip code afterall right?

Answer 5

Your question exposes one of the hardest parts of testing for developers just getting into it:

"What the hell do I test?"

Your example isn't very interesting because it just glues some API calls together so if you were to write a unit test for it you would end up just asserting that methods were called. Tests like this tightly couple your implementation details to the test. This is bad because now you have to change the test every time you change the implementation details of your method because changing the implementation details breaks your test(s)!

Having bad tests is actually worse than having no tests at all.

In your example:

void DoIt(IZipper zipper, IFileSystem fileSystem, IDllRunner runner)
{
   string path = zipper.Unzip(theZipFile);
   IFakeFile file = fileSystem.Open(path);
   runner.Run(file);
}

While you can pass in mocks, there's no logic in the method to test. If you were to attempt a unit test for this it might look something like this:

// Assuming that zipper, fileSystem, and runner are mocks
void testDoIt()
{
  // mock behavior of the mock objects
  when(zipper.Unzip(any(File.class)).thenReturn("some path");
  when(fileSystem.Open("some path")).thenReturn(mock(IFakeFile.class));

  // run the test
  someObject.DoIt(zipper, fileSystem, runner);

  // verify things were called
  verify(zipper).Unzip(any(File.class));
  verify(fileSystem).Open("some path"));
  verify(runner).Run(file);
}

Congratulations, you basically copy-pasted the implementation details of your DoIt() method into a test. Happy maintaining.

When you write tests you want to test the WHAT and not the HOW. See Black Box Testing for more.

The WHAT is the name of your method (or at least it should be). The HOW are all the little implementation details that live inside your method. Good tests allow you to swap out the HOW without breaking the WHAT.

Think about it this way, ask yourself:

"If I change the implementation details of this method (without altering the public contract) will it break my test(s)?"

If the answer is yes, you are testing the HOW and not the WHAT.

To answer your specific question about testing code with file system dependencies, let's say you had something a bit more interesting going on with a file and you wanted to save the Base64 encoded contents of a byte[] to a file. You can use streams for this to test that your code does the right thing without having to check how it does it. One example might be something like this (in Java):

interface StreamFactory {
    OutputStream outStream();
    InputStream inStream();
}

class Base64FileWriter {
    public void write(byte[] contents, StreamFactory streamFactory) {
        OutputStream outputStream = streamFactory.outStream();
        outputStream.write(Base64.encodeBase64(contents));
    }
}

@Test
public void save_shouldBase64EncodeContents() {
    OutputStream outputStream = new ByteArrayOutputStream();
    StreamFactory streamFactory = mock(StreamFactory.class);
    when(streamFactory.outStream()).thenReturn(outputStream);

    // Run the method under test
    Base64FileWriter fileWriter = new Base64FileWriter();
    fileWriter.write("Man".getBytes(), streamFactory);

    // Assert we saved the base64 encoded contents
    assertThat(outputStream.toString()).isEqualTo("TWFu");
}

The test uses a ByteArrayOutputStream but in the application (using dependency injection) the real StreamFactory (perhaps called FileStreamFactory) would return FileOutputStream from outputStream() and would write to a File.

What was interesting about the write method here is that it was writing the contents out Base64 encoded, so that's what we tested for. For your DoIt() method, this would be more appropriately tested with an integration test.

Answer 6

There's nothing wrong with hitting the file system, just consider it an integration test rather than a unit test. I'd swap the hard coded path with a relative path and create a TestData subfolder to contain the zips for the unit tests.

If your integration tests take too long to run then separate them out so they aren't running as often as your quick unit tests.

I agree, sometimes I think interaction based testing can cause too much coupling and often ends up not providing enough value. You really want to test unzipping the file here not just verify you are calling the right methods.