Comparing two dictionaries with numpy matrices as values

Question

I want to assert that two Python dictionaries are equal (that means: equal amount of keys, and each mapping from key to value is equal; order is not important). A simple way

Answer 1

I'm going to answer the half-question hidden in your question's title and first half, because frankly, this is a much more common problem to be solved and the existing answers don't address it very well. This question is "How do I compare two dicts of numpy arrays for equality"?

The first part of the problem is checking the dicts "from afar": see that their keys are the same. If all the keys are the same, the second part is comparing each corresponding value.

Now the subtle issue is that a lot of numpy arrays are not integer-valued, and double-precision is imprecise. So unless you have integer-valued (or other non-float-like) arrays you will probably want to check that the values are almost the same, i.e. within machine precision. So in this case you wouldn't use np.array_equal (which checks exact numerical equality), but rather np.allclose (which uses a finite tolerance for the relative and absolute error between two arrays).

The first one and a half parts of the problem are straightforward: check that the keys of the dicts agree, and use a generator comprehension to compare every value (and use all outside the comprehension to verify that each item is the same):

import numpy as np

# some dummy data

# these are equal exactly
dct1 = {'a': np.array([2, 3, 4])}
dct2 = {'a': np.array([2, 3, 4])}

# these are equal _roughly_
dct3 = {'b': np.array([42.0, 0.2])}
dct4 = {'b': np.array([42.0, 3*0.1 - 0.1])}  # still 0.2, right?

def compare_exact(first, second):
    """Return whether two dicts of arrays are exactly equal"""
    if first.keys() != second.keys():
        return False
    return all(np.array_equal(first[key], second[key]) for key in first)

def compare_approximate(first, second):
    """Return whether two dicts of arrays are roughly equal"""
    if first.keys() != second.keys():
        return False
    return all(np.allclose(first[key], second[key]) for key in first)

# let's try them:
print(compare_exact(dct1, dct2))  # True
print(compare_exact(dct3, dct4))  # False
print(compare_approximate(dct3, dct4))  # True

As you can see in the above example, the integer arrays compare fine exactly, and depending on what you're doing (or if you're lucky) it could even work for floats. But if your floats are the result of any kind of arithmetic (linear transformations for instance?) you should definitely use an approximate check. For a complete description of the latter option please see the docs of numpy.allclose (and its elementwise friend, numpy.isclose), with special regard to the rtol and atol keyword arguments.

Answer 2

Consider this code

>>> import numpy as np
>>> np.identity(5)
array([[ 1.,  0.,  0.,  0.,  0.],
       [ 0.,  1.,  0.,  0.,  0.],
       [ 0.,  0.,  1.,  0.,  0.],
       [ 0.,  0.,  0.,  1.,  0.],
       [ 0.,  0.,  0.,  0.,  1.]])
>>> np.identity(5)+np.ones([5,5])
array([[ 2.,  1.,  1.,  1.,  1.],
       [ 1.,  2.,  1.,  1.,  1.],
       [ 1.,  1.,  2.,  1.,  1.],
       [ 1.,  1.,  1.,  2.,  1.],
       [ 1.,  1.,  1.,  1.,  2.]])
>>> np.identity(5) == np.identity(5)+np.ones([5,5])
array([[False, False, False, False, False],
       [False, False, False, False, False],
       [False, False, False, False, False],
       [False, False, False, False, False],
       [False, False, False, False, False]], dtype=bool)
>>> 

Note the the result of the comparison is a matrix, not a boolean value. Dict comparisons will compare values using the values cmp methods, which means that when comparing matrix values, the dict comparison will get a composite result. What you want to do is use numpy.all to collapse the composite array result into a scalar boolean result

>>> np.all(np.identity(5) == np.identity(5)+np.ones([5,5]))
False
>>> np.all(np.identity(5) == np.identity(5))
True
>>> 

You would need to write your own function to compare these dictionaries, testing value types to see if they are matricies, and then comparing using numpy.all, otherwise using ==. Of course, you can always get fancy and start subclassing dict and overloading cmp if you want too.

Answer 3

You can use numpy.testing.assert_equal

http://docs.scipy.org/doc/numpy/reference/generated/numpy.testing.assert_equal.html