TypeError: Mismatch between array dtype ('object') and format specifier ('%.18e')

Question

I have the following array:

X = np.array([image_array_to_vector1,image_array_to_vector2,image_array_to_vector3,image_array_to_vector4])

A print out of X looks as follows:

[array([167, 167, 169, ...,   1,   1,   1], dtype=uint8)
 array([42, 43, 43, ..., 41, 36, 34], dtype=uint8)
 array([0, 0, 0, ..., 0, 0, 0], dtype=uint8)
 array([0, 0, 0, ..., 0, 0, 0], dtype=uint8)]

When I try to save the data as txt:

X_to_text_file = np.savetxt('x.txt',X)

I get the following:

File "/Library/Python/2.7/site-packages/numpy/lib/npyio.py", line 1258, in savetxt
    % (str(X.dtype), format))
TypeError: Mismatch between array dtype ('object') and format specifier ('%.18e')

Why is that? How can I solve the issue?

Thanks.

Answer 1

It's a little difficult to duplicate this without some example data, but here is what I came up with.

arr = np.array([np.array([1,2,3]), np.array([1,2,3,4])])
arr
array([array([1, 2, 3]), array([1, 2, 3, 4])], dtype=object)
np.savetxt('x.txt', arr)
TypeError: Mismatch between array dtype ('object') and format specifier ('%.18e')

As pointed out by @Artier there is a little snippet at the end of the accepted answer in Write object array to .txt file that points out you can just save the array as a string with fmt='%s'. Using this format seems to solve the problem for me (again I can't recreate your issue exactly without a sample of data).

np.savetxt('x.txt', arr, fmt='%s')

I would point out that if you are looking to save disparate arrays and want a single location to keep them savez is very useful.

Answer 2

In essence savetxt is doing:

for row in your_array:
    print(fmt % tuple(row))

where fmt is constructed your fmt parameter (or the default one) and the number of columns, and the delimiter.

You have a 1d array of objects (arrays). So the write/print will be

 print(fmt % tuple(element))

%s is the only format that can handle an array (or other general object).

savetxt is meant to be used with 2d numeric arrays, the kind of thing that will produce need csv columns. Trying to use it on other things like this object array is going to give you headaches.

In [2]: A = np.empty((3,),dtype=object)
In [3]: A[:] = [np.arange(3),np.arange(1,4), np.arange(-3,0)]
In [4]: A
Out[4]: array([array([0, 1, 2]), array([1, 2, 3]), array([-3, -2, -1])], dtype=object)

In [6]: np.savetxt('test',A,fmt='%s')
In [7]: cat test
[0 1 2]
[1 2 3]
[-3 -2 -1]

Iterating on a 1d array it must be skipping the tuple. In any case the best you can do is a %s format. Otherwise write your own custom file writer. savetxt isn't anything special or powerful.

In [9]: for row in A:
   ...:     print('%s'%row)  
[0 1 2]
[1 2 3]
[-3 -2 -1]

Answer 3

I had the same error message, and it appeared that the problem was with arrays having different length. So you need to be sure to give the np.savetxt arrays of equal length.

Answer 4

For me error occurred when using an older conda environment which had numpy=1.13.1.

Updating to numpy=1.16.2 gave no such errors.