Reading labview binary files in Matlab?

问题

I have large .bin files (10GB-60GB) created by Labview software, the .bin files represent the output of two sensors used from experiments that I have done.

The problem I have is importing the data into Matlab, the only way I have achieved this so far is by converting the .bin files to .txt files in Labview software then Importing the data into MATLAB using the following code:

Nlines = 1e6; % set number of lines to sample per cycle
sample_rate = (1); %sample rate
DECE= 1000;% decimation factor

TIME = (0:sample_rate:sample_rate*((Nlines)-1));%first inctance of time vector
format = '%f\t%f';
fid = fopen('H:\PhD backup\Data/ONK_PP260_G_text.txt');

while(~feof(fid))

   C = textscan(fid, format, Nlines, 'CollectOutput', true);
   d = C{1};  % immediately clear C at this point you need the memory! 
   clearvars C ;
   TIME = ((TIME(end)+sample_rate):sample_rate:(sample_rate*(size(d,1)))+(TIME(end)));%shift Time along 
   plot((TIME(1:DECE:end)),(d(1:DECE:end,:)))%plot and decimate
   hold on;
   clearvars d;
end

fclose(fid);

The basic idea behind my code is to conserve RAM by reading Nlines of data from .txt on disk to Matlab variable C in RAM, plotting C then clearing C. This process occurs in loop so the data is plotted in chunks until the end of the .txt file is reached.

I want to read the .bin file directly into MATLAB rather than converting it to .txt first because it takes hours for the conversion to complete and I have a lot of data. Here are some examples of my data but in manageable sizes:

https://www.dropbox.com/sh/rzut4zbrert9fm0/q9SiZYmrdG

Here is a description of the binary data: http://forums.ni.com/t5/LabVIEW/Loading-Labview-Binary-Data-into-Matlab/td-p/1107587

Someone has all ready written a Matlab script to import Labveiw .bin files but their script will only work with very small files:

% LABVIEWLOAD  Load Labview binary data into Matlab
%
%   DATA = LABVIEWLOAD(FNAME,DIM); % Loads the Labview data in binary
%      format from the file specified by FNAME, given the NUMBER of 
%      dimensions (not the actual dimensions of the data in the binary 
%      file) of dimensions specified by DIM.
%
%      LABVIEWLOAD will repeatedly grab data until the end of the file.
%      Labview arrays of the same number of dimensions can be repeatedly 
%      appended to the same binary file.  Labview arrays of any dimensions
%      can be read.
%
%   DATA = LABVIEWLOAD(FNAME,DIM,PREC); % Loads the data with the specified
%      precision, PREC.
%
%   Note:  This script assumes the default parameters were used in the
%      Labview VI "Write to Binary File".  Labview uses the Big Endian
%      binary number format.
%
%   Examples:
%       D = labviewload('Data.bin',2);  % Loads in Data.bin assuming it
%          contains double precision data and two dimensions.
%
%       D = labviewload('OthereData.bin',3,'int8');  % Loads in
%          OtherData.bin assuming it contains 8 bit integer values or
%          boolean values.
%
% Jeremiah Smith
% 4/8/10
% Last Edit: 5/6/10

function data = labviewload(fname,dim,varargin)
siz = [2^32 2^16 2^8 1]';  % Array dimension conversion table

% Precision Input
if nargin == 2
    prec = 'double';
elseif nargin == 3
    prec = varargin{1};
else
    error('Too many inputs.')
end

%% Initialize Values
fid = fopen(fname,'r','ieee-be');  % Open for reading and set to big-endian binary format
fsize = dir(fname);  % File information
fsize = fsize.bytes;  % Files size in bytes

%% Preallocation
rows = [];
columns = [];
I = 0;
while fsize ~= ftell(fid)
    dims = [];
    for i=1:1:dim
        temp = fread(fid,4);
        temp = sum(siz.*temp);
        dims = [dims,temp];
    end
    I = I + 1;
%      fseek(fid,prod(dims)*8,'cof');  % Skip the actual data
    temp = fread(fid,prod(dims),prec,0,'ieee-be');  % Skip the actual data (much faster for some reason)
end
fseek(fid,0,'bof'); % Reset the cursor
data = repmat({NaN*ones(dims)},I,1);  % Preallocate space, assumes each section is the same

%% Load and parse data
for j=1:1:I
    dims = [];  % Actual array dimensions
    for i=1:1:dim
        temp = fread(fid,4);
        temp = sum(siz.*temp);
        dims = [dims,temp];
    end
    clear temp i

    temp = fread(fid,prod(dims),prec,0,'ieee-be');  % 11 is the values per row,
        % double is the data type, 0 is the bytes to skip, and
        % ieee-be specified big endian binary data

%% Reshape the data into the correct array configuration
    if dim == 1
        temp = reshape(temp,1,dims);
    else
        evalfunc = 'temp = reshape(temp';
        for i=1:1:dim
            evalfunc = [evalfunc ',' num2str(dims(dim-i+1))];
        end
        if dim ~= 2
            eval([evalfunc ');'])
        else
            eval([evalfunc ')'';'])
        end
    end

    data{j} = temp;  % Save the data
end

fclose(fid);  % Close the file

The code has the following error message, when you try to process even relatively small .bin files:

Error using ones
Maximum variable size allowed by the program is exceeded.

Error in labviewload (line 65)
data = repmat({NaN*ones(dims)},I,1);  % Preallocate space, assumes each section is the same

Can you help me modify the code so that I can open large .bin files? Any help will be much appreciated.

Cheers, Jim

来源：https://stackoverflow.com/questions/21281788/reading-labview-binary-files-in-matlab

标签

matlab

bigdata

fread