Reading an array from a text file in Common Lisp

Question

I am trying to read data (which is actually an array) in Lisp from a text file. I tried to use with-open-file and read-line stuff but could not achieve my goal. What I am looking for is equivalent to doing data=load('filename.txt') in MATLAB, so that I get an array called data which has loaded the whole information in filename.txt.

The text file will be in a format like

1.0 2.0 3.0 ...
1.5 2.5 3.5 ...
2.0 3.0 4.0 ...
 .....

The size may also vary. Thanks a lot in advance.

Answer 1

The basic way to do that is to use with-open-file for getting the input stream, read-line in a loop to get the lines, split-sequence (from the library of the same name) to split it into fields, and parse-number (from the library of the same name) to transform the strings into numbers. All libraries mentioned are available from Quicklisp.

EDIT: Just to get you started, this is a simple version without validation:

(defun load-array-from-file (filename)
  (with-open-file (in filename
                      :direction :input)
    (let* ((data-lol (loop :for line := (read-line in nil)
                           :while line
                           :collect (mapcar #'parse-number:parse-number
                                            (cl-ppcre:split "\\s+" line))))
           (rows (length data-lol))
           (columns (length (first data-lol))))
      (make-array (list rows columns)
                  :initial-contents data-lol))))

You should add some checks and think about what you want to get in case they are not fulfilled:

• Are the rows all the same length?
• Are all fields valid numbers?

Answer 2

Assuming your file follows the formatting pattern you gave in your question: a sequence of numbers separated with white spaces, this is a quick snippet that should do what you want.

(defun read-array (filename)
  (with-open-file (in filename)
    (loop for num = (read in nil)
          until (null num)
          collect num)))

Answer 3

Another approach is to leverage the lisp reader to parse the data in the text file. To do this, I'd probably convert the entire file into a string first, and then call

(eval (read-from-string (format nil "~a~a~a" "(initial wrapper code " str ")")))

For example, if you wanted to read in a data file that is all numbers, delimited by whitespace/newlines, into a list, the previous command would look like:

(eval (read-from-string (format nil "~a~a~a" "(list " str ")")))

Answer 4

I followed Svante's advice. I just needed a single column in the text file, this is what I am using for this purpose.

(defun load_data (arr column filename)
(setf lnt (first (array-dimensions arr)))
 (with-open-file (str (format nil "~A.txt" filename) :direction :input)
   (loop :for i :from 0 :to (1- lnt) :do
       (setf (aref arr i 0) (read-from-string (nth (1- column) (split-sequence:SPLIT-SEQUENCE #\Space (read-line str))))))))

Thank you all for your help.