Is clojure's read file structure, i.e. with-open and clojure.java.io/reader, efficient enough for frequent access?

问题

Suppose I write a function to parse data from a txt file with with-open and clojure.java.io/reader, and then I wrote another function to call the reader function multiple of times in order to process data, e.g.

(defn grabDataFromFile [file patternString]   
     (let [data (atom [])]
        (with-open [rdr (clojure.java.io/reader file)] 
         (doseq [line (line-seq rdr)] 
           (if  (re-matches  (re-pattern  patternString) line) (swap! data conj line))))
        @data))


(defn myCalculation [file ]
  (let [data1 (grabDataFromFile file "pattern1")
        data2 (grabDataFromFile file "pattern2")
        data3 (grabDataFromFile file "pattern3")]
    ;calculations or processes of data1, data2, data3....))

My question is, inside this myCalculation function, is the underlying code smart enough to open the file just once with clojure reader, and get all data needed in one shot? Or does it open and close the file as many times as number of calls for function grabDataFromFile ? ( In this example, 3)

A follow up question would be, what can I do to speed up if the reader is not smart enough, and if I have to intentionally separate "parser" code with "processing" code?

回答1:

grabDataFromFile will open and close reader (on exit) every time it is called. The underlying code cannot be that smart such that a function can detect the context of its caller without some explicitly provided information.

Make grabDataFromFile to accept another function which is your parser logic that operates on each line (or it could be any function that you want to perform on each line)

(defn grabDataFromFile [file patternString process-fn]   
  (with-open [rdr (clojure.java.io/reader file)] 
    (doseq [line (line-seq rdr)] 
      (process-fn line))))




(defn myCalculation [file]
  (let [patterns [["pattern1" (atom [])]
                  ["pattern2" (atom [])]
                  ["pattern3" (atom [])]]
        pattern-fns (map (fn [[p data]]
                           (fn [line]
                             (if  (re-matches (re-pattern  p) line)                              
                               (swap! data conj line)))) patterns)
        pattern-fn (apply juxt pattern-fns)]
    (grabDataFromFile file pattern-fn)
    ;perform calc on patterns atoms
    ))

来源：https://stackoverflow.com/questions/15563919/is-clojures-read-file-structure-i-e-with-open-and-clojure-java-io-reader-eff