Clojure - walk with path

Question

I am looking for a function similar to those in clojure.walk that have an inner function that takes as argument :

• not a key and a value, as is the case with the clojure.walk/walk function
• but the vector of keys necessary to access a value from the top-level data structure.
• recursively traverses all data

Example :

;; not good since it takes `[k v]` as argument instead of `[path v]`, and is not recursive.
user=> (clojure.walk/walk (fn [[k v]] [k (* 10 v)]) identity {:a 1 :b {:c 2}})
;; {:a 10, :c 30, :b 20}

;; it should receive as arguments instead :
[[:a] 1]
[[:b :c] 2]

Note:

• It should work with arrays too, using the keys 0, 1, 2... (just like in get-in).
• I don't really care about the outer parameter, if that allows to simplify the code.

Answer 1

Currently learning clojure, I tried this as an exercise. I however found it quite tricky to implement it directly as a walk down the tree that applies the inner function as it goes.

To achieve the result you are looking for, I split the task in 2:

• First transform the nested structure into a dictionary with the path as key, and the value,
• Then map the inner function over, or reduce with the outer function.

My implementation:

;; Helper function to have vector's indexes work like for get-in
(defn- to-indexed-seqs [coll]
  (if (map? coll)
    coll
    (map vector (range) coll)))

;; Flattening the tree to a dict of (path, value) pairs that I can map over
;; user>  (flatten-path [] {:a {:k1 1 :k2 2} :b [1 2 3]})
;; {[:a :k1] 1, [:a :k2] 2, [:b 0] 1, [:b 1] 2, [:b 2] 3}
(defn- flatten-path [path step]
  (if (coll? step)
    (->> step
         to-indexed-seqs
         (map (fn [[k v]] (flatten-path (conj path k) v)))
         (into {}))
    [path step]))

;; Some final glue
(defn path-walk [f coll]
  (->> coll
      (flatten-path [])
      (map #(apply f %))))

;; user> (println (clojure.string/join "\n" (path-walk #(str %1 " - " %2) {:a {:k1 1 :k2 2} :b [1 2 3]})))
;; [:a :k1] - 1
;; [:a :k2] - 2
;; [:b 0] - 1
;; [:b 1] - 2
;; [:b 2] - 3

Answer 2

It turns out that Stuart Halloway published a gist that could be of some use (it uses a protocol, which makes it extensible as well) :

(ns user)

(def app
  "Intenal Helper"
  (fnil conj []))

(defprotocol PathSeq
  (path-seq* [form path] "Helper for path-seq"))

(extend-protocol PathSeq
  java.util.List
  (path-seq*
   [form path]
   (->> (map-indexed
         (fn [idx item]
           (path-seq* item (app path idx)))
         form)
        (mapcat identity)))

  java.util.Map
  (path-seq*
   [form path]
   (->> (map
         (fn [[k v]]
           (path-seq* v (app path k)))
         form)
        (mapcat identity)))

  java.util.Set
  (path-seq*
   [form path]
   (->> (map
         (fn [v]
           (path-seq* v (app path v)))
         form)
        (mapcat identity)))


  java.lang.Object
  (path-seq* [form path] [[form path]])

  nil
  (path-seq* [_ path] [[nil path]]))

(defn path-seq
  "Returns a sequence of paths into a form, and the elements found at
   those paths.  Each item in the sequence is a map with :path
   and :form keys. Paths are built based on collection type: lists
   by position, maps by key, and sets by value, e.g.

   (path-seq [:a [:b :c] {:d :e} #{:f}])

   ({:path [0], :form :a}
    {:path [1 0], :form :b}
    {:path [1 1], :form :c}
    {:path [2 :d], :form :e}
    {:path [3 :f], :form :f})
   "
  [form]
  (map
   #(let [[form path] %]
      {:path path :form form})
   (path-seq* form nil)))

(comment
  (path-seq [:a [:b :c] {:d :e} #{:f}])

  ;; finding nils hiding in data structures:
  (->> (path-seq [:a [:b nil] {:d :e} #{:f}])
       (filter (comp nil? :form)))

  ;; finding a nil hiding in a Datomic transaction
  (->> (path-seq {:db/id 100
                  :friends [{:firstName "John"}
                            {:firstName nil}]})
       (filter (comp nil? :form)))


  )

Note : in my case I could also have used Specter, so if you are reading this, you may want to check it out as well.

Answer 3

There is also https://github.com/levand/contextual/

(def node (:b (first (root :a))))

(= node {:c 1}) ;; => true

(c/context node) ;; => [:a 0 :b]