In the following snippet, you can see my two collatz functions I wrote in Haskell. For the recursive application I used parentheses in the first example (collatz) to get the
$ has lower precedence then : (and also anything else) so your function is parsing as
(n : collatz') $ (n `div` 2)
This leads to your type error. The second argument of : expects a list but you are passing the collatz function instead.
If you still want to avoid the parenthesis around the 3n+1 part you can do something like the following
(n:) . collatz' $ n `div` 2
n : (collatz' $ n `div` 2)
although these are not necessarily cleaner then the original. In case you are wondering, the (n:) in the first example is a syntactic sugar for \x -> n : x