I have been trying to wrap my head around functional programming for a while now? I have looked up lambda calculus, LISP, OCML, F# and even combinatorial logic but the main
Given that most programs have some effects on the outside world (writing to files, modifying data in a database...) programs as whole are rarely side-effect free. Outside of academic exercises, there is no point in even trying.
But programs are assembled out of building blocks (subroutine, function, method, call it what you want), and pure functions make for very well-behaved building blocks.
Most functional programming languages do not require functions to be pure, although good functional programmers will try to make as many of their functions pure as is feasible and practical, in order to reap the benefits of referential transparency.
Haskell goes further. Every part of a Haskell Programm is pure (at least in the absence of sins such as "unsafePerformIO"). All functions that you write in Haskell are pure.
Side-effects are introduced through monads. They can be used to introduce a sort of "shopping-list -- shopper"-separation. Essentially your program writes a shopping list (which is just data and can be manipulated in a pure fashion), while the language runtime interprets the shopping list and does the effectful shopping. All your code is pure and friendly to equational reasoning and such, whereas the impure code is provided by the compiler-writers.