Large-scale design in Haskell? [closed]

Answer 1

I did learn structured functional programming the first time with this book. It may not be exactly what you are looking for, but for beginners in functional programming, this may be one of the best first steps to learn to structure functional programs - independant of the scale. On all abstraction levels, the design should always have clearly arranged structures.

The Craft of Functional Programming

http://www.cs.kent.ac.uk/people/staff/sjt/craft2e/

Answer 2

Perhaps you have to go an step back and think of how to translate the description of the problem to a design in the first place. Since Haskell is so high level, it can capture the description of the problem in the form of data structures , the actions as procedures and the pure transformation as functions. Then you have a design. The development start when you compile this code and find concrete errors about missing fields, missing instances and missing monadic transformers in your code, because for example you perform a database Access from a library that need a certain state monad within an IO procedure. And voila, there is the program. The compiler feed your mental sketches and gives coherence to the design and the development.

In such a way you benefit from the help of Haskell since the beginning, and the coding is natural. I would not care to do something "functional" or "pure" or enough general if what you have in mind is a concrete ordinary problem. I think that over-engineering is the most dangerous thing in IT. Things are different when the problem is to create a library that abstract a set of related problems.

Answer 3

Designing large programs in Haskell is not that different from doing it in other languages. Programming in the large is about breaking your problem into manageable pieces, and how to fit those together; the implementation language is less important.

That said, in a large design it's nice to try and leverage the type system to make sure you can only fit your pieces together in a way that is correct. This might involve newtype or phantom types to make things that appear to have the same type be different.

When it comes to refactoring the code as you go along, purity is a great boon, so try to keep as much of the code as possible pure. Pure code is easy to refactor, because it has no hidden interaction with other parts of your program.

Answer 4

I'm currently writing a book with the title "Functional Design and Architecture". It provides you with a complete set of techniques how to build a big application using pure functional approach. It describes many functional patterns and ideas while building an SCADA-like application 'Andromeda' for controlling spaceships from scratch. My primary language is Haskell. The book covers:

• Approaches to architecture modelling using diagrams;
• Requirements analysis;
• Embedded DSL domain modelling;
• External DSL design and implementation;
• Monads as subsystems with effects;
• Free monads as functional interfaces;
• Arrowised eDSLs;
• Inversion of Control using Free monadic eDSLs;
• Software Transactional Memory;
• Lenses;
• State, Reader, Writer, RWS, ST monads;
• Impure state: IORef, MVar, STM;
• Multithreading and concurrent domain modelling;
• GUI;
• Applicability of mainstream techniques and approaches such as UML, SOLID, GRASP;
• Interaction with impure subsystems.

You may get familiar with the code for the book here, and the 'Andromeda' project code.

I expect to finish this book at the end of 2017. Until that happens, you may read my article "Design and Architecture in Functional Programming" (Rus) here.

UPDATE

I shared my book online (first 5 chapters). See post on Reddit

Answer 5

I talk a bit about this in Engineering Large Projects in Haskell and in the Design and Implementation of XMonad. Engineering in the large is about managing complexity. The primary code structuring mechanisms in Haskell for managing complexity are:

The type system

• Use the type system to enforce abstractions, simplifying interactions.
• Enforce key invariants via types
  • (e.g. that certain values cannot escape some scope)
  • That certain code does no IO, does not touch the disk
• Enforce safety: checked exceptions (Maybe/Either), avoid mixing concepts (Word, Int, Address)
• Good data structures (like zippers) can make some classes of testing needless, as they rule out e.g. out of bounds errors statically.

The profiler

• Provide objective evidence of your program's heap and time profiles.
• Heap profiling, in particular, is the best way to ensure no unnecessary memory use.

Purity

• Reduce complexity dramatically by removing state. Purely functional code scales, because it is compositional. All you need is the type to determine how to use some code -- it won't mysteriously break when you change some other part of the program.
• Use lots of "model/view/controller" style programming: parse external data as soon as possible into purely functional data structures, operate on those structures, then once all work is done, render/flush/serialize out. Keeps most of your code pure

Testing

• QuickCheck + Haskell Code Coverage, to ensure you are testing the things you can't check with types.
• GHC + RTS is great for seeing if you're spending too much time doing GC.
• QuickCheck can also help you identify clean, orthogonal APIs for your modules. If the properties of your code are difficult to state, they're probably too complex. Keep refactoring until you have a clean set of properties that can test your code, that compose well. Then the code is probably well designed too.

Monads for Structuring

• Monads capture key architectural designs in types (this code accesses hardware, this code is a single-user session, etc.)
• E.g. the X monad in xmonad, captures precisely the design for what state is visible to what components of the system.

Type classes and existential types

• Use type classes to provide abstraction: hide implementations behind polymorphic interfaces.

Concurrency and parallelism

• Sneak par into your program to beat the competition with easy, composable parallelism.

Refactor

• You can refactor in Haskell a lot. The types ensure your large scale changes will be safe, if you're using types wisely. This will help your codebase scale. Make sure that your refactorings will cause type errors until complete.

Use the FFI wisely

• The FFI makes it easier to play with foreign code, but that foreign code can be dangerous.
• Be very careful in assumptions about the shape of data returned.

Meta programming

• A bit of Template Haskell or generics can remove boilerplate.

Packaging and distribution

• Use Cabal. Don't roll your own build system. (EDIT: Actually you probably want to use Stack now for getting started.).
• Use Haddock for good API docs
• Tools like graphmod can show your module structures.
• Rely on the Haskell Platform versions of libraries and tools, if at all possible. It is a stable base. (EDIT: Again, these days you likely want to use Stack for getting a stable base up and running.)

Warnings

• Use -Wall to keep your code clean of smells. You might also look at Agda, Isabelle or Catch for more assurance. For lint-like checking, see the great hlint, which will suggest improvements.

With all these tools you can keep a handle on complexity, removing as many interactions between components as possible. Ideally, you have a very large base of pure code, which is really easy to maintain, since it is compositional. That's not always possible, but it is worth aiming for.

In general: decompose the logical units of your system into the smallest referentially transparent components possible, then implement them in modules. Global or local environments for sets of components (or inside components) might be mapped to monads. Use algebraic data types to describe core data structures. Share those definitions widely.

Answer 6

Gabriel's blog post Scalable program architectures might be worth a mention.

Haskell design patterns differ from mainstream design patterns in one important way:

• Conventional architecture: Combine a several components together of type A to generate a "network" or "topology" of type B

• Haskell architecture: Combine several components together of type A to generate a new component of the same type A, indistinguishable in character from its substituent parts

It often strikes me that an apparently elegant architecture often tends to fall out of libraries that exhibit this nice sense of homogeneity, in a bottom-up sort of way. In Haskell this is especially apparent - patterns that would traditionally be considered "top-down architecture" tend to be captured in libraries like mvc, Netwire and Cloud Haskell. That is to say, I hope this answer will not be interpreted as an attempt replace any of the others in this thread, just that structural choices can and should ideally be abstracted away in libraries by domain experts. The real difficulty in building large systems, in my opinion, is evaluating these libraries on their architectural "goodness" versus all of your pragmatic concerns.

As liminalisht mentions in the comments, The category design pattern is another post by Gabriel on the topic, in a similar vein.