Optimizing Haskell code

Question

I\'m trying to learn Haskell and after an article in reddit about Markov text chains, I decided to implement Markov text generation first in Python and now in Haskell. Howev

Answer 1

1) I'm not clear on your code. a) You define "fox" but don't use it. Were you meaning for us to try to help you using "fox" instead of reading the file? b) You declare this as "module Markov" then have a 'main' in the module. c) System.Random isn't needed. It does help us help you if you clean code a bit before posting.

2) Use ByteStrings and some strict operations as Don said.

3) Compile with -O2 and use -fforce-recomp to be sure you actually recompiled the code.

4) Try this slight transformation, it works very fast (0.005 seconds). Obviously the input is absurdly small, so you'd need to provide your file or just test it yourself.

{-# LANGUAGE OverloadedStrings, BangPatterns #-}
module Main where

import qualified Data.Map as M
import qualified Data.ByteString.Lazy.Char8 as B


type Database = M.Map (B.ByteString, B.ByteString) (M.Map B.ByteString Int)

train :: [B.ByteString] -> Database
train xs = go xs M.empty
  where
  go :: [B.ByteString] -> Database -> Database
  go (x:y:[]) !m = m
  go (x:y:z:xs) !m =
     let m' =  M.insertWithKey' (\key new old -> M.insertWithKey' (\_ n o -> n + 1) z 1 old) (x, y) (M.singleton z 1) m
     in go (y:z:xs) m'

main = print $ train $ B.words fox

fox="The quick brown fox jumps over the brown fox who is slow jumps over the brown fox who is dead."