Best way to replace tokens in a large text template

Question

I have a large text template which needs tokenized sections replaced by other text. The tokens look something like this: ##USERNAME##. My first instinct is just to use Stri

Answer 1

Well, depending on how many variables you have in your template, how many templates you have, etc. this might be a work for a full template processor. The only one I've ever used for .NET is NVelocity, but I'm sure there must be scores of others out there, most of them linked to some web framework or another.

Answer 2

FastReplacer implements token replacement in O(n*log(n) + m) time and uses 3x the memory of the original string.

FastReplacer is good for executing many Replace operations on a large string when performance is important.

The main idea is to avoid modifying existing text or allocating new memory every time a string is replaced.

We have designed FastReplacer to help us on a project where we had to generate a large text with a large number of append and replace operations. The first version of the application took 20 seconds to generate the text using StringBuilder. The second improved version that used the String class took 10 seconds. Then we implemented FastReplacer and the duration dropped to 0.1 seconds.

Answer 3

System.Text.RegularExpressions.Regex.Replace() is what you seek - IF your tokens are odd enough that you need a regex to find them.

Some kind soul did some performance testing, and between Regex.Replace(), String.Replace(), and StringBuilder.Replace(), String.Replace() actually came out on top.

Answer 4

If your template is large and you have lots of tokens, you probably don't want walk it and replace the token in the template one by one as that would result in an O(N * M) operation where N is the size of the template and M is the number of tokens to replace.

The following method accepts a template and a dictionary of the keys value pairs you wish to replace. By initializing the StringBuilder to slightly larger than the size of the template, it should result in an O(N) operation (i.e. it shouldn't have to grow itself log N times).

Finally, you can move the building of the tokens into a Singleton as it only needs to be generated once.

static string SimpleTemplate(string template, Dictionary<string, string> replacements)
{
   // parse the message into an array of tokens
   Regex regex = new Regex("(##[^#]+##)");
   string[] tokens = regex.Split(template);

   // the new message from the tokens
   var sb = new StringBuilder((int)((double)template.Length * 1.1));
   foreach (string token in tokens)
      sb.Append(replacements.ContainsKey(token) ? replacements[token] : token);

   return sb.ToString();
}