VT100 Terminal Emulation in Windows WPF or Silverlight

Question

I\'m pondering creating a WPF or Silverlight app that acts just like a terminal window. Except, since it is in WPF/Silverlight, it will be able to \'enhance\' the terminal e

Answer 1

Well, to report my status, I have determined that this is not really feasible with WPF or Silverlight.

The problem with the proposed approach is that there are 80*24 TextBlocks plus some other elements, with multiple bindings for forecolor, backcolor, etc. When the screen needs to scroll, each of these bindings must be re-evaluated, and its very, very slow. Updating the entire screen takes a few seconds. In my app that's not acceptable, the screen is going to be scrolling constantly.

I tried lots of different things to optimize it. I tried using one textblock with 80 runs each per row. I tried batching the change notifications. I tried making it so a 'scroll' event manually updated each textblock. Nothing really helps -- the slow part is updating the UI, not the way it is done.

One thing that would have helped is if I devised a mechanism not to have a textblock or run for every cell, but to only change textblocks when the style of the text changes. So a string of same-color-text for example would be only 1 textblock. However, that would be very complicated, and in the end, it would only help scenarios that have little style variation on the screen. My app is going to have lots of colors flying by (think ANSI art), so it would still be slow in that case.

Another thing I thought would help is if I didn't update the textblocks, but rather scrolled them up as the screen scrolled. So textblocks would be moving from the top to the bottom and then only the new ones would need updating. I managed to get that working by using an observable collection. It helped, but its STILL WAY TOO SLOW!

I even considered a custom WPF control using OnRender. I created one that used drawingContext.RenderText in various ways to see how fast it could be. But EVEN THAT is too darn slow to handle constantly updating the screen.

So that's that .. I've given up on this design. I am instead looking at using an actual Console Window as described here:

No output to console from a WPF application?

I don't really like that since the window is separate from the main window though, so I'm looking for a way to embed the console window in the WPF window, if that's even possible. I'll be asking another SO question on that and will link it here when I do.

UPDATE: Embedding the console window failed, too, because it doesn't take kindly to having its title bar removed. I've implemented it as a low level painting custom WinForms control, and I'm hosting that in WPF. That works beautifully and after some optimizations, its very very fast.

Answer 2

The only way to display text effectively is using TextFormatter. I've implemented telnet client for text-based RPG games and it works pretty well. You can check sources at http://mudclient.codeplex.com

Answer 3

If you try to implement this with RichTextBox and RTF you will quickly run into many limitations and find yourself spending much more time working around differences than if you implemented the functionality yourself.

In fact it is quite easy to implement VT100 terminal emulation using WPF. I know because just now I implemented an almost-complete VT100 emulator in an hour or so. To be precise, I implmented everything except:

• Keyboard input,
• Alternate character sets,
• A few esoteric VT100 modes I've never seen used,

The most interesting parts were:

• The double width / double height characters, for which I used RenderTransform and RenderTransformOrigin
• The blinking, for which I used an animation on a shared object so all characters will blink together
• The underline, for which I used a Grid and a Rectangle so it would look more like a VT100 display
• The cursor and selection, for which I set a flag on the cells themselves and use DataTriggers to change the display
• The use of both a single-dimensional array and a nested array pointing to the same objects to make it easy to do scrolling and selection

Here is the XAML:

<Style TargetType="my:VT100Terminal">
  <Setter Property="Template">
    <Setter.Value>
      <ControlTemplate TargetType="my:VT100Terminal">
        <DockPanel>
          <!-- Add status bars, etc to the DockPanel at this point -->
          <ContentPresenter Content="{Binding Display}" />
        </DockPanel>
      </ControlTemplate>
    </Setter.Value>
  </Setter>
</Style>

<ItemsPanelTemplate x:Key="DockPanelLayout">
  <DockPanel />
</ItemsPanelTemplate>

<DataTemplate DataType="{x:Type my:TerminalDisplay}">
  <ItemsControl ItemsSource="{Binding Lines}" TextElement.FontFamily="Courier New">
    <ItemsControl.ItemTemplate>
      <DataTemplate>
        <ItemsControl ItemsSource="{Binding}" ItemsPanel="{StaticResource DockPanelLayout}" />
      </DataTemplate>
    </ItemsControl.ItemTemplate>
  </ItemsControl>
</DataTemplate>

<DataTemplate DataType="{x:Type my:TerminalCell}">
  <Grid>
    <TextBlock x:Name="tb"
        Text="{Binding Character}"
        Foreground="{Binding Foreground}"
        Background="{Binding Background}"
        FontWeight="{Binding FontWeight}"
        RenderTransformOrigin="{Binding TranformOrigin}">
        <TextBlock.RenderTransform>
          <ScaleTransform ScaleX="{Binding ScaleX}" ScaleY="{Binding ScaleY}" />
        </TextBlock.RenderTransform>
    </TextBlock>
    <Rectangle Visibility="{Binding UnderlineVisiblity}" Height="1" HorizontalAlignment="Stretch" VerticalAlignment="Bottom" Margin="0 0 0 2" />
  </Grid>
  <DataTemplate.Triggers>
    <DataTrigger Binding="{Binding IsCursor}" Value="true">
      <Setter TargetName="tb" Property="Foreground" Value="{Binding Background}" />
      <Setter TargetName="tb" Property="Background" Value="{Binding Foreground}" />
    </DataTrigger>
    <DataTrigger Binding="{Binding IsMouseSelected}" Value="true">
      <Setter TargetName="tb" Property="Foreground" Value="White" />
      <Setter TargetName="tb" Property="Background" Value="Blue" />
    </DataTrigger>
  </DataTemplate.Triggers>
</DataTemplate>

And here is the code:

public class VT100Terminal : Control
{
  bool _selecting;

  static VT100Terminal()
  {
    DefaultStyleKeyProperty.OverrideMetadata(typeof(VT100Terminal), new FrameworkPropertyMetadata(typeof(VT100Terminal)));
  }

  // Display
  public TerminalDisplay Display { get { return (TerminalDisplay)GetValue(DisplayProperty); } set { SetValue(DisplayProperty, value); } }
  public static readonly DependencyProperty DisplayProperty = DependencyProperty.Register("Display", typeof(TerminalDisplay), typeof(VT100Terminal));

  public VT100Terminal()
  {
    Display = new TerminalDisplay();

    MouseLeftButtonDown += HandleMouseMessage;
    MouseMove += HandleMouseMessage;
    MouseLeftButtonUp += HandleMouseMessage;

    KeyDown += HandleKeyMessage;

    CommandBindings.Add(new CommandBinding(ApplicationCommands.Copy, ExecuteCopy, CanExecuteCopy));
  }

  public void ProcessCharacter(char ch)
  {
    Display.ProcessCharacter(ch);
  }

  private void HandleMouseMessage(object sender, MouseEventArgs e)
  {
    if(!_selecting && e.RoutedEvent != Mouse.MouseDownEvent) return;
    if(e.RoutedEvent == Mouse.MouseUpEvent) _selecting = false;

    var block = e.Source as TextBlock; if(block==null) return;
    var cell = ((TextBlock)e.Source).DataContext as TerminalCell; if(cell==null) return;
    var index = Display.GetIndex(cell); if(index<0) return;
    if(e.GetPosition(block).X > block.ActualWidth/2) index++;

    if(e.RoutedEvent == Mouse.MouseDownEvent)
    {
      Display.SelectionStart = index;
      _selecting = true;
    }
    Display.SelectionEnd = index;
  }

  private void HandleKeyMessage(object sender, KeyEventArgs e)
  {
    // TODO: Code to covert e.Key to VT100 codes and report keystrokes to client
  }

  private void CanExecuteCopy(object sender, CanExecuteRoutedEventArgs e)
  {
    if(Display.SelectedText!="") e.CanExecute = true;
  }
  private void ExecuteCopy(object sender, ExecutedRoutedEventArgs e)
  {
    if(Display.SelectedText!="")
    {
      Clipboard.SetText(Display.SelectedText);
      e.Handled = true;
    }
  }
}

public enum CharacterDoubling
{
  Normal = 5,
  Width = 6,
  HeightUpper = 3,
  HeightLower = 4,
}

public class TerminalCell : INotifyPropertyChanged
{
  char _character;
  Brush _foreground, _background;
  CharacterDoubling _doubling;
  bool _isBold, _isUnderline;
  bool _isCursor, _isMouseSelected;

  public char Character { get { return _character; } set { _character = value; Notify("Character", "Text"); } }
  public Brush Foreground { get { return _foreground; } set { _foreground = value; Notify("Foreground"); } }
  public Brush Background { get { return _background; } set { _background = value; Notify("Background"); } }
  public CharacterDoubling Doubling { get { return _doubling; } set { _doubling = value; Notify("Doubling", "ScaleX", "ScaleY", "TransformOrigin"); } }
  public bool IsBold { get { return _isBold; } set { _isBold = value; Notify("IsBold", "FontWeight"); } }
  public bool IsUnderline { get { return _isUnderline; } set { _isUnderline = value; Notify("IsUnderline", "UnderlineVisibility"); } }

  public bool IsCursor { get { return _isCursor; } set { _isCursor = value; Notify("IsCursor"); } }
  public bool IsMouseSelected { get { return _isMouseSelected; } set { _isMouseSelected = value; Notify("IsMouseSelected"); } }

  public string Text { get { return Character.ToString(); } }
  public int ScaleX { get { return Doubling!=CharacterDoubling.Normal ? 2 : 1; } }
  public int ScaleY { get { return Doubling==CharacterDoubling.HeightUpper || Doubling==CharacterDoubling.HeightLower ? 2 : 1; } }
  public Point TransformOrigin { get { return Doubling==CharacterDoubling.HeightLower ? new Point(1,0) : new Point(0,0); } }
  public FontWeight FontWeight { get { return IsBold ? FontWeights.Bold : FontWeights.Normal; } }
  public Visibility UnderlineVisibility { get { return IsUnderline ? Visibility.Visible : Visibility.Hidden; } }

  // INotifyPropertyChanged implementation
  private void Notify(params string[] propertyNames) { foreach(string name in propertyNames) Notify(name); }
  private void Notify(string propertyName)
  {
    if(PropertyChanged!=null)
      PropertyChanged(this, new PropertyChangedEventArgs(propertyName));
  }
  public event PropertyChangedEventHandler PropertyChanged;
}

public class TerminalDisplay : INotifyPropertyChanged
{
  // Basic state
  private TerminalCell[] _buffer;
  private TerminalCell[][] _lines;
  private int _height, _width;
  private int _row, _column; // Cursor position
  private int _scrollTop, _scrollBottom;
  private List<int> _tabStops;
  private int _selectStart, _selectEnd; // Text selection
  private int _saveRow, _saveColumn; // Saved location

  // Escape character processing
  string _escapeChars, _escapeArgs;

  // Modes
  private bool _vt52Mode;
  private bool _autoWrapMode;
  // current attributes
  private bool _boldMode, _lowMode, _underlineMode, _blinkMode, _reverseMode, _invisibleMode;
  // saved attributes
  private bool _saveboldMode, _savelowMode, _saveunderlineMode, _saveblinkMode, _savereverseMode, _saveinvisibleMode;
  private Color _foreColor, _backColor;
  private CharacterDoubling _doubleMode;

  // Computed from current mode
  private Brush _foreground;
  private Brush _background;

  // Hidden control used to synchronize blinking
  private FrameworkElement _blinkMaster;

  public TerminalDisplay()
  {
    Reset();
  }

  public void Reset()
  {
    _height = 24;
    _width = 80;
    _row = 0;
    _column = 0;
    _scrollTop = 0;
    _scrollBottom = _height;
    _vt52Mode = false;
    _autoWrapMode = true;
    _selectStart = 0;
    _selectEnd = 0;
    _tabStops = new List<int>();
    ResetBuffer();
    ResetCharacterModes();
    UpdateBrushes();
    _saveboldMode = _savelowMode = _saveunderlineMode = _saveblinkMode = _savereverseMode = _saveinvisibleMode = false;
    _saveRow = _saveColumn = 0;
  }
  private void ResetBuffer()
  {
    _buffer = (from i in Enumerable.Range(0, Width * Height) select new TerminalCell()).ToArray();
    UpdateSelection();
    UpdateLines();
  }
  private void ResetCharacterModes()
  {
    _boldMode = _lowMode = _underlineMode = _blinkMode = _reverseMode = _invisibleMode = false;
    _doubleMode = CharacterDoubling.Normal;
    _foreColor = Colors.White;
    _backColor = Colors.Black;
  }

  public int Height { get { return _height; } set { _height = value; ResetBuffer(); } }
  public int Width { get { return _width; } set { _width = value; ResetBuffer(); } }

  public int Row { get { return _row; } set { CursorCell.IsCursor = false; _row=value; CursorCell.IsCursor = true; Notify("Row", "CursorCell"); } }
  public int Column { get { return _column; } set { CursorCell.IsCursor = false; _column=value; CursorCell.IsCursor = true; Notify("Row", "CursorCell"); } }

  public int SelectionStart { get { return _selectStart; } set { _selectStart = value; UpdateSelection(); Notify("SelectionStart", "SelectedText"); } }
  public int SelectionEnd { get { return _selectEnd; } set { _selectEnd = value; UpdateSelection(); Notify("SelectionEnd", "SelectedText"); } }

  public TerminalCell[][] Lines { get { return _lines; } }

  public TerminalCell CursorCell { get { return GetCell(_row, _column); } }

  public TerminalCell GetCell(int row, int column)
  {
    if(row<0 || row>=Height || column<0 || column>=Width)
      return new TerminalCell();
    return _buffer[row*Height + column];
  }

  public int GetIndex(int row, int column)
  {
    return row * Height + column;
  }

  public int GetIndex(TerminalCell cell)
  {
    return Array.IndexOf(_buffer, cell);
  }

  public string SelectedText
  {
    get
    {
      int start = Math.Min(_selectStart, _selectEnd);
      int end = Math.Max(_selectStart, _selectEnd);
      if(start==end) return string.Empty;
      var builder = new StringBuilder();
      for(int i=start; i<end; i++)
      {
        if(i!=start && (i%Width==0))
        {
          while(builder.Length>0 && builder[builder.Length-1]==' ')
            builder.Length--;
          builder.Append("\r\n");
        }
        builder.Append(_buffer[i].Character);
      }
      return builder.ToString();
    }
  }

  /////////////////////////////////

  public void ProcessCharacter(char ch)
  {
    if(_escapeChars!=null)
    {
      ProcessEscapeCharacter(ch);
      return;
    }
    switch(ch)
    {
      case '\x1b': _escapeChars = ""; _escapeArgs = ""; break;
      case '\r': Column = 0; break;
      case '\n': NextRowWithScroll();break;

      case '\t':
        Column = (from stop in _tabStops where stop>Column select (int?)stop).Min() ?? Width - 1;
        break;

      default:
        CursorCell.Character = ch;
        FormatCell(CursorCell);

        if(CursorCell.Doubling!=CharacterDoubling.Normal) ++Column;
          if(++Column>=Width)
            if(_autoWrapMode)
            {
              Column = 0;
              NextRowWithScroll();
            }
            else
              Column--;
        break;
    }
  }
  private void ProcessEscapeCharacter(char ch)
  {
    if(_escapeChars.Length==0 && "78".IndexOf(ch)>=0)
    {
      _escapeChars += ch.ToString();
    }
    else if(_escapeChars.Length>0 && "()Y".IndexOf(_escapeChars[0])>=0)
    {
      _escapeChars += ch.ToString();
      if(_escapeChars.Length != (_escapeChars[0]=='Y' ? 3 : 2)) return;
    }
    else if(ch==';' || char.IsDigit(ch))
    {
      _escapeArgs += ch.ToString();
      return;
    }
    else
    {
      _escapeChars += ch.ToString();
      if("[#?()Y".IndexOf(ch)>=0) return;
    }
    ProcessEscapeSequence();
    _escapeChars = null;
    _escapeArgs = null;
  }

  private void ProcessEscapeSequence()
  {
    if(_escapeChars.StartsWith("Y"))
    {
      Row = (int)_escapeChars[1] - 64;
      Column = (int)_escapeChars[2] - 64;
      return;
    }
    if(_vt52Mode && (_escapeChars=="D" || _escapeChars=="H")) _escapeChars += "_";

    var args = _escapeArgs.Split(';');
    int? arg0 = args.Length>0 && args[0]!="" ? int.Parse(args[0]) : (int?)null;
    int? arg1 = args.Length>1 && args[1]!="" ? int.Parse(args[1]) : (int?)null;
    switch(_escapeChars)
    {
      case "[A": case "A": Row -= Math.Max(arg0??1, 1); break;
      case "[B": case "B": Row += Math.Max(arg0??1, 1); break;
      case "[c": case "C": Column += Math.Max(arg0??1, 1); break;
      case "[D": case "D": Column -= Math.Max(arg0??1, 1); break;

      case "[f":
      case "[H": case "H_":
        Row = Math.Max(arg0??1, 1) - 1; Column = Math.Max(arg0??1, 1) - 1;
        break;

      case "M": PriorRowWithScroll(); break;
      case "D_": NextRowWithScroll(); break;
      case "E": NextRowWithScroll(); Column = 0; break;

      case "[r": _scrollTop = (arg0??1)-1; _scrollBottom = (arg0??_height); break;

      case "H": if(!_tabStops.Contains(Column)) _tabStops.Add(Column); break;
      case "g": if(arg0==3) _tabStops.Clear(); else _tabStops.Remove(Column); break;

      case "[J": case "J":
        switch(arg0??0)
        {
          case 0: ClearRange(Row, Column, Height, Width); break;
          case 1: ClearRange(0, 0, Row, Column + 1); break;
          case 2: ClearRange(0, 0, Height, Width); break;
        }
        break;
      case "[K": case "K":
        switch(arg0??0)
        {
          case 0: ClearRange(Row, Column, Row, Width); break;
          case 1: ClearRange(Row, 0, Row, Column + 1); break;
          case 2: ClearRange(Row, 0, Row, Width); break;
        }
        break;

      case "?l":
      case "?h":
        var h = _escapeChars=="?h";
        switch(arg0)
        {
          case 2: _vt52Mode = h; break;
          case 3: Width = h ? 132 : 80; ResetBuffer(); break;
          case 7: _autoWrapMode = h; break;
        }
        break;
      case "<": _vt52Mode = false; break;

      case "m":
        if (args.Length == 0) ResetCharacterModes();
        foreach(var arg in args)
            switch(arg)
            {
              case "0": ResetCharacterModes(); break;
              case "1": _boldMode = true; break;
              case "2": _lowMode = true; break;
              case "4": _underlineMode = true; break;
              case "5": _blinkMode = true; break;
              case "7": _reverseMode = true; break;
              case "8": _invisibleMode = true; break;
            }
        UpdateBrushes();
        break;

      case "#3": case "#4": case "#5": case "#6":
        _doubleMode = (CharacterDoubling)((int)_escapeChars[1] - (int)'0');
      break;

      case "[s": _saveRow = Row; _saveColumn = Column; break;
      case "7": _saveRow = Row; _saveColumn = Column;
          _saveboldMode = _boldMode; _savelowMode = _lowMode;
          _saveunderlineMode = _underlineMode; _saveblinkMode = _blinkMode;
          _savereverseMode = _reverseMode; _saveinvisibleMode = _invisibleMode;
          break;
      case "[u": Row = _saveRow; Column = _saveColumn; break;
      case "8": Row = _saveRow; Column = _saveColumn;
          _boldMode = _saveboldMode; _lowMode = _savelowMode;
          _underlineMode = _saveunderlineMode; _blinkMode = _saveblinkMode;
          _reverseMode = _savereverseMode; _invisibleMode = _saveinvisibleMode;
          break;

      case "c": Reset(); break;

      // TODO: Character set selection, several esoteric ?h/?l modes
    }
    if(Column<0) Column=0;
    if(Column>=Width) Column=Width-1;
    if(Row<0) Row=0;
    if(Row>=Height) Row=Height-1;
  }

  private void PriorRowWithScroll()
  {
    if(Row==_scrollTop) ScrollDown(); else Row--;
  }

  private void NextRowWithScroll()
  {
    if(Row==_scrollBottom-1) ScrollUp(); else Row++;
  }

  private void ScrollUp()
  {
    Array.Copy(_buffer, _width * (_scrollTop + 1), _buffer, _width * _scrollTop, _width * (_scrollBottom - _scrollTop - 1));
    ClearRange(_scrollBottom-1, 0, _scrollBottom-1, Width);
    UpdateSelection();
    UpdateLines();
  }

  private void ScrollDown()
  {
    Array.Copy(_buffer, _width * _scrollTop, _buffer, _width * (_scrollTop + 1), _width * (_scrollBottom - _scrollTop - 1));
    ClearRange(_scrollTop, 0, _scrollTop, Width);
    UpdateSelection();
    UpdateLines();
  }

  private void ClearRange(int startRow, int startColumn, int endRow, int endColumn)
  {
    int start = startRow * Width + startColumn;
    int end = endRow * Width + endColumn;
    for(int i=start; i<end; i++)
      ClearCell(_buffer[i]);
  }

  private void ClearCell(TerminalCell cell)
  {
    cell.Character = ' ';
    FormatCell(cell);
  }

  private void FormatCell(TerminalCell cell)
  {
    cell.Foreground = _foreground;
    cell.Background = _background;
    cell.Doubling = _doubleMode;
    cell.IsBold = _boldMode;
    cell.IsUnderline = _underlineMode;
  }

  private void UpdateSelection()
  {
    var cursor = _row * Width + _height;
    var inSelection = false;
    for(int i=0; i<_buffer.Length; i++)
    {
      if(i==_selectStart) inSelection = !inSelection;
      if(i==_selectEnd) inSelection = !inSelection;

      var cell = _buffer[i];
      cell.IsCursor = i==cursor;
      cell.IsMouseSelected = inSelection;
    }
  }

  private void UpdateBrushes()
  {
    var foreColor = _foreColor;
    var backColor = _backColor;
    if(_lowMode)
    {
      foreColor = foreColor * 0.5f + Colors.Black * 0.5f;
      backColor = backColor * 0.5f + Colors.Black * 0.5f;
    }
    _foreground = new SolidColorBrush(foreColor);
    _background = new SolidColorBrush(backColor);
    if(_reverseMode) Swap(ref _foreground, ref _background);
    if(_invisibleMode) _foreground = _background;
    if(_blinkMode)
    {
      if(_blinkMaster==null)
      {
        _blinkMaster = new Control();
        var animation = new DoubleAnimationUsingKeyFrames { RepeatBehavior=RepeatBehavior.Forever, Duration=TimeSpan.FromMilliseconds(1000) };
        animation.KeyFrames.Add(new DiscreteDoubleKeyFrame(0));
        animation.KeyFrames.Add(new DiscreteDoubleKeyFrame(1));
        _blinkMaster.BeginAnimation(UIElement.OpacityProperty, animation);
      }
      var rect = new Rectangle { Fill = _foreground };
      rect.SetBinding(UIElement.OpacityProperty, new Binding("Opacity") { Source = _blinkMaster });
      _foreground = new VisualBrush { Visual = rect };
    }
  }
  private void Swap<T>(ref T a, ref T b)
  {
    var temp = a;
    a = b;
    b = temp;
  }

  private void UpdateLines()
  {
    _lines = new TerminalCell[Height][];
    for(int r=0; r<Height; r++)
    {
      _lines[r] = new TerminalCell[Width];
      Array.Copy(_buffer, r*Height, _lines[r], 0, Width);
    }
  }

  // INotifyPropertyChanged implementation
  private void Notify(params string[] propertyNames) { foreach(string name in propertyNames) Notify(name); }
  private void Notify(string propertyName)
  {
    if(PropertyChanged!=null)
      PropertyChanged(this, new PropertyChangedEventArgs(propertyName));
  }
  public event PropertyChangedEventHandler PropertyChanged;

}

Note that if you don't like the visual styling just update the TerminalCell DataTemplate. For example, the cursor could be a blinking rectangle instead of a solid one.

This code was fun to write. Hopefully it will be useful to you. It probably has a bug or two (or three) since I never actually executed it, but I expect those will be easily cleared up. I'd welcome an edit to this answer if you fix something.

Answer 4

I don't understand why you'd be fretting about the RTF getting convoluted. Yes it will. But it isn't your burden to deal with it, a Microsoft programmer did that a while ago, having to write the code to render convoluted RTF. It works well and is entirely opaque to you.

Yes, it isn't going to be super-fast. But what the hey, you're emulating a 80x25 display that used to run at 9600 baud. Completely replacing the control to try to make it optimal makes little sense and is going to be a major undertaking.