You could just use the HTML entity, if it's for HTML.
-- Edit
'
-- Edit
Just to make this post not wrong, as everyone else has noted, escaping works fine :)
string s = t.ToString("MMM d \\'yy 'at' H:mmm");
And that's the last time I don't test something based on who is posting :)