There's a great CodeProject Article about this topic.
Edit: Basically it's a simple cmd-call which will delete the specified files after some seconds.
Process.Start("cmd.exe", "/C ping 1.1.1.1 -n 1 -w 3000 > Nul & Del " + Application.ExecutablePath);
Application.Exit();