I have the exact opposite Problem of this question: Visual Studio Code always reopens previous file or folder
VSCode never seems to remembery my previously opened fi
They added this functionality in November 2016.
Here is the setting for changing hot exit behavior:
"files.hotExit": "onExit"