Settings.settings generates Settings.Designer.cs which presumably generates app.config which then is copied to output directory as
Settings.settings
Settings.Designer.cs
app.config
You can store your configuration in that file.
The .Net framework will automatically load a config file with the exe-name.config.
If you dont use any configurations in your application, then nothing bad will happen...