I have a class library that contains a valid connectionString inside the app.config. Inside that class library I want to use it with
ConfigurationManager.Con
In this case you put the same entry (the in question) in your web app's web.config, ConfigurationManager.ConnectionStrings always looks at the current config, that's a web.config in your case.
web.config