I am posting this in the hope that it will save someone else the time and effort of figuring this one out:
My current setup is VS2015 again
My case was that I needed to add NuGet reference directly in the test project (my custom attribute was situated in another project and dll was not copied to the test project).