For a WPF application which will need 10 - 20 small icons and images for illustrative purposes, is storing these in the assembly as embedded resources the right way to go?>
Full description how to use resources: WPF Application Resource, Content, and Data Files
And how to reference them, read "Pack URIs in WPF".
In short, there is even means to reference resources from referenced/referencing assemblies.