The correct HTML to use for images with captions, is with .
There's no Markdown equivalent for this, so if you're only adding the occasional caption, I'd encourage you to just add that html into your Markdown document:
Lorem ipsum dolor sit amet, consectetur adipiscing elit...
This is my caption text.
Vestibulum eu vulputate magna...
The Markdown spec encourages you to embed HTML in cases like this, so it will display just fine. It's also a lot simpler than messing with plugins.
If you're trying to use other Markdown-y features (like tables, asterisks, etc) to produce captions, then you're just hacking around how Markdown was intended to be used.