So it turns out you can use block quotes to caption images within a post in Kinja.

This is how you caption an image using the basic block quote.

This is the Emma style of captioned images

Which uses a table.