If I use the following code without runat=\"server\" the input\'s src works fine and I see the image coming through.
When you add runat="server to that html tag, Asp.Net converts it from string to HtmlControl - in this case of type HtmlInputImage. You can see this happen by adding:
<%= testButton.GetType() %>
Then the only thing you need to do is set the Src-property, which, contrary to other comments, you CAN do in inline aspx - no need for a code-behind file:
<%
testButton.Src = "/content/logo.png";
%>
You need to set the Src-property BEFORE the actual input, which is a bit non-intuitive, the reason is that the code is run at render-time, so if the setting of Src-property is after the control, it is too late.