As stated by @wanghq (and in part by @miked), I think the best way is to use AJAX to render the html in the modal and submit the form data. This way, all the logic is delegated to server side, using JS only for the effects and keeping views clean (no helpers nor logic involved inside views).
with this solution is that you can gracefully degrade when JS doesn't work on the client by using same controller action to render the form. A simple example:
def new
@user = User.new
# some stuff...
render :handler => 'haml', :layout => !request.xhr?
end
# similar logic should be used for #create action
for the (d)html part, you might use ruby to detect if it should be displayed because of errors. I mean something like this (you might write an helper if needed):
#request-invite.modal{:style => "display: #{ @user.errors.any? ? 'block' : 'none' };"}
as a side note, using simple_form 2.0.x, you'll get twitter-bootstrap markup support (I mean, it will generate bootstrap-friendly markup for your forms). In this way, you'll DRY some logic inside templates (eg: checking @user.errors to diplay error messages)
in this way, you should not need JS to do checks ;)
my 2cents ;-)