I think this is a common problem. I have a form where I show/hide fields dynamically using jQuery, depending on some radio buttons.
I have RequiredFieldValidator\'s
If I remember correctly there's a function called ValidatorEnable(validatorClientId, isEnabled) that allows you to disable/enable the ASP.NET validators via javascript. You could use jQuery right before your form submit to disable all your invisible validators.
There's some documentation about client side API available from the validators here http://msdn.microsoft.com/en-us/library/aa479045.aspx