exclude characters in HTML input pattern, instead is excluding words

Question

Trying to exclude special characters, I wrote this HTML input pattern

pattern="^[^\\x27]$|[^\\x22#\\[\\]\\*\\^%\\$£@~\\\\]"

I\'m su