The rm function does not have an exclude function. Instead you can use rm *.txt to delete only text files, rm *.pdf to delete only PDF files, rm a* to delete only files starting with the letter a, rm *d to delete only files ending with the letter d, etc.
Using this method, you can relatively quickly delete all the files except the ones you don't want to.