The system I work on here was written before .net 2.0 and didn\'t have the benefit of generics. It was eventually updated to 2.0, but none of the code was refactored due to
Depends on how much is out there in your code. If you binding or display large lists in the UI, you would probably see a great gain in performance.
If your ArrayList are just sprinkled about here and there, then it probably wouldn't be a big deal to just get it cleaned up, but also wouldn't impact overall performance very much.
If you are using a lot a ArrayLists throughout your code and it would be a big untertaking to replace them (something that may impact your schedules), then you could adopt a if-you-touch-it-change-it approach.
Main thing is, though, that Generics are a lot easier to read, and are more stable across the app due to the strong typing you get from them. You'll see gains not just from performance, but from code maintainablity and stability. If you can do it quickly, I'd say do it.
If you can get buy-in from the Product Owner, I'd recommend getting it cleaned up. You love your code more afterward.