I\'ve recently discovered that Imagick can support color profiles and thus produce images of better quality compared to GD (see this question / answer for more details), so
As there is not much support for ICM in browsers the profiles are essentially a waste of bandwidth. Thus if your images are in sRGB you can safely trash the profile, otherwise it is better to convert an image into sRGB and trash its profile afterwards.
The reason for removing a profile of sRGB images is that sRGB is effectively a standard on the Internet, on computers, and on printers, and even Firefox applies sRGB color profile to untagged images.
There is another reason for removing all profiles altogether, thou I'm not sure if it applies to your case: if you're planning to mix images with embedded profiles with other profile-less images, e.g. GIF images, which cannot contain a profile by definition, you'll end up with a messy result on an ICC-enabled browser. It'll render some images as per their embedded color space and other with a some other color profile, which leads to a situation in which you'll see a boundary between an image with an embedded ICC profile with a solid background color adjoining other profile-less image with the same color background color. Even if you manage to get a profile for every image on your page, there are a lot of users who use ancient ICC-disabled browsers.
Bottom line: color profiles are evil. Only use them if you actually need them.
What I said is right only if you target your site for a widest audience possible. Otherwise YMMV.