I have 7 devices plugged into my development machine.
Normally I do adb install and can install to just a single device.
Now
The following command should work:
$ adb devices | tail -n +2 | head -n -1 | cut -f 1 | xargs -I X adb -s X install -r path/to/your/package.apk
adb devices returns the list of devices. Use tail -n +2 to start from the 2nd line and head -n -1 to remove the last blank line at the end. Piping through cut with the default tab delimiter gets us the first column which are the serials.
xargs is used to run the adb command for each serial. Remove the -r option if you are not re-installing.