I have a problem about select function when I worked on a Linux socket program. The select function worked fine as the man page says if the client connected the server side
You have to fill your FD_SET at each iteration. The best way to do so is to maintain a collection of your FDs somewhere and put the one that you need for the select call in a temporary FD_SET.
If you need to handle a lot of clients, you might have to change the FD_SETSIZE (in /usr/include/sys/select.h) macro.
Happy network programming :)