Just to complement the answer given by Thibaut Barrère.
I used
wget -r -nH --cut-dirs=5 -nc ftp://user:pass@server//absolute/path/to/directory
Note the double slash after the server name. If you don't put an extra slash the path is relative to the home directory of user.
-nH avoids the creation of a directory named after the server name
-nc avoids creating a new file if it already exists on the destination (it is just skipped)
--cut-dirs=5 allows to take the content of /absolute/path/to/directory and to put it in the directory where you launch wget. The number 5 is used to filter out the 5 components of the path. The double slash means an extra component.