I need to create 5 methods on the server side, which will work with binary data. The remote clients are applet and JavaScript. The Client will send files to the server, and
First, you're speaking in terms of two different paradigms. It's kinda apples-and-oranges.
REST is a style of service that uses HTTP operations (GET, PUT, etc.) to read and write the state of resources. Think of resources as "nouns" and "things".
Servlet, on the other hand, is a software specification originally provided by Sun Microsystems for connecting HTTP requests to custom Java code. Servlets often speak in terms of method calls: "verbs" and "actions".
Since your question implies that you are looking to deal with input->output methods, just a plain servlet should do the job.