How to use direct streaming for SOAP with Spring-WS?

Question

We want to enable direct streaming of our payload in webservice endpoints. We have to process a large amount of data and want to stream the data while processing.

We use spring-ws-core, in version 2.0.0 and use the PayloadRootQNameEndpointMapping as endpoint mapper. As message factory we are using the AxiomSoapMessageFactory. We implement the StreamingPayload and the corresponding writeTo(XMLStreamWriter writer) method, which we use to write our payload (According to the spring-ws JIRA ticket, SWS-352).

This works fine without any errors, but we wanted to stream directly! This is apparently not possible. We made an easy test where we streamed some data to evaluate the behaviour.

writer.writeStartElement("exampleResponse")

10000.times
{
    writer.writeStartElement("example")
    writer.writeEndElement()    
}

writer.writeEndElement()

We assumed that this will be directly streamed to the consumer/client, so the soap header is already written to our writer and closes after the endpoint is completed. Unfortunately this is not possible, the stream can not be used directly! The stream is wrapped in a ByteArrayInputStream, found in the spring-ws source.

The implementation of StreamingOMDataSource shows this (can be viewed in springs FishEye). The StreamingOMDataSource calls your StreamingPayload implementation and gives you a writer for this.

public XMLStreamReader getReader() throws XMLStreamException {
   ByteArrayOutputStream bos = new ByteArrayOutputStream();
   serialize(bos, null);

   ByteArrayInputStream bis = new ByteArrayInputStream(bos.toByteArray());
   return StAXUtils.createXMLStreamReader(bis);
}

The method #serialize() creates the XMLStreamWriter with the ByteArrayOutputStream and calls the payload to enable writing, as described above.

public void serialize(OutputStream output, OMOutputFormat format) 
       throws XMLStreamException
{
   XMLStreamWriter streamWriter;
   if ([...]) {
      // Create stream writer with defined charset
   }
   else {
       streamWriter = StAXUtils.createXMLStreamWriter(output);
   }
   serialize(streamWriter);
}

public void serialize(XMLStreamWriter xmlWriter) throws XMLStreamException {
   payload.writeTo(xmlWriter);
   xmlWriter.flush();
}

So this is not usable for me. Is it possible to achieve direct streaming? Any ideas for this? Thank you in advance!

---

Update: I finally created a JIRA ticket (SWS-704) for Spring WS. If you want to see it implemented consider watching/voting it on the JIRA page. Hopefully we get at least an useful reply.

Answer 1

You have also to disable the payload caching:

<bean id="messageFactory" 
      class="org.springframework.ws.soap.axiom.AxiomSoapMessageFactory">
     <property name="payloadCaching" value="false"/>
</bean> 

With this setting we are finally able to perform direct streaming for SOAP with Spring WS!

Answer 2

You cannot (should never) stream the data within a web service, as in continously sending XML over the HTTP connection in a single web-service request over time. You will have to make many single web-service calls, or accumulate several calls into one.

If you need high performance, web-services are not great. But you can hand-optimize simple web-services, it is not that hard. But switching to another transport format would be more 'on the money' if you need higher performance. I'd still keep things over HTTP - especially if you have some authentication requirements.

Answer 3

I can only think of a hack for this - stacks(cxf, spring ws etc) would buffer the entire message as they have to validate the response xml, to be able to calculate the cryptographic keys if security is enabled etc.

So the hack would be to write your own custom servlet/spring controller which would handle this specific response and stream out the soap envelope, then your payload, then the end tag of soap envelope. This is assuming you don't have any WSS requirements.