Transfer large amount of data in WCF service

Question

I have created a web service in WCF which returns more than 54000 data-rows with 10 data in each row. I have used the wsHttpBinding for communication. The s

Answer 1

After a lot of investigation finnally i got the solution. Actually a number of things need to be changed.

The following changes needed to be done in Server-side.

First I had to set a maxRequestLength to a larger value in my httpRuntime element to run the request for longer period.

<system.web>    
<httpRuntime maxRequestLength="102400" />
</system.web>

Second i introduced netTcpBinding binnding with custom changes on maxBufferSize, maxBufferPoolSize, maxReceivedMessageSize with a large value of 2147483647.

<binding name="myNetTcpBinding" 
maxBufferPoolSize="2147483647" 
maxBufferSize="524288" 
maxReceivedMessageSize="2147483647">

Third add maxItemsInObjectGraph in both of the serviceBehaviors and endpointBehaviors like bellow (dont forget to mention the behaviour names in the service and endpoint node)

    <behaviors>
      <serviceBehaviors>        
        <behavior name="myNetTcpBehaviour">
          <serviceMetadata httpGetEnabled="true"/>
          <serviceDebug includeExceptionDetailInFaults="true"/>
          <dataContractSerializer maxItemsInObjectGraph="2147483647"/>
        </behavior>
      </serviceBehaviors>
      <endpointBehaviors>
        <behavior name="myNetTcpEndPointBehaviour">
          <dataContractSerializer maxItemsInObjectGraph="2147483647"/>
        </behavior>
      </endpointBehaviors>
    </behaviors>

Finally my server configuration looks like this

<system.web>    
    <httpRuntime maxRequestLength="102400" />
</system.web>


  <system.serviceModel>
    <bindings>
      <wsHttpBinding>
        <binding name="MyWsHttpBinding" />
      </wsHttpBinding>
      <netTcpBinding>
        <binding name="myNetTcpBinding"
                 closeTimeout="00:01:00"
                 openTimeout="00:01:00"
                 receiveTimeout="00:10:00"
                 sendTimeout="00:01:00"
                 transactionFlow="false"
                 transferMode="Buffered"
                 transactionProtocol="OleTransactions"
                 hostNameComparisonMode="StrongWildcard"
                 listenBacklog="10"
                 maxBufferPoolSize="2147483647"
                 maxBufferSize="524288"
                 maxConnections="10"
                 maxReceivedMessageSize="2147483647">
          <readerQuotas maxDepth="32"
                        maxStringContentLength="8192"
                        maxArrayLength="16384"
                        maxBytesPerRead="4096"
                        maxNameTableCharCount="16384" />
          <reliableSession ordered="true"
                           inactivityTimeout="00:10:00"
                           enabled="false" />
          <security mode="Transport">
            <transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" />
          </security>
        </binding>
      </netTcpBinding>
    </bindings>
    <services>
      <service name="AdminService" behaviorConfiguration="myNetTcpBehaviour">
        <endpoint address="AdminSrv" 
                  binding="netTcpBinding" 
                  bindingConfiguration="myNetTcpBinding"
                  contract="IAdminService"
                  behaviorConfiguration="myNetTcpEndPointBehaviour"/>

        <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />
        <host>
          <baseAddresses>
            <add baseAddress="/Bus/IRfotoWCF" />
          </baseAddresses>
        </host>
      </service>
    <behaviors>
      <serviceBehaviors>        
        <behavior name="myNetTcpBehaviour">
          <serviceMetadata httpGetEnabled="true"/>
          <serviceDebug includeExceptionDetailInFaults="true"/>
          <dataContractSerializer maxItemsInObjectGraph="2147483647"/>
        </behavior>
      </serviceBehaviors>
      <endpointBehaviors>
        <behavior name="myNetTcpEndPointBehaviour">
          <dataContractSerializer maxItemsInObjectGraph="2147483647"/>
        </behavior>
      </endpointBehaviors>
    </behaviors>
    <serviceHostingEnvironment multipleSiteBindingsEnabled="true"></serviceHostingEnvironment>
  </system.serviceModel>

Now on the client-side configuratioin you need to change the maxBufferSize="2147483647" maxBufferPoolSize="524288" maxReceivedMessageSize="2147483647"

and also you need to add maxItemsInObjectGraph="2147483647" in endpoint behaviour configuration.

        <endpointBehaviors>
            <behavior name="myEndPointBehavior">
                <dataContractSerializer maxItemsInObjectGraph="2147483647" />
            </behavior>
        </endpointBehaviors>

Now i can transmit 30000 rows within 5.30 min where the query executed for 10 sec so the transmission time is 5.20 min - still a lot.

Feel free to comment and any suggestion for improvement.

Answer 2

If you look at the binding details they do not match entirely on the server and that of the client side. The attributes for maxBufferSize, maxBufferPoolSize, maxReceivedMessageSize are to be defined in the server side as well. And then you need to put the values according to the size you are looking at.

Answer 3

Instead of using for loop over WCF for bulky data , Use user defined table type ( if you are using SQL) . It will reduce the time from 6 min to 15-20 sec.