Returning Large volume of records from SOAP based WCF service

Objective

In this article I will explain; How to return large volume of data (around 50000 records) from SOAP based WCF service to any client. I will show; what service configuration setting and client configuration setting is needed to handle large volume of data returning for WCF service.

Follow the below steps,

Step 1: Create a WCF service

.To creates WCF service; select File -> New -> Project-> Web -> WCF Application.

Service will contain

One Operation contract. This function will pull large data from database using stored procedure.
One Data Contract. This class will act as Data Transfer object (DTO) between client and service.
basicHttpBinding is being used in the service. You are free to use any binding as of your requirement.

Data Transfer Class

[DataContract]

public class DTOClass
{

[DataMember]
public string SystemResourceId
{
get;set;

}
[DataMember]
public string SystemResourceName
{
get;
set;
}
[DataMember]
public string Created
{
get;
set;}
[DataMember]
public string Creater
{
get;set;
}
[DataMember]
public string Updated
{
get;set;
}
[DataMember]
public string Updater
{
get;
set;
}

Name of DTO class is DTOClass. You can give any name of your choice.
There are 6 string properties.
All properties are attributed with DataMember.

Contract

[ServiceContract]

[ServiceKnownType(typeof(DTOClass))]
public interfaceIService1
{
[OperationContract]
List<DTOClass> GetData();
}

Service is returning List of DTOClass.
To get serialized at run time, making sure Data Contract is known to contract by using known type.

Service Implementation

public class Service1 : IService1{

string cs = @”Data Source=xxxserver;Initial Catalog=Sampledatabase;User=dhananjay;Password=dhananjay”;
List<DTOClass> restDto = new List<DTOClass>();
DTOClass dto;
public List<DTOClass> GetData()
{
SqlConnection con = new SqlConnection(cs);
SqlCommand cmd = new SqlCommand(“GetAllSystemResourceDetails”, con);
cmd.CommandType = CommandType.StoredProcedure;
DataTable dt = new DataTable();
SqlDataAdapter adp = new SqlDataAdapter(cmd);
adp.Fill(dt);
for (int i = 0; i < dt.Rows.Count; i++)
{
dto = new DTOClass();
dto.SystemResourceId = dt.Rows[i][0].ToString();
dto.SystemResourceName = dt.Rows[i][1].ToString();
dto.Created = dt.Rows[i][8].ToString();
dto.Updated = dt.Rows[i][10].ToString();
dto.Creater = dt.Rows[i][9].ToString();
dto.Updater = dt.Rows[i][11].ToString();
restDto.Add(dto);
}
return restDto; ;
}} }

This is simple implementation. Where ADO.Net is being used to fetch data from Database.
GetAllSystemResourceDetails is name of the stored procedure.

Note: Purpose of this article is to show how to push large volume of data from WCF service. So, I am not emphasizing ADO.Net part here. See the other articles for detail explanation on ADO.Net

Configuration setting at service side

<system.serviceModel><services>

<bindings>

<basicHttpBinding><binding name=“LargeBuffer” maxBufferSize=“2147483647” maxReceivedMessageSize=“2147483647“>

<readerQuotas maxDepth=“2147483647” maxStringContentLength=“2147483647“maxArrayLength=“2147483647“
maxBytesPerRead=“2147483647” maxNameTableCharCount=“2147483647“ /></binding>

</basicHttpBinding></bindings></system.serviceModel>

In service behavior, for dataContractSerializer I am increasing the number of object that can be serialized or de serialized at a time. By default it is set to 3565. Since our requirement is to push or return large volume of data, so I am giving value for maxItemsInObjectGraph to maximum integer number.
Since, I am using basicHttpBinding, so I am modifying values for this binding. I am setting all the attributes to maximum integer value.
Binding I am using is basicHttpBinding.

Compile the service and run to test, whether service is successfully created or not?

Step 2: Creating a client and Consuming the service.

Create any type of client. For my purpose I am creating a console client. After creating a console project, right click at Service Reference and add service reference. While adding service reference make sure, in advance setting as collection type System.Collection.Generic.List is selected.

Configuration setting at client side

<?xml version=“1.0” encoding=“utf-8“ ?>

<configuration>
<system.serviceModel>
<bindings>
<basicHttpBinding>
<binding name=“BasicHttpBinding_IService1“closeTimeout=“00:01:00”
openTimeout=“00:01:00“
receiveTimeout=“00:10:00“
sendTimeout=“00:01:00”
allowCookies=“false“
bypassProxyOnLocal=“false“
hostNameComparisonMode=“StrongWildcard”
maxBufferSize=“2147483647“
maxBufferPoolSize=“2147483647“
maxReceivedMessageSize=“2147483647”
messageEncoding=“Text“
textEncoding=“utf-8“
transferMode=“Buffered”
useDefaultWebProxy=“true“>
<security mode=“None“>
<transport clientCredentialType=“None” proxyCredentialType=“None”
realm=“” />
<message clientCredentialType=“UserName“algorithmSuite=“Default“ />
</security>
</binding>
</basicHttpBinding></bindings>
<client>
<endpoint address=“http://localhost:55771/Service1.svc“
binding=“basicHttpBinding”
bindingConfiguration=“BasicHttpBinding_IService1“
contract=“ServiceReference1.IService1”
name=“BasicHttpBinding_IService1“
behaviorConfiguration =“r1“/>
</client>
<behaviors>
<endpointBehaviors>
<behavior name =“r1“>

<dataContractSerializermaxItemsInObjectGraph =“2147483647“/>
</behavior>
</endpointBehaviors>
</behaviors>
</system.serviceModel>
</configuration>

Attributes of basicHttpBinding has been modified for the maximum integer values.
In endpoint behavior maxItemsInObjectGraph for dataContractSerializer has been modified to maximum integer value.

Program.cs

class Program{

static void Main(string[] args)
{
Service1Client proxy = new Service1Client();
List<DTOClass> res = new List<DTOClass>();
res = proxy.GetData();
foreach (DTOClass i in res)
{
Console.WriteLine(i.SystemResourceId + i.SystemResourceName + i.Updated + i.Updater);
}
Console.Read();
}}}

Conclusion
In this article, I explained how we can return large volume of data from WCF service and how at client side large volume of data can be consumed. This article explained on WCF SOAP based service. Next article I will explain how to achieve same for WCF REST service. Thanks for reading.

Rate this:

6 responses to “Returning Large volume of records from SOAP based WCF service”

parshuram

November 18, 2010 at 2:48 pm

Hi,

Thank you for a nice post on large data transmission over the WCF.

In client the following code is not working well
List res = new List();
res = proxy.GetData();
as the collections are converted into array by default. But, when I gave the following code
DTOClass []res = proxy.GetData();

it don’t display any compilation errors but, there are run time errors and I can’t get the program stops with errors. Can yo please provide the solution for this?

-Ram.
Monthly Report January 2010: Total Posts 4 « debug mode……

October 6, 2011 at 11:51 am

[…] Returning large volume of records from SOAP based WCF service […]
santosh

December 29, 2012 at 10:01 pm

thank you very much it is working fine.
Revan

May 27, 2013 at 4:20 pm

IS RETURN THE DATA TO PHP USER AS HERE IS USING LIST COLLECTION?
Returning Large Volume Of Records From SOAP Based WCF Service | sgrathikala

August 9, 2013 at 7:41 pm

[…] Returning Large Volume Of Records From SOAP Based WCF Service […]
Ash

February 9, 2016 at 1:31 am

What happens if the ‘Dictionary Collection Type’ is not set to ‘System.Collections.Generic.List’? Why I ask this question is I have all the other configuration parameter set right but this and not able to receive large data from the server. Was wondering if this configuration is causing that problem? I am new to WCF any help would be much appreciated.

Enter your comment here…

Fill in your details below or click an icon to log in:

Email (required) (Address never made public)

Name (required)

Website

You are commenting using your WordPress.com account. ( Log Out / Change )

You are commenting using your Twitter account. ( Log Out / Change )

You are commenting using your Facebook account. ( Log Out / Change )

Cancel

Connecting to %s

Dhananjay Kumar