Returning Large volume of records from SOAP based WCF service

Objective

In this article I will explain; How to return large volume of data (around 50000 records) from SOAP based WCF service to any client. I will show; what service configuration setting and client configuration setting is needed to handle large volume of data returning for WCF service.

Follow the below steps,

Step 1: Create a WCF service

.To creates WCF service; select File -> New -> Project-> Web -> WCF Application.

Service will contain

  1. One Operation contract. This function will pull large data from database using stored procedure.
  2. One Data Contract. This class will act as Data Transfer object (DTO) between client and service.
  3. basicHttpBinding is being used in the service. You are free to use any binding as of your requirement.

Data Transfer Class

[DataContract]

public class DTOClass
{

[DataMember]
public string SystemResourceId
{
get;set;

}
[DataMember]
public string SystemResourceName
{
get;
set;
}
[DataMember]
public string Created
{
get;
set;}
[DataMember]
public string Creater
{
get;set;
}
[DataMember]
public string Updated
{
get;set;
}
[DataMember]
public string Updater
{
get;
set;
}

  1. Name of DTO class is DTOClass. You can give any name of your choice.
  2. There are 6 string properties.
  3. All properties are attributed with DataMember.

Contract

[ServiceContract]

[ServiceKnownType(typeof(DTOClass))]
public interfaceIService1
{
[OperationContract]
List<DTOClass> GetData();
}

  1. Service is returning List of DTOClass.
  2. To get serialized at run time, making sure Data Contract is known to contract by using known type.

Service Implementation

public class Service1 : IService1{

string cs = @”Data Source=xxxserver;Initial Catalog=Sampledatabase;User=dhananjay;Password=dhananjay”;
List<DTOClass> restDto = new List<DTOClass>();
DTOClass dto;
public List<DTOClass> GetData()
{
SqlConnection con = new SqlConnection(cs);
SqlCommand cmd = new SqlCommand(“GetAllSystemResourceDetails”, con);
cmd.CommandType = CommandType.StoredProcedure;
DataTable dt = new DataTable();
SqlDataAdapter adp = new SqlDataAdapter(cmd);
adp.Fill(dt);
for (int i = 0; i < dt.Rows.Count; i++)
{
dto = new DTOClass();
dto.SystemResourceId = dt.Rows[i][0].ToString();
dto.SystemResourceName = dt.Rows[i][1].ToString();
dto.Created = dt.Rows[i][8].ToString();
dto.Updated = dt.Rows[i][10].ToString();
dto.Creater = dt.Rows[i][9].ToString();
dto.Updater = dt.Rows[i][11].ToString();
restDto.Add(dto);
}
return restDto; ;
}} }

  1. This is simple implementation. Where ADO.Net is being used to fetch data from Database.
  2. GetAllSystemResourceDetails is name of the stored procedure.

Note: Purpose of this article is to show how to push large volume of data from WCF service. So, I am not emphasizing ADO.Net part here. See the other articles for detail explanation on ADO.Net

Configuration setting at service side

<system.serviceModel><services>

<service name=TestingLargeData.Service1behaviorConfiguration=TestingLargeData.Service1Behavior>                

<endpoint
address=“”
binding=basicHttpBinding
contract=TestingLargeData.IService1
bindingConfiguration=LargeBuffer>
 

<identity>
<dns
value=localhost/>
</identity></endpoint>
<endpoint address=mexbinding=mexHttpBinding contract=IMetadataExchange/>
</service>
</services>
<behaviors>
<serviceBehaviors>
<behavior name=TestingLargeData.Service1Behavior>                    
<serviceMetadata httpGetEnabled=true/>                    
<serviceDebug includeExceptionDetailInFaults=false/>
<dataContractSerializer maxItemsInObjectGraph=2147483647/>
</behavior>
</serviceBehaviors>
</behaviors>

<bindings>

<basicHttpBinding><binding name=LargeBuffermaxBufferSize=2147483647maxReceivedMessageSize=2147483647>

<readerQuotas maxDepth=2147483647maxStringContentLength=2147483647maxArrayLength=2147483647
maxBytesPerRead=2147483647maxNameTableCharCount=2147483647 />
</binding>

</basicHttpBinding></bindings></system.serviceModel>

  1. In service behavior, for dataContractSerializer I am increasing the number of object that can be serialized or de serialized at a time. By default it is set to 3565. Since our requirement is to push or return large volume of data, so I am giving value for maxItemsInObjectGraph to maximum integer number.
  2. Since, I am using basicHttpBinding, so I am modifying values for this binding. I am setting all the attributes to maximum integer value.
  3. Binding I am using is basicHttpBinding.

Compile the service and run to test, whether service is successfully created or not?

Step 2: Creating a client and Consuming the service.

Create any type of client. For my purpose I am creating a console client. After creating a console project, right click at Service Reference and add service reference. While adding service reference make sure, in advance setting as collection type System.Collection.Generic.List is selected.

Configuration setting at client side

<?xml version=1.0encoding=utf-8 ?>

<configuration>
<system.serviceModel>
<bindings>
<basicHttpBinding>
<binding name=BasicHttpBinding_IService1closeTimeout=00:01:00
openTimeout=00:01:00
receiveTimeout=00:10:00
sendTimeout=00:01:00
allowCookies=false
bypassProxyOnLocal=false
hostNameComparisonMode=StrongWildcard
maxBufferSize=2147483647
maxBufferPoolSize=2147483647
maxReceivedMessageSize=2147483647
messageEncoding=Text
textEncoding=utf-8
transferMode=Buffered
useDefaultWebProxy=true>
<security mode=None>
<transport clientCredentialType=NoneproxyCredentialType=None
realm=“” />
<message clientCredentialType=UserNamealgorithmSuite=Default />
</security>
</binding>
</basicHttpBinding></bindings>
<client>
<endpoint address=http://localhost:55771/Service1.svc
binding=basicHttpBinding
bindingConfiguration=BasicHttpBinding_IService1
contract=ServiceReference1.IService1
name=BasicHttpBinding_IService1
behaviorConfiguration =r1/>
</client>
<behaviors>
<endpointBehaviors>
<behavior name =r1>

<dataContractSerializermaxItemsInObjectGraph =2147483647/>
</behavior>
</endpointBehaviors>
</behaviors>
</system.serviceModel>
</configuration>

  1. Attributes of basicHttpBinding has been modified for the maximum integer values.
  2. In endpoint behavior maxItemsInObjectGraph for dataContractSerializer has been modified to maximum integer value.

Program.cs

class Program{

static void Main(string[] args)
{
Service1Client proxy = new Service1Client();
List<DTOClass> res = new List<DTOClass>();
res = proxy.GetData();
foreach (DTOClass i in res)
{
Console.WriteLine(i.SystemResourceId + i.SystemResourceName + i.Updated + i.Updater);
}
Console.Read();
}}}

 Conclusion
In this article, I explained how we can return large volume of data from WCF service and how at client side large volume of data can be consumed. This article explained on WCF SOAP based service. Next article I will explain how to achieve same for WCF REST service. Thanks for reading.

About these ads

5 thoughts on “Returning Large volume of records from SOAP based WCF service

  1. Hi,

    Thank you for a nice post on large data transmission over the WCF.

    In client the following code is not working well
    List res = new List();
    res = proxy.GetData();
    as the collections are converted into array by default. But, when I gave the following code
    DTOClass []res = proxy.GetData();

    it don’t display any compilation errors but, there are run time errors and I can’t get the program stops with errors. Can yo please provide the solution for this?

    -Ram.

  2. Pingback: Monthly Report January 2010: Total Posts 4 « debug mode……

  3. Pingback: Returning Large Volume Of Records From SOAP Based WCF Service | sgrathikala

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s