locked
Multi-thread web service client with momery leak RRS feed

  • Question

  •  

    Dear All,

     

    I'm try find a solution my multi-thread web service client, it will process 10,000,000,000 record from remote web service, i don't mind if the process that 300 days. but the problem is that the processing client eat up memory or memeny leakage and die. Can anyone help me identify where is the memory not being collected by GC. I test this model with a profiler, it state that the System.string is growing and not collected by GC.

     

    Here the code of the client;

    Code Snippet

    using System;

    using System.Collections.Generic;

    using System.Linq;

    using System.Text;

    using System.Threading;

    using System.ComponentModel;

    namespace WebServiceClientWorkload

    {

    class Program

    {

    static void Main(string[] args)

    {

    // The number of thread to be loading

    int ThreadNumber = int.Parse(args[0]);

    // 1 mean run as static webservice client

    bool RunStatic = args[1].Equals("1")? true:false;

    Worker[] worker = Worker.ArrWorker(ThreadNumber, RunStatic);

    while (!Console.KeyAvailable)

    {

    Console.WriteLine("Momery Usage {0:0,0} Bytes",System.Diagnostics.Process.GetCurrentProcess().PagedMemorySize64);

    Thread.Sleep(1000);

    }

    }

    }

    class Worker

    {

    BackgroundWorker bw;

    bool RunStatic;

    int ThreadId;

    Worker(int threadId,bool runStatic)

    {

    ThreadId = threadId;

    RunStatic = runStatic;

    bw = new BackgroundWorker();

    bw.DoWork += bw_DoWork;

    bw.RunWorkerAsync();

    }

    void bw_DoWork(object sender, DoWorkEventArgs e)

    {

    for (int i = 0; i < 1000; i++)

    {

    if (RunStatic)

    {

    try

    {

    StaticClient.HelloWorld();

    //Console.WriteLine("{0}:{1} OK", ThreadId, i);

    }

    catch { /*Console.WriteLine("{0}:{1} ERR", ThreadId, i); */}

    }

    }

    }

    public static Worker[] ArrWorker(int number,bool runStatic)

    {

    Worker[] array = new Worker[number];

    for (int i = 0; i < number; i++)

    {

    array[i] = new Worker(i, runStatic);

    Console.WriteLine("Loaded thread {0}", i);

    }

    return array;

    }

    }

    static class StaticClient

    {

    static TestReference.ServiceSoapClient webservice; //my webservice object

    static StaticClient()

    {

    webservice = new WebServiceClientWorkload.TestReference.ServiceSoapClient();

    }

    public static string HelloWorld()

    {

    return webservice.HelloWorld();

    }

    }

    }

     

    Also you would need to change the maxconnection to see the result quicker, edit this in the app.config.

    Code Snippet

    <system.net>

    <connectionManagement>

    <add address="*" maxconnection="10000"/>

    </connectionManagement>

    </system.net>

     

     

    And here the Server Codes, nothing much really simple;

    Code Snippet

    using System;

    using System.Linq;

    using System.Web;

    using System.Web.Services;

    using System.Web.Services.Protocols;

    using System.Xml.Linq;

     

    [WebService(Namespace = "http://tempuri.org/")]

    [WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]

    // To allow this Web Service to be called from script, using ASP.NET AJAX, uncomment the following line.

    // [System.Web.Script.Services.ScriptService]

    public class Service : System.Web.Services.WebService

    {

    public Service()

    {

    //Uncomment the following line if using designed components

    //InitializeComponent();

    }

    [WebMethod]

    public string HelloWorld()

    {

    return "HelloWorld";

    }

    }

     

    I write the code that will show the current usage of the client application memory every 1 second untill a keypress (My front end thread command waiting).

     

    Just find a solution to my memory leakage issue here, it has been a months now.

     

    Thank you advance for your help.

     

    James NW

     

     

    Monday, February 4, 2008 2:18 AM

Answers

  • You code seems to be a lesson in  a multi-threading attempt and not real world operations. Calling the web-service is not the issue...two issues I see are:
    1. The client program's objects need to clean up after themselves appropriately. Dispose of memory needs to be handled better.
    2. Any worker thread should not attempt to be writing to the console or updating any UI that is the main thread's job. 
    The design of this program is the real issue...I recommend that you read up on mult-threading to see all the problems that your code contains. A good place to start is Multi-threading in .NET: Introduction and suggestions

    good luck hth.
    Tuesday, February 5, 2008 5:11 PM