none
(WCF) Gateway service - design considerations RRS feed

  • Question

  • Hi,

     

    I have sort of architectural question related to the architecture of the Gateway service ...

    I have a WCF service, providing pub-sub functionality. It runs in some server farm (let call it Farm 1) inside of some LAN (say LAN 1); the binding is netTcpBinding. So, there is a single pub-sub service and a number of clients using it. Each client can subscribe to some topics and send topic updates. Whoever is registered to get the updates - receives them from the pub-sub (using CallbackContract). So far so good…

     

    Now I need to provide connectivity between a number of such server farms (amount of farms is restricted and predefined). I.e. I need to enable clients in Farm 1 update clients in other Farms and get updates from them. My question is how to implement in the best way…

     

    I going to do it by adding a gateway proxy per each Farm (e.g. for 3 farms configuration - in Farm 1 - there would be two gateway proxies representing Farm 2 and Farm 3).

    Those proxies would get and topic update and would forward them to the corresponding gateways in other Farms. In the same way they would receive topic updates from other farms.

     

    Is it a good solution? Do you see any potential caveats? May be there is something that may ease development of such functionality?

     

    Any information would be helpful.

     

    Best regards,

                Alex Pinsker

                http://alexpinsker.blogspot.com

    Wednesday, April 23, 2008 3:17 PM

Answers

  • Based on what you have mentioned,

     

    the following is based on my experience with WCMS solutions in the past.

     

    You want to create a small application (like a windows service) called Agent that would run on each farm.  The purpose of the agents is to keep the data in sync.  Depending on how fresh you need the data to be,

     

    you can create the agents

     

    1) like IRC (Internet Relay Chat).  This would keep the information "in-context" and "true real-time" for all the users, in all farms, in all LANs.  As you may have already guessed, this would be very chatty, quite a lot of information will be transmitted back and forth among your Agents.  Dont know if your network is okay to get an hit like this.

     

    2) Agents wake up every 'X' mins, probably in a round-robin basis so it is easy to control informatoin flow among agents.  Agent A wakesup first, gets updates from Agent B and C.  Next Agent B wakes up and gets update from A and C, and so on...

     

    To increase efficiency,

    basically to reduce data going back and forth, mark all the updates from each user as event message.  So you just transmit the event (for eg: For deleted row/information, then just mark the rowid/text as deleted.  All you need to transmit is about rowid/text, not the whole row).

     

    Hope this helps.

     

     

     

     

    Friday, April 25, 2008 2:11 AM

All replies

  • I am explaining the question below again to make sure I am understanding your problem.

     

    FarmA/LanA
    Topic SportsA, WeatherA

    UserA1
    UserA2

     

    FarmB/LanB
    Topic SportsB, NewsB
    UserB1
    UserB2

     

    FarmC/LanC
    Topic SportsC, EntertainmentC
    UserC1
    UserC2

     

    Validate the below
    1. Sports A, B and C all are same?  just different network?
    2. UserA1 updates basketball information on SportsA.  You want UserA2, UserB1, UserB2, UserC1 and UserC2 all to see the information since they all have subscribed to Sports.
    3. your problem statement sounds like a WCMS (Web Content Management System)...
         If yes, cant you use the existing products in the market (Open source/Vended)
    4.1 Can FarmA,B and C all use one single datasource?  So you dont need to send updates back and forth...
    4.2 Since FarmA,B and C are in different LANs (A,B and C) different networks, can you use a database that can handle replication like SQL Server which does this out of the box easy for you...

     

    Hopefully based on your response, I may be able to help you.

    Wednesday, April 23, 2008 5:43 PM
  • Hi,

    Thanks for your answer.

     

    1. Yes.

    2. Yes.

    3. It's not WCMS and unfortunately I didn't found anything ready. Pub/sub is inhouse solution. The applications are real-time.

    4. a) No. Each farm uses its own data-source (SQL 2005 DB). This is because LANs are physically in remote locations, say LanA in Europe and LanB in Australia. So, the round trip between LAN's is quite long. If all users would work vs. single datastore, which is, say, in Europe - for users in Australia apllication would be very slow.

        b) We do have a DB per Farm and replication synchronizes the data between them. Unfortunately this doesn't solve the problem. If it was a web-application, where each application does polling of a data-store and the data-stores are synchronized with replication - this would work. But since data-updates aren't based on polling but rather arrive from pub-sub server - I must transmit published data-updates between farms (and data-stores would be synchronized 'offline' by replication).

     

    If you have any idea or any considerations on the way I going to solve it – it would really help.

     

    A side-note… I think that what I doing falls to the category of kind-of distributed cache/data systems. There are some commercial solutions giving such functionality, but they are really expensive, I think around hundreds K$.

     

    Regards,

    Alex Pinsker

    http://alexpinsker.blogspot.com

     

    Thursday, April 24, 2008 7:50 AM
  • Any idea?

    Thursday, April 24, 2008 2:54 PM
  • How accurate, or should I say stale, can the difference between the data on the different nodes be?

    Thursday, April 24, 2008 7:42 PM
  • Based on what you have mentioned,

     

    the following is based on my experience with WCMS solutions in the past.

     

    You want to create a small application (like a windows service) called Agent that would run on each farm.  The purpose of the agents is to keep the data in sync.  Depending on how fresh you need the data to be,

     

    you can create the agents

     

    1) like IRC (Internet Relay Chat).  This would keep the information "in-context" and "true real-time" for all the users, in all farms, in all LANs.  As you may have already guessed, this would be very chatty, quite a lot of information will be transmitted back and forth among your Agents.  Dont know if your network is okay to get an hit like this.

     

    2) Agents wake up every 'X' mins, probably in a round-robin basis so it is easy to control informatoin flow among agents.  Agent A wakesup first, gets updates from Agent B and C.  Next Agent B wakes up and gets update from A and C, and so on...

     

    To increase efficiency,

    basically to reduce data going back and forth, mark all the updates from each user as event message.  So you just transmit the event (for eg: For deleted row/information, then just mark the rowid/text as deleted.  All you need to transmit is about rowid/text, not the whole row).

     

    Hope this helps.

     

     

     

     

    Friday, April 25, 2008 2:11 AM