locked
Design Service Self testing strategy RRS feed

  • Question

  • Hi,

    I was wondering if some one would be able to help me out in my current design approach to the following problem.

    We have many independent WCF services that are exposed by our system and can be called by our silverlight client. We also have a routing layer that looks for version of the client and may be also look at the load balancing etc in the future and direct the call to the currect service.

    I want to build a tool that will periodically check if these services are working currectly what we calll as "Service Integrity checker".  Idea is to know if the service is up and if it has all the resources needed to function correctly. i.e a service may be responding but it may not be able to connect to the database which makes it unusable.

    What I am thinking at the moment is to kind of have another endpoint on each of these service that exposes a IServiceDiagnostic interface which has a method to check its integrity and return result.

    I was wondering if we have anyone here can help me out in this design approach or may be share their exprience if they ever attempted to do something similar.

    Regards, Kiran


    • Edited by nkiranrao Monday, August 8, 2011 11:28 AM
    Saturday, August 6, 2011 6:39 PM

Answers

  • If you only want to check whether the services are running (no need to check whether they're running properly), you can use a configuration file to put all services' endpoint addresses. Your tool will iterate through all addresses in the configuration file, and issue a GET request to the service. Assume all services are SOAP services with mex endpoint configured properly, a GET request to the mex endpoint should return a WSDL file. You can then do some simple checks on the result to see if it's a WSDL file (such as checking the first line in the result to see if it's a WSDL header). When you add new services, you only need to update the configuration file. You don't need to recompile the tool.

    Obviously this approach won't check if the services are running properly. But assume you've done enough tests beforehand, and you have other monitoring tools such as Windows Server AppFabric and System Center that help to log and report errors, it should be fine.


    Lante, shanaolanxing This posting is provided "AS IS" with no warranties, and confers no rights.
    Windows Azure Technical Forum Support Team Blog
    • Marked as answer by Yi-Lun Luo Friday, August 12, 2011 8:29 AM
    Tuesday, August 9, 2011 2:39 AM

All replies

  • Hello, I think you can just write a client tool that add service references to all your services. Inside the tool, you use a timer to invoke all services at a regular interval. Then you use the same technique as you use in unit tests to check the result. If something goes wrong, don't assume the service is down. Instead, try the operation again after several seconds to see if it works this time. If an operation fails for, say, 3 times, it probably means it really has some problems.
    Lante, shanaolanxing This posting is provided "AS IS" with no warranties, and confers no rights.
    Windows Azure Technical Forum Support Team Blog
    Monday, August 8, 2011 3:39 AM
  • Hi,

    Thanks for your reply.

    With the approach you are proposing, I would end up with recompiling the client tool every time some one introduces a new service. I would prefer creating a tool that is service agnostic so that any servies added/removed in the future works without any change to this tool. Also running the full test cases is probably is not what I am looking for, we already know the service works as its gone through multiple rounds of testing before being deployed, intention is to know if the service has access to all the resources it needs to be useful. for e.g service may be up and running, but it may not have access to connect to some other service/database which may be down.

    Basically what i am looking for is to have a health check on any of the services deployed on my server and report its availability. My feeling is that its hard to detect these kind of issues from outside the service itself, and that the reason I was looking to introduce another endpoint on each of these service that an external tool can call and check if the service is useable even if its responding.

    would appreciate any feedback on this.

    Regards, Kiran  

     

    Monday, August 8, 2011 5:23 AM
  • Hi,

    I am not an expert but still i just wanted to share my thoughts on this. Even I feel writing a client for checking the services as an overhead. My thought process suggest creating a new service which will iterate through all the services in that machine and check the status of it. This service can have an endpoint which exposes methods for the checks. The advantage with this approach is that, since it dynamically search for the services, we dont have add new codes when ever there is a new service added or an old service is modified.

    This is just a suggestion. I will be glad to know your suggestions

     

    Regards,

    Crazy Tech


    Crazy Tech
    Monday, August 8, 2011 8:59 AM
  • If you only want to check whether the services are running (no need to check whether they're running properly), you can use a configuration file to put all services' endpoint addresses. Your tool will iterate through all addresses in the configuration file, and issue a GET request to the service. Assume all services are SOAP services with mex endpoint configured properly, a GET request to the mex endpoint should return a WSDL file. You can then do some simple checks on the result to see if it's a WSDL file (such as checking the first line in the result to see if it's a WSDL header). When you add new services, you only need to update the configuration file. You don't need to recompile the tool.

    Obviously this approach won't check if the services are running properly. But assume you've done enough tests beforehand, and you have other monitoring tools such as Windows Server AppFabric and System Center that help to log and report errors, it should be fine.


    Lante, shanaolanxing This posting is provided "AS IS" with no warranties, and confers no rights.
    Windows Azure Technical Forum Support Team Blog
    • Marked as answer by Yi-Lun Luo Friday, August 12, 2011 8:29 AM
    Tuesday, August 9, 2011 2:39 AM
  • I fully understand the approach of using a configuration file to keep track of all the services in the system to be monitored, which is inline with my thinking

    Well, as I mentioned earlier, I am also interested in knowing if the service is running correctly to an extent that I want to know if its able to make connection to database or any other external system which it has dependency on.

    Actually I am not sure what all resources a service is seeking as thay may be arbitrarily added or removed. Thats the reason why I am looking to introduce an endpoint IServiceDiagnostic that can be used to initiate the call to the service itself to do a diagnostic check and report back the result to the checker tool. what do you think about this approach and is there an easier alternative to this?

    also you mention about the "Windows Server AppFabric and System Center" can you help me understand how this can be used for the purpose I am trying to achieve, or point me to any link that can help me understand this.

    Regards, Kiran

    Tuesday, August 9, 2011 3:06 AM
  • Yes, you can delegate the diagonstic tasks to each service, if the process is complex.

    As for Windows Server AppFabric, it allows you to monitor your services hosted in IIS. If something goes wrong, you'll see the detailed errors in IIS Manager. You can learn more from http://msdn.microsoft.com/en-us/windowsserver/ee695849. This tool is free.

    As for System Center, it is a professional ITPro product that helps ITPros to control, monitor, recover their systems. It can even automatically recover certain errors. But this product is not free. You can learn more from http://www.microsoft.com/systemcenter/en/us/default.aspx. I recommend you only to purchase this tool if your IT environment is quite complex (for example, you have thousands of servers).


    Lante, shanaolanxing This posting is provided "AS IS" with no warranties, and confers no rights.
    Windows Azure Technical Forum Support Team Blog
    Tuesday, August 9, 2011 3:12 AM
  • Can App Frabic be used to do a periodic check on the service and report any such issues?

    or dose it only report any exceptions generated by the service
    Tuesday, August 9, 2011 10:54 AM
  • AppFabric only reports exceptions and application logs written using the standard Trace class. For example, if a service is unable to connect to a remote database, chances are either an excepion is thrown, or the service catches the exception, and write it to logs. So AppFabric should be able to report it.
    Lante, shanaolanxing This posting is provided "AS IS" with no warranties, and confers no rights.
    Windows Azure Technical Forum Support Team Blog
    Wednesday, August 10, 2011 2:05 AM