none
HTTP style gzip compression using http headers(Accept/Content-encoding) ina SELF HOSTED WCF Service.

    General discussion

  • [Update]
    Feb 08 2010
    Wow, just realized its been a year since i made this post.

    Today i extended this gzip encoder that encapsulates another encoder. Basically, i created a MultiContentTypeMessageEncoder that encapsulates gzip-text, gzip-mtom, gzip-binary, gzip-fastinfoset (from noemax) and successfully hosted it on a single endpoint.

    Basically, i switched the MessageEncoding binding element during configuration initialization via a endpoint style behavior.
    I did this to simplify

    a) Scenarios where multiple wcf self hosted services are running behind load balancers. In this scenario, a client only cares about the load balancer's uri. However, this load balancer's uri cannot (and should not) be added to your 'address' on the endpoint) because in practice there can be N load balancers between teh client and the eventual service.

    b) Absolutely not having the client and the service provider, both , having to deal with multiple uris just because the message encoding on the bindings is different.

    So here is how my  app.config's fragment looks like. I Use a standard WS-HttpBinding which gets modified by the endpoint behavior to do all the fancy stuff including end to end activity + log correlation.

    Please do NOT ask me for code, i can atmost help answer your questions using what i have for reference.

    <services>
          <service    behaviorConfiguration="svcBehaviorCfg" name="TestJig.SOASvc">

            <endpoint  binding="wsHttpBinding" bindingConfiguration="stdTextCfg"
                       name="epHttpV1_0_0"
                       bindingNamespace="urn:expedia:bfs:testjig"
                       behaviorConfiguration="stdHttpServerSideSoaEndpointBehaviorCfg"
                       contract="TestJig.V1_0_0.ISOAService"
                       address="urn:logicalAddr:testjig:epHttpV1_0_0"
                       listenUri="http://localhost:4000/testjig"
                      
                       />
           </service>
    </services>

        <behaviors>
          <endpointBehaviors>

            <behavior name="stdHttpServerSideSoaEndpointBehaviorCfg">
              <stdHttpServerSideSoaEndpointBehavior/>
            </behavior>

        </endpointBehaviors>


    ------------------------------------------------------------------------------------------------------------------------------------
    Feb 07, 2009
    After beating around the bush the solution is rather straight forward and i now have it working with .NET 3.5 SP1.

    I am using
    BasicHttpBinding
    the service is selfhosted so i had to do this.

    Notes: From the GZip encoder i return the mediatype and content-type of the inner encoder(TextEncoder) so that everything works as expected.


    What you need to do
    a) Write a custom encoder (use Microsoft's sample GzipEncoder or write something a little that encapsulates TextEncoder ) , but it works with content-type only...also the http request part is not accessible here but no worries.

    b) In ReadMessage if array starts with 0x1f,0x8B,0x08 , then it is a gzipped data, decompress the data and then pass it over to TextEncoder to do teh usual stuff. So you see you dont really need to look at Content-Encoding here.

    c) Write a EndPoint Behavior.

    d) In the EndPoint behavior's AfterReceiveRequest return request->Properties["httpRequest"] , this is the correlation object and will end up at BeforeSendReply.

    e) in BeforeSendReply, check fi the correlation object has the header "Accept-Encoding" set, it is set create System::ServiceModel::Channels::HttpResponseMessageProperty , set Content-Encoding: gzip AND attach it to reply->Properties["httpResponse"]

    f) in the WriteMessage of the encoder , first use TextEncoder to get the array and then check if message->Properties["httpResponse"] (passed argument) and based upon its existence either decide if you would like to compress teh array. before returning it.

    And thats all you need to do.

    TO summarize.
    a) You have a gzip Encoder sitting right above the transport with an encapsulated TextEncoder, examples of these can be found over the net or you can use microsoft's sample gzip encoder.

    b) you have a EndPoint behavior sitting higher up in the stack , the AfterReceiveRequest is use to grab the request headers and correlate them in BeforeSendReply, where you 'attach' a HttpResponse header "Content-Encoding" to the outgoing msg.

    c) in the gzip encoder's WriteMessge you do a selective compression.


    • Edited by Siddharth Sawe Tuesday, February 09, 2010 8:53 AM Update to the post
    • Changed type Siddharth Sawe Tuesday, February 09, 2010 8:55 AM This was miscategorized
    Saturday, February 07, 2009 5:10 AM

All replies

  • Hi, how to use GZip encoder that downloaded from the sample given by msdn, and use the same security type of  wsHttpBinding? any guildline?
    Tuesday, October 06, 2009 3:07 AM
  • wsHttpBinding has a hardcoded MessageEncoder logic that does nto allow any encoder other than the Text and the Binary encoder. so u cant really use it with wsHttpBinding. but you can use a CustomBinding to do exactly what the wsHttpBinding does, with the difference that CustomBinding will allow you to plugin whatever encoder you wish.

    Tuesday, October 06, 2009 6:25 AM
  • yeah, i noticed that, i had try it for don't know how long, do you have any workaround? Or sample project that custom binding, which serve the same setting(binding element) with WSHttpBinding and with the gzipMessageEncoder?
    Really appreciate your reply.
    Tuesday, October 06, 2009 7:33 AM
  • I had try to create a new Custom Binding by coding, not by config file.
    public class MyCustomBinding : CustomBinding
    override a function BindingElementCollection:
    public override BindingElementCollection CreateBindingElements()
    {
       WSHttpBinding wSHttpBinding = new WSHttpBinding("RMSKeberosBinding"); //this is to load the configuration from app.config. because i want to copy the setting of wsHttpConfig to my custom binding.
    
       BindingElementCollection wSHttpBindingElementCollection = wSHttpBinding.CreateBindingElements();
    
       TransactionFlowBindingElement transactionFlowBindingElement = wSHttpBindingElementCollection.Remove<TransactionFlowBindingElement>();
       SymmetricSecurityBindingElement securityElement = wSHttpBindingElementCollection.Remove<SymmetricSecurityBindingElement>();
       MessageEncodingBindingElement textElement = wSHttpBindingElementCollection.Remove<MessageEncodingBindingElement>();
       HttpTransportBindingElement transportElement = wSHttpBindingElementCollection.Remove<HttpTransportBindingElement>();
    
       GZipMessageEncodingBindingElement gzipElement = new GZipMessageEncodingBindingElement(); // this is from microsoft sample. i want to add gzip as a compress to my message.
    
       BindingElementCollection newCol = new BindingElementCollection();
       newCol.Add(transactionFlowBindingElement);
       newCol.Add(securityElement);
       newCol.Add(gzipElement);
       newCo .Add(transElement);
       return newCol;
    }
    what i am trying to do is copy all setting from wshttpbinding, and add on gzip as the message encoder.Compress an encrypted data will lead to a bigger size of the original data size. this is because the SymmetricSecurityBindingElement from WSHttpBinding did the encryption. How to do this in correct way? i want the security setting from wshttpbinding, and also the gzip to work.
    Friday, October 09, 2009 12:56 AM
  • Actually u are creating a custom binding. Custom binding can be created programmatically with ease we have "named" binding stacks that auto create our bindings I will post the config file for getting the encoder into ur code we don't use config files anymore since all of our configuration comes from a db or some legac. Data src or somtines from the network
    Friday, October 09, 2009 1:13 AM
  • Hi, Siddharth Sawe, thanks for the quick reply.
    Yes, i understand that doing via programmatically.
    Can explain in more detail how to do this?
    I need the security from WSHttpBinding, and also GZip compression to work.
    Thanks a lot.
    Friday, October 09, 2009 1:16 AM
  • hi Siddharth,

    I am also looking for example of Compression using GzipEncoder. I tried a lot but couldn't succeded.

    We have done with the sync service implementation on device using WCF service. Now we want to implement the data compression in it.

     

    I have done all things as per the example given by Andy at- http://msdn.microsoft.com/en-us/library/dd938879.aspx

     

    When I run the WCF service it get hosted perfectly but when I called a function from the WCF service wizard it gives me following error.

     

    Content Type application/soap+xml; charset=utf-8 was not supported by service http://10.85.0.224:8731/PocketSHEWebService/ .  The client and service bindings may be mismatched.

     

    ( I called the function from the WCF service startup wizard….so didn’t understood the  error “The client and service bindings may be mismatched" who is the client in this case, because I call the function from WCF service)

     

    I searched a lot on net but couldn’t get rid of it. Could you please give your valuable suggestion on it
    Monday, November 23, 2009 8:54 AM
  • I dontknow what microsoft's example does, hoever, the basic server side design would involve encapsulating a text or mtom encoder inside a custom gzip encoder however for this to get recognized correctly you should override the contenttype and mediatype properties in in your custom encoder and simply call the corresponding impls of the encapsulated encoder in your case your client is sending the standard contenttype which probably does not match what your encoder returns possibly bcoz you changed it in your encoder if possible post the contenttype and mediatype overrides from your encoder gzip compression can be implemented transparently because the stream has a signature
    Monday, November 23, 2009 9:20 AM
  • hi Sid,

    i used the text/xml contenttype as well as media type but still i am  not able to solve the problem.

    could you please send the example you have successfully done on my mail id dinkar.bhingarde@comops.com.au.

    i am very pleased for your valuable reply.
    Tuesday, November 24, 2009 5:36 AM
  • hi sid,

    following are the Content & media type i m using in example...

    class GZipMessageEncoder : MessageEncoder
            {
                static string GZipContentType = "text/xml";

                //This implementation wraps an inner encoder that actually converts a WCF Message
                //into textual XML, binary XML or some other format. This implementation then compresses the results.
                //The opposite happens when reading messages.
                //This member stores this inner encoder.
                MessageEncoder innerEncoder;

                //We require an inner encoder to be supplied (see comment above)
                internal GZipMessageEncoder(MessageEncoder messageEncoder)
                    : base()
                {
                    if (messageEncoder == null)
                        throw new ArgumentNullException("messageEncoder", "A valid message encoder must be passed to the GZipEncoder");
                    innerEncoder = messageEncoder;
                }

                public override string ContentType
                {
                    get { return GZipContentType; }
                }

                public override string MediaType
                {
                    get { return GZipContentType; }
                }

                //SOAP version to use - we delegate to the inner encoder for this
                public override MessageVersion MessageVersion
                {
                    get { return innerEncoder.MessageVersion; }
                }
    }
    Tuesday, November 24, 2009 10:37 AM
  • The problem is that you are returning a contenttype and mediatype that the client proxy does not understand here is what u need to do a) delegate contenttype and mediatype to the innerencoder. Just the way you have done for soap version
    Tuesday, November 24, 2009 10:48 AM
  • Dude Another thing , contenttype identifies the format media type identifies the encoding such as utf-8 you are better off delegating this to the inner encoder secondly unless you use the same gzip encoder with the same innerencoder on the clientvside you will get hosed coz your impl is hardcoded to gzip read my first post on this thread on what you need to do for a dynamic impl
    Tuesday, November 24, 2009 10:53 AM
  • Dude I read thru your org post again you are using the auto created proxy to talk to ur svc that uses ur gzip encoder you need to plug you gzip encoder on the client side too and yes ur failure is bcoz the clientside encoder is expecting application/soapxml with mediatype utf 8 you are returning some hardcoded value even if you fix this part , your svc side encoder will barf, coz you are trying to gunzip a stream while the auto gen proxy will always send uncompressed data
    Tuesday, November 24, 2009 10:59 AM
  • A) create a endpoint behavior , let's call it gzipbehaviorextensionelemwnt b) creat a class called gzipbinding derived from Binding. With a member property of type Binding called inner binding c) override all overridables from binding and pass the calls to innerbinding d) initialize all non overridables in the contructor with the vales from inner binding e) in CreateBindingElements() call inner bibding's method but wtapp the returned messageencodingelement with gzupnsgencodingelement before returning the collection to the caller f) implemt both iwsdlexport and ciwsdlpolicy , call inner binding's method g) in the "Validate" method of endpoint behavior, replace the binding on tje argument endpoint with gzipbinding encapsulating the original binding that is being replaced now in your configs, you can use whatever bindings necessary, all u sneed is to declare the endpointvas having the gzipbehavior the advantage is that the bibding's can be edited using the svcconfigeditor GUI abd still be able to use gzip with ease sorry cannot post code. Just remember to replace the binding in validate method only .
    Saturday, December 12, 2009 6:59 PM
  • Hi,

    Do you have a complete demo application i can download? I want to build a self hosted wcf service that serves gzipped data when I access it with a browser. Looks like your code can fix that...?

    Steinar

    Tuesday, March 08, 2011 1:27 PM