locked
System URI parameters with WCF and Azure tables RRS feed

  • Question

  • I am currently using System.Data.Services.DataService<T> to expose Azure table storage as a WCF service as demonstrated in the following blog:

    http://blogs.southworks.net/fboerr/2010/07/29/create-a-wcf-data-service-odata-to-share-an-azure-table/

    This method works fine for most queries and I can get json data for ajax calls. I am able to use $filter to filter the data, but other system URI parameters are causing exceptions. The issue is that I now need to page the data and I am getting an exception when I try to use the $top and $skip system URI parameters:

    An error occurred while processing this request.
    System.Data.Services.Client.DataServiceQueryException
       at System.Data.Services.Client.DataServiceRequest.Execute[TElement](DataServiceContext context, QueryComponents queryComponents)
       at System.Data.Services.Client.DataServiceQuery`1.Execute()
       at System.Data.Services.Client.DataServiceQuery`1.GetEnumerator()
       at System.Data.Services.Client.DataServiceQuery`1.System.Collections.IEnumerable.GetEnumerator()
       at System.Data.Services.WebUtil.GetRequestEnumerator(IEnumerable enumerable)
       at System.Data.Services.DataService`1.SerializeResponseBody(RequestDescription description, IDataService dataService)
       at System.Data.Services.DataService`1.HandleRequest()
    -- Inner Exception --
    NotImplemented: The requested operation is not implemented on the specified resource.
    System.Data.Services.Client.DataServiceClientException
       at System.Data.Services.Client.QueryResult.Execute()
       at System.Data.Services.Client.DataServiceRequest.Execute[TElement](DataServiceContext context, QueryComponents queryComponents)
    

    I can see from using Fiddler that the underlying WCF service (abcxyz.table.core.windows.net) supports $top and uses NextPartitionKey and NextRowKey to implement paging/continuation. I would like to enable this functionality or duplicate this type of paging in my application.

    Is DataService<T> the correct class to use if I need client-side paging/continuation in a browser? Is there a better way to expose the Azure table data?

    Tuesday, August 24, 2010 6:16 PM

All replies

  • The problem is, that the Data Service client LINQ provider only supports limited set of functionality. In particular it does not support all the queries the Data Service server will generate. In your above sample you're implementing a Data Service server over a Data Service client (the TableStorageContext is in fact a DataServiceContext and thus a data service client). This is usually not the best way to do this, mainly because of the mentioned limitations in the LINQ provider.

    So the question is, why do you need to wrap already existing data service (the table storage is exposed as an OData data service) with yet another data service, why can't you simply expose the table storage itself.

    This approach we call Astoria over Astoria, and it's really hard to make it work correctly. You can see a sample code of how one could go about it in the OData Provider Toolkit : http://www.odata.org/developers/odata-sdk#/media/7579/odataprovidertoolkit.zip

    It's under Experimental/AstoriaOverAstoria. But please do not use this in a production environment, it's really meant as a sample only.

    Thanks,


    Vitek Karas [MSFT]
    Wednesday, August 25, 2010 3:50 PM
    Moderator
  • Vitek Karas [MSFT]


    ... In your above sample you're implementing a Data Service server over a Data Service client (the TableStorageContext is in fact a DataServiceContext and thus a data service client). This is usually not the best way to do this, mainly because of the mentioned limitations in the LINQ provider.

    So the question is, why do you need to wrap already existing data service (the table storage is exposed as an OData data service) with yet another data service, why can't you simply expose the table storage itself.

    Vitek, you are correct - this implementation is implementing a Data Service server over a Data Service client. The only reason we are doing this is for simplicity and to avoid exposing any authorization key details and reuse the website authentication and authorization. We have a website where we want to display details from the Azure Table Storage, so the client has already been authenticated and we also need to ensure that only users with the correct access level can get data from the Azure tables. Essentially all I really want is a simple proxy to expose the underlying Data Service server while hiding any implementation details, and if possible to simplify the client-side exposure to any paging (NextRowKey/NextPartitionKey) details.

    Is it possible have a WCF service or even ashx that essentially just:

    • filters the allowed URI parameters
    • performs any authorization or custom logic
    • queries Azure Table Storage
    • retrieves the results
    • and finally passes the results back to the client?

    We still need to get json data from the service for easy jQuery access on the client-side pages as well.

    Thanks for your help.

    Wednesday, September 1, 2010 4:22 PM
  • It is possible, but hard. Simply put you won't be able to use WCF Data Services directly due to the limitations mentioned above.

    You could either no use the server piece, but use the client piece and expose the data manually. Or you could use the server piece but not use the client piece and read the data from the underlying table storage manually.

    Or you could write a true HTTP proxy which just passes requests/responses around. This is probably the simplest solution. Note that you will have to open the payloads of the requests/responses and fixup URIs in them (since the Azure Table Storage will use URIs of its endpoint, but you want all the URIs to point to your proxy). Other than that you should be able to do most of the requirements above.

    The cleanest solution would be probably to use the server piece and issue the queries to underlying Azure Table Storage manually. On the other hand this requires you to write IQueryable implementation for the queries you want to understand, which will be able to translate them into the URL query language. You might be able to use the WCF DS client to do this for you, but not to materialize the results as well. The advantage is that you get the correct OData behavior of your service "for free" (including ATOM and JSON and so on).

    Thanks,


    Vitek Karas [MSFT]
    • Proposed as answer by edhickey Thursday, September 9, 2010 5:39 PM
    Thursday, September 2, 2010 4:17 PM
    Moderator