Feedback to SSDS team

Question

We are interested in your wish list for new features -- a usage scenario for the new feature would help us understand the context of your needs

We are interested in things you would like to see improved.

---

Tonyp

Answer 1

I use REST, so here are the things I'd like to see for SSDS:

- support for HTTP Authentication (Basic at minimum, Digest is better) to make it easier to write desktop and other clients and braoden the reach of the service
- support for Caching Headers (Cache-Contol, If-*,ETag,Expires, etc.) to cut down on traffic and improve response
- support for Content-Encoding (gzip,deflate) to cut down on bandwidth
- use the Location header (redirects) on POST and possibly PUT instead of sending the entity back (to cut down on bandwidth for large entities)

I'd be happy to give some examples of each if you like.

Mike Amundsen

---

mamund

Answer 2

 

Mike,

Thanks for the feedback -- Currently the service is using basic HTTP Authentication can you send me an email if you are seeing issues with the authentican?

We have open items for your other comments and I'll be posting a list of the new features and changes every eight weeks when we update the service so keep a look out for the postings about new features/changes/fixes. 

my email is tonypet at microsoft . com

Thanks

---

Tonyp

Answer 3

Tony:

Thanks for the follow up. I was out for the weekend and will check into this today. If I have any additional questions, I'll ping ya.

---

mamund

Answer 4

Tony:

HTTP Basic Auth works just fine - not sure what I was doing before<g>.

And I'm looking forward to the results of the latest 'sprint.'

Thanks again.

Mike Amundsen

---

mamund

Answer 5

Suggestions on SQL Functions, etc.

Once you add paging, you really need support for sorting; so sorting would be high on my list of features. You might require Entities to include explicit sort fields - possibly as an attribute of the field element  (<my-field xsi:sort="true" />) or a special stika element  (<s:SortElement xsi:element-name="my-field" />) that is included with each entiity.

After sorting, aggregate functions would be my next request (COUNT(), SUM([element-name]), MIN([element-name]), MAX([element-name]), AVG([element-name]) - in that order, IMHO)

Also, it would be nice to be able to support page-size settings in the query. This would help in implementing things like TOP or RECCOUNT()-type queries.

---

Mike Amundsen http://amundsen.com/blog/

Answer 6

Totally a 'sideline' suggestion here...

Can you hide or move the non-SSDS threads from this area?  It's kind of a drag to have to sift through them to find the SSDS 'nuggets.'

---

Mike Amundsen [http://amundsen.com/blog/]

Answer 7

A few things:

1) The main thing I'd like to see is some sort of revenue share scheme. I'm currently building an app that will (if I ever finish it) make heavy heavy use of SSDS. Its a command-line app which will, in effect, be chucking log data into SSDS. My thinking at the moment is that each user will have to provision their own authority and basically pass the URI for that authority into the app from the command-line. All quite simple really.
The thing is, the data isn't going anywhere near any servers that I own hence there's no way for me to track app usage and hence charge for it (which is what I'd like to do). Hence, my feature request is that you provide a mechanism for my app to identify itself and then Microsoft charge the user a slight premium over what they would normally pay for SSDS. Microsoft takes its cut for SSDS as appropriate and the premium comes to my company somehow. Like I say, its a revenue share scheme. A really compelling scenario for anyone that wants to use SSDS for providing application storage - as far as I know your competitors don't offer anything like this.
Headline here is: I want a way of getting at some of the revenue that is generated through users storing data in SSDS when using my app.

2) In this blog entry: http://blogs.conchango.com/jamiethomson/archive/2008/06/05/ssds-will-we-get-a-cloud-based-metric-aggregator.aspx I wonder about the possibility of you pre-aggregating data for us, thus speeding up queries. I got the idea from Analysis Services (SSAS) of course.
The big disadvantage of SSAS (in my opinion) is that its a seperate server process, seperate query language etc... I would rather issue queries against my base data and the engine be intelligent enough to know whether to go against the base data or against the aggregations. Given the abstraction that SSDS inherently provides I believe that it presents a great opportunity to do this. With the abstraction layer in place I don't care where my data resides - as long as I get it quickly. Pre-aggregating data can help here.


More to come.

-Jamie
P.S. I agree with Mike's suggestion about deleting the none-SSDS threads.

---

http://jamiethomson.spaces.live.com/ | http://blogs.conchango.com/jamiethomson

Answer 8

My vote would be some synergy for ANDS and SSDS as well as an Authentication Scheme that was valid and as secure as possible from Silverlight (true security isn't strictly possible, but we can do some tricks to make it harder to crack).

More SQL syntax would also be welcome (Aggregators, LIKE, type conversion). Also, I wouldn't be opposed to optional schema...it looks like right now its 'add a property and it exists' which makes it a little too easy to add a misnamed property and voila your object has that property.  Its unclear to me so far whether *only* that entity would have it or all entities of type X would.

---

http://www.silverlight-tour.com

Answer 9

Something similar to SQL  Server's EXEC sp_spaceused 'dbo.myContainer' to check the current size of a container.

Answer 10

c.c.chai said:

Something similar to SQL  Server's EXEC sp_spaceused 'dbo.myContainer' to check the current size of a container.

Out of interest...why do you want that? sp_spaceused is typically used by someone who cares about disk space and such things like that... who would you care how much space a data service is taking up?

---

http://jamiethomson.spaces.live.com/ | http://blogs.conchango.com/jamiethomson

Answer 11

The current container size limit is 2GB. There will always be a limit no matter how SSDS team changes the threshold.

If I want to store BLOB data in my container, it will be nice to know how much space is left, rather than get a strange exception suddenly.

Answer 12

c.c.chai said:

The current container size limit is 2GB. There will always be a limit no matter how SSDS team changes the threshold.

If I want to store BLOB data in my container, it will be nice to know how much space is left, rather than get a strange exception suddenly.

+1

And actually, that 2GB limit is a bit...cosy. I assume what we should do once we hit that limit is to create another container to serve as some sort of partition, and then run our queries against both containers?

In other words, could something like Flickr ever be built on top of SSDS with the 2GB limit?

Thoughts anyone?

Answer 13

Kwez/c.c.chai


FWIW,  I'm curious on how the whole BLOB  thing will play out.  I  *assumed* that  SSDS is a database-type store (ala  Amazon SimpleDB) and *not* a  content store (ala Amazon S3).  I'm cool with  combining the two as long as it does not produce barriers (lowered performance and/or frustrating storage limits).

For now, I plan on focusing on the database-type approach while I explore the BLOB aspects. But I suspect using SSDS as a primary content store will not take advantage of it's primary strengths.

  

---

Mike Amundsen [http://amundsen.com/blog/]

Answer 14

Guys,

I fed this stuff back to the DPE guys in the UK and not sure if it reached you, so posting it here.

1. How about being able to access an entity field directly through SOAP or REST rather than fetching the whole entity?
2. If you could do this, then what about being able to turn of the XML packaging of the response and get raw data particulary for blobs (is this coming in sprint3)? I built an SSDS uploader for map tiles from MapCruncher and have to proxy the tiles through an http handler on my own server in the web app. I'd rather not have the traffic coming through there but direct to the ajax client.

3. To use SSDS from Ajax and for a public site, how would you authenticate without having to store the SSDS credentials in javascript? It would need some kind of secure token issuing service or do you see the integration coming with the Identity services in Biztalk services?
4. I spotted a minor bug that has probably already been reported, but if you try to reate an authority that already exists, it increases the version number.

Kind regards,

Richard.

Answer 15

Richard:

Just jumping in here...

for authentication w/ a browser, I implemented the Basic Auth hashing on the client and then pushed that into a cookie for the server. The server then checks first for the Basic Auth header and, if that fails, checks for the known cookie. This is no less secure than the HTTP Auth header itself.  You can check out the source at the SSDS Examples project in Codeplex for details.

---

Mike Amundsen [http://amundsen.com/blog/]

Answer 16

Mike,

Thanks for the tip, I'll take a look at Codeplex once the site is back up!

Richard.

Answer 17

Richard:

Yeah, CP is down for me, too. You can also pick up the project code via my GoogleCode space.

Cheers.

---

Mike Amundsen [http://amundsen.com/blog/]

Answer 18

Kwez said:

In other words, could something like Flickr ever be built on top of SSDS with the 2GB limit?

 

We sure hope the answer is yes.  I'd like to mention a few things here.  First of all, the blob storage doesn't count against the 2GB container limit.  Second, we are looking at a set of APIs that will allow you programmatically access information on your containers (for example, how much space you've used up), so you can make informed decisions about whether or not you need a new container at runtime.

Let me know if you have other suggestions,
Stan

---

Program Manager, Cloud DB

Answer 19

I've played with SSDS for a while now, and work with lots of customers who might validly consider SSDS as a utility service in the near future.  For me, the features that would really raise the game for SSDS and increase the chances of adoption are (in approximate order of complexity!):

• Ability to specify and selectively enforce acceptable set of schema(s) for a container (schema(s) could be just 1, which would increase the reach/use cases of SSDS while keeping the container flexible)
• Abilty to enforce a minimum set of constraints (I would suggest UNIQUE, PRIMARY KEY, FOREIGN KEY)
• Features to query data usage of containers for an Authority (essential for any S+S strategy)
• Ability to automatically create a new 'partitioned' container if the capacity was exceeded.  Coupled with some form of view or concept of a logical container (so the app doesn't have to understand the container 'partitions' and the rules about how/when to partition can be defined at the authority level ahead of time).
• Flexible security model (authorisation) - shouldn't just rely on LiveID but that would be a start...!  For example, in Identity Services the cardspace tie-in was removed adding support for passing a plain text user / password to the service.  SSDS needs therefore to support multiple authentication methods, but should have a common method of authorisation regardless of the authentication scheme (e.g. group concept and the equivalent of ACL's on entities or containers).

---

--- RW

Answer 20

@RWil

Sorry I don't agree with your first 2 feature request. Cloud DB should not have those features, IMO, those obviously belong to a traditional RDBMS. Customers need to start thinking differently when dealing with cloud DBs.

@KWez

Yeah for sure you can. Even Flickr does not store all their data in one DB, it's partitioned across various shards (farms). You could do the same thing here ex: by assigning users A-L in one container M-Z in another container. Or split it up even more. Or even by authority. Up to you.

Answer 21

Geospatial support would be awesome. For instance, I'd like to load in a series of points, and then run a query against them to find points of interest closest to the user.

That, and some support for either anonymous or a specific credential read-only access. I'd like to deploy an application that can query SSDS directly, but wouldn't want to put read/write credentials in the app.

Answer 22

There is work being performed to this end.  As we align the different initiatives across SQl and Microsoft in general, we should be seeing some rich functionality and user based scenarios enabled. 
Look for some of this scenarios being enabled at PDC.

-Anil

Answer 23

When doing some coding, I notice we need wildcard filtering support, e.g.

from e in entities where e["Name"] Like "%bob%" select e

Answer 24

I have only been looking into SDS for a little while but I haven't seen this anywhere so I thought I would throw it out there:

Wildcard characters or LIKE operators in the where clauses.

The one thing I have not figured out how to work around was supporting simple adhoc user searches.  If I have 1,000,000 widgets in the system how do I facilitate an end user in finding a single widget when they only have a vauge recollection of the name.

Something like (* is the wildcard):

from widget in entities where widget["Name"] == "Sta*" select widget

Answer 25

We need "Skip" to be implemented.  Can you give us a timeframe for this? 

Answer 26

Here is my bucket list...

"I would like schema / metadata support so that I can generate C# data access statements that will perform my basic crud operations on entities and containers in the cloud.  As a software engineer, I find writing data access code very boring and mundane.  This will allow me to focus on what I do best, solve business problems, and not write Linq expressions."

"As a software engineer, I have data in existing SQL 2005 databases that I need to migrate to SDS.  I would like a set of tools that will help me accomplish this task."

"Often times I use the processing power of the database server to fetch multiple result sets in one shot.  For example, I may want to fetch a customer, their orders, their order payments, and the order detail for each order.  I want to avoid multiple round trips from the services layer to the database and I would like to get back a CustomerEntity, OrderEntityCollection, OrderPaymentsCollection, and OrderDetailsCollection for each OrderEntity in the OrderEntityCollection."

This may be a really long shot, but here goes... "As a software engineer that is used to writing ORM style data access operations, I would like Inserts, Updates, and Deletes to cascade.  For example I would like to write the following code....

TasEntity task = new TaskEntity(){field info goes here};
task.Details.Add(new TaskDetailEntity(){field info goes here};
task.Details.Add(new TaskDetailEntity(){field info goes here};
context.AddObject("Entities", task);
context.SaveChanges();

and when this code executes, I end up with 1 new task and 2 new related task detail entities.  Repeat the operation for deletes and updates"

As I play around more, I may come up with some more compelling stories.