how to locate the location of block in the Stream Layer for the row of Windows azure storage table?

Question

According to  the following paper  "Windows Azure Storage: A Highly Available Cloud Storage Service with Strong Consistency", the block is the minimum unit of data for writing and reading ,that is to say,if I want  to read a row, we must need to locate  the location of block  in the Stream Layer.

So I think the steps to locate  the location of block  are:

1 get the block number/extent offset Based on the row

2 use that block number/extent offset getting from step1  to locate the block in the extent in the  Stream Layer

So my questions are:

1 how to get the block number/extent offset Based on the row?

2how to use that block number/extent offset to locate the block in the extent in the  Stream Layer?

Are there any map for row to block number/extent offset   and map for block number/extent offset to  location of block  in  Partition Layer.?

I just cannot find those information in that paper.

http://blogs.msdn.com/b/windowsazurestorage/archive/2011/11/20/windows-azure-storage-a-highly-available-cloud-storage-service-with-strong-consistency.aspx

http://sigops.org/sosp/sosp11/current/2011-Cascais/printable/11-calder.pdf  

Answer 1

You're asking about the internals of how the table storage service works? Why?

Answer 2

Hello Steve,

When I study one technology , I am accustomed to know what's that tech can provide, and how it works instead of just knowing the API.

Because I think if you not only know the API but also know the internal mechanism for that tech, you can have a more deep understanding for that tech, and also the API keeps changing, but the  internal mechanism  is keep steadily. So I don't think you master a tech if just know the API.

That's  my main reasons. 

Answer 3

valid reason to a certain extent. However one of the points of an API is abstraction from underlying implementation. MSFT publications such as the above mentioned may help you understand the platform and contain a whole lot of information. For further details also recommend watching the various tech talks and conference sessions available.

But you should also be careful using anything beyond the API knowledge. It's typically the opposite of what you describe, implementation may change at any point, and as long an API remains stable this wouldn't impact any solution beside the ones which try to trick the api.

So yes, there may be a map to locate a row in a block and the block in the partition. But do you need this knowledge anywhere in your solution? You can still master the tech, you're the user not the vendor. Like driving car vs building cars - both can have master, but I think the driver champions are more famous ;)

Answer 4

@perpetualKid

thank you so much.

"So yes, there may be a map to locate a row in a block and the block in the partition. "

But I didn't see that description about "row to block" map in the paper. 

Answer 5

I'm not sure the answer is public information.

Answer 6

Hello Steve,

So you means the questions, which I asked , aren't answered in that paper?  so maybe,it isn't a  public information?

Answer 7

Hi Jacky - as Steve said, we haven't publically stated that information on the internals of the service.  Internals can change over time, so it's best not to rely on them.  Is there a specific problem you're trying to solve, that we might be able to recommend an approach to?

---

-Jeff

Answer 8

Hello Jeff,

I want to internals of the service just for learning and study. 

Answer 9

Hi Jacky - our blog contains most of the internals we've made public at this time: http://blogs.msdn.com/b/windowsazurestorage/

The two papers we've published that are most "internal" are both referenced here (one referenced is the SOSP paper you've already read), but neither answers your original question directly.
http://blogs.msdn.com/b/windowsazurestorage/archive/2012/06/13/usenix-best-paper-award-erasure-coding-in-windows-azure-storage.aspx

Also, for more information, you can view some of our past BUILD and PDC talks here:
Brad Calder from BUILD 2011: http://channel9.msdn.com/Events/BUILD/BUILD2011/SAC-961T
Jai Haridas from PDC 2010: http://channel9.msdn.com/Events/PDC/PDC10/CS11
Brad Calder and Jai Haridas from PDC 2009: http://channel9.msdn.com/Events/PDC/PDC09?sort=sequential&direction=desc&term=&s=Brad-Calder&s=Jai-Haridas

Keep watching our blog and activity at major conferences for more information in the future!

Thanks for the interest in our service!

---

-Jeff

Answer 10

Hi Jeff

Although I still didn't get the answer, anyway ,thank you for providing those information to me.