Performance of Soap compared to Rest

Question

I've just done a simple test of uploading 100 22Kb images. Uploading them as a Blob using Rest took 111s, uploading them as a byte array in a Soap Entity took 54s. That seems like a signicant difference? Naively I guess I would expect the 2 protocols to be close to each other in performance?

--
Dan 

Answer 1

I've found similar differences in performance between SOAP and REST using data from Northwind. See Updated SQL Server Data Services (SSDS) Test Harness: Northwind REST and SOAP Uploads, which shows roughly the same performance ratio for ordinary data.

---

--rj (OakLeaf Blog)

Answer 2

i've been watching this for some time, too.

i suspect that SOAP is the 'native' interface for SSDS and that REST requests are first captured, then converted into SOAP calls and passed to the SOAP interface.  it'd be nice if i were wrong on this, but i haven't heard any different.

---

Mike Amundsen [http://amundsen.com/blog/]

Answer 3

You're wrong on this.  :)

REST and SOAP are two separate interfaces that both boil down to a common layer.  There is no SOAP dependency on REST and vice versa.

Answer 4

Ryan:

thanks for the quick post. glad to hear i'm off base on REST/SOAP dependencies, too.

that said, i am surprised to find this perf difference.  seems counter-intuitive since i *assume* SOAP payloads will always be larger than REST payloads (both GET/DELETE and PUT/POST varieties) and that the SOAP payloads will require more processing. i'm esp. suprised that Dan reports binary uploads take longer via the REST interface.

has the team found any details on this?

---

Mike Amundsen [http://amundsen.com/blog/]

Answer 5

I'll chime in here...

To get a true apples to apples comparision, you would upload both as byte arrays. Blobs are handled differently than byte arrays internally in SSDS.

-Dave

Answer 6

Dave:

thanks for the reminder.  i was also referring to the work Roger did some time ago (see his link in this thread) comparing uploading the Northwind data (not BLOBs) using both SOAP and REST.  looked like REST was coming close to twice as slow as SOAP.

is this just client-coding issues or is there more to the differences?

---

Mike Amundsen [http://amundsen.com/blog/]

Answer 7

Thanks for the quick response, I'll try the equivalent byte array Rest example and post the results here.

Performance does look like it's going to be an issue though, even the good case indicates 1/2s to upload 22kb.

Download is quicker, ~20s to get 100 22Kb arrays, but that's still relatively slow ?

--
Dan

Answer 8

Mike,

Due to protocol differences between SOAP and REST, SOAP theoretically can be slightly faster than REST, but that speed difference is negligible. To be honest, I have seen times where SOAP is faster and I have seen times where REST is faster. Please keep in mind that the time at which these operations take to complete is affected by your connection to the internet and the route that request must take to get from you to us.


DRD

How long does it take to upload a 44kb file? a 66kb file? Try it out. I think you will see the times to be the same. The net-net here is that there is overhead in making that roundtrip to the server. I would expect that the majority of the time is in the roundtripping, not it the actual insert. This will get better once we add batch support for entities and partial reads/writes for Blob entities.


-Dave

Answer 9

Dave:

I agree that the diffs between SOAP and REST should be negligible. I just wish someone would post results against SSDS that showed that. So far, the only results I've seen show a pretty remarkable difference. I understand about connection costs, networks, and such. That's why Roger's tests caught my attention. I thought he did a pretty good job of accounting for variances of that type and he still showed a significant speed difference.

I'll leave this alone for now. As long as the differences are client- and/or network-related  and not something 'systemic' in the SSDS code base, then it's not a big deal.

Thanks for listenin'

---

Mike Amundsen [http://amundsen.com/blog/]

Answer 10

Mike,

2 things...

First, I'll add to my ever growing list of things to do to run some tests and publish the results for you folks...

Second, Don't feel the need to "leave anything alone for now" If this is something you want to know, say so. There are plenty of issues we need to just say "No Comment" because we have things in flight that we can't talk about. Why we can't talk about is another issue I am working on (transparency/translucency). This topic is not one of them...Know what I mean?

Answer 11

Dave:

Thanks for offering to get us some test results. All info is helpful.

And I appreciate the level of sharing going on here. I respect the fact that some stuff is 'off limits' for now. I'll still ask until I hear otherwise [g]. 

Frankly, the only negative I have about things so far is that you guys 'go dark' every once in a while. I figure you're hip-deep in the current sprint details, etc. Hence my 'hand grenade toss' about SOAP as the code base just to rattle some bars over there [grin].

Seriously, I'm up to my ears in the BLOB stuff right now, enjoying every minute of it, and perf is not on my mind. I  figure it'll take me a couple weeks to 'play out' this sprint before  I start  scanning the room for my next target.

Keep 'em flyin'

BTW - I finally got a good look at the creepy cartoon head I posted for my profile last week - yipes! It's gotta go...

---

Mike Amundsen [http://amundsen.com/blog/]

Answer 12

Hi David,

   Here are some more test results

   Create

                                    22kb       78kb      134kb    210kb           
            Rest Blob Create       91s       137s       175s     216s
            Rest Byte Array        50s       114s       171s     240s
            Soap Byte Array        39s        77s        96s     144s

   Query
                                  22kb       78kb      134kb    210kb           
            Rest Byte Array        17s        34s        59s      91s
            Soap Byte Array         9s        32s        55s      93s

   So SOAP wins quite handily, not much difference between Blob and Array on the REST side apart from maybe when the data size is small?  Querying doesn't exhibit the same disparity. I only timed querying the array case as you get that data back in 1 query, it would take 100 queries for Blobs (?)

--
Dan

Answer 13

Dave:

Glad to hear "you're working on transparency/translucency."

I've been agitating for increased transparency on the part of the SSDS team based on the approach started by Pablo Castro for Astoria and continued by Tim Mallalieu for Entity Framework.

See “A Vote for Transparency” Guest Opinion for Visual Studio Magazine’s August 2008 Issue and my SQL Server Data Services Needs an Open Development Process Similar to that for ADO.NET Data Services and Entity Framework post of June 27, 2008.

Keep at it.

Cheers,

 --rj

---

OakLeaf Blog

Answer 14

DRD said:

<snip>
  Query
                                  22kb       78kb      134kb    210kb           
            Rest Byte Array        17s        34s        59s      91s
            Soap Byte Array         9s        32s        55s      93s
</snip>

thanks for posting these. it's encouraging that queries seem to show a smaller disparity in your example. since queries usually account for the larger portion of app activity, that's a plus.

---

Mike Amundsen [http://amundsen.com/blog/]

Answer 15

DRD said:

   Query
                                  22kb       78kb      134kb    210kb           
            Rest Byte Array        17s        34s        59s      91s
            Soap Byte Array         9s        32s        55s      93s

Seem to me SOAP query is getting slower at a greater rate when entity size grows. I wonder what if the entity size grows beyond 300 kb - SOAP is slower or not?

Answer 16

!DOH!

i got it backwards!

thanks for the catch.

---

Mike Amundsen [http://amundsen.com/blog/]