EF: Best way to compare apples to apples when comparing perf of materialized objects vs datareader?

Question

I'm sorry I didn't do this in BEta2 for another point of reference, but I didn't want to even think about performance then. But now it's time!  I'm interested in comparing the performance for materializing objects to streaming data with EntityClient since that is one of the big improvements for Beta3.

 

With a datareader, you haven't really finished streaming until you have iterated through the results, so I am doing that in both cases.

 

In the first I am timing this way:

 

Code Block

Dim custs = aw.Customer

Dim sw As New Stopwatch

sw.Start()

For Each cust In custs

 objcount += 1

Next

sw.Stop()

Dim MaterializedObjectsTime = sw.Elapsed

 

In the second I use this code:

 

Code Block

Using awconn As New EntityConnection("name=AdventureWorksLTEntities")

 Dim query As String = " Select VALUE c From AdventureWorksLTEntities.Customer as c"

 Using cmd As EntityCommand = awconn.CreateCommand()

  cmd.CommandText = query

  awconn.Open()

  sw.Reset()

  sw.Start()

  Using rdr As EntityDataReader = cmd.ExecuteReader(CommandBehavior.SequentialAccess)

   While rdr.Read

     rowcount += 1

   End While

  End Using

  sw.Stop()

 End Using

End Using

Dim streamedDataTime = sw.Elapsed

 

 

When all is said and done, I verified that objcount and rowcount are equal (verifying that I have iterated through all records in both cases), yet the first test is 100-150 times slower than the second.

 

Am I using unfair examples here? NOt comparing apples to apples (well, the best we can considering they are completely different realizations)? Is there a better way to see /prove that the performance for materialized objects is closing in on streamed?

 

Just as a note of interest, SqlClient seems to perform the query/iteration about twice as fast as entity client.

Answer 1

Can you make sure that the cost of querying "aw.Customer" does not include view generation? Viewgen is something that needs to happen before you can read from EntitySet. This usually happens when first query is executed so it adds to the preceived run time of that query.

 

You can eliminate the cost of view generation by pre-generating views at build time using EdmGen.exe /mode:viewgeneration

 

For comparison purposes it would be best if you either did that or forced viewgen at the beginning of your program (by running something like aw.Customer.First()). Views are cached at app-domain level, so you can use a temporary EntityConnection/ObjectContext for triggering viewgen.

Answer 2

 

Got it on the query generation

So I separated out the view generation time by calling First on the query to get that out of the way.

 

I also called DBCC FreeProcCache in between each test to ensure that SQL Server's query compiling didn't impact the results.

 

The resulting times looked like this:

 

ViewGneration: ranges around 1110 milliseconds

Iterate through Materialized Objects: 14 to 15 ms

Iterate EDM shaped data through datareader: 17 to 18 ms

Iterated sqlclient datareader (classic ado.net): about 4 ms

 

Is this what you would expect to see?

 

Oh and where should I put the EDMGen? I tried it as a pre-build event and postbuild:

 

EDMGen.exe /mode:viewgeneration  /inssdl:Model1.ssdl /inmsl:Model1.msl
    /incsdl:Model1.csdl

 

but it threw an error.

 

Thanks

Answer 3

OK. Here's step-by-step recipe for including precompiled views in your project:

 

1. Add *.edmx to your project

2. Save everything

3. Right-click on the project in Solution Explorer, select "Unload Project"

4. Right-click again, select "Edit projectname.csproj"

5. Add this piece of XML somewhere before the closing Project tag

 

Code Block

<Target Name="BeforeCompile" Inputs="$(ProjectDir)\Model1.edmx" Outputs="$(ProjectDir)\MyViews.cs">

<Exec Command='%25WINDIR%25\Microsoft.NET\Framework\v3.5\EdmGen.exe /nologo /mode:viewgeneration "/incsdl:$(TargetDir)\Model1.csdl" "/inssdl:$(TargetDir)\Model1.ssdl" "/inmsl:$(TargetDir)\Model1.msl" "/outviews:$(ProjectDir)\MyViews.cs"' />

</Target>

 

 

 

Change items in bold to match your naming convention. For VB you have to add /language:VB.

 

6. Reload your project

7. Build it once

8. Add MyViews.cs to be included in compilation

 

BeforeCompile target is called after Edm files (*.csdl, *.ssdl, *.msl) have been deployed to bin\Debug, but before they are compiled. I have just verified that it works fine with clean project (when bin\Debug) does not exist.

 

Answer 4

Cool, I'll try this out in a short while.

 

ANy thoughts on the timings of my tests?

 

Thanks!

Answer 5

The timings are pretty reasonable.  The exact numbers and ratios between the various techniques will depend a lot on the complexity of the model and other factors, but view generation is typically very time consuming (which is why we made it possible to do it at compile time) while executing queries through object services has improved dramatically to the point where for some scenarios it is the same speed or better than using the DataReader on EntityClient (as you observed).

 

- Danny