none
Unacceptable performance for more than 50 records

    Question

  • We have created simple product management application using LightSwitch. Unfortunately, one of requirements from our customer is that we disable paging or at least set it to display reasonable number (500) of rows in a single page. So, we tried to fullfil those requirements and removed paging for some queries.

    And the result was huge performance hit. Screen with 2600 records opened in 30 seconds. At first we thought that slowest part is SQL data retrieval, but when we looked in SQL Profiler, it showed that SQL request took only 1-2 seconds. During window opening there is network activity for approx. 3 seconds, then CPU usage goes up to 50 percent and stays there until screen fully loads. So, we think that the problem is with data serialization/deserialization or rendering.

    OK, you could say: If you request so many records, of course, you have to wait a little longer. Okay, I agree. But waiting for screen to load is just one problem, even fully loaded screen with more than 50 records is laggy and scrolling is really slow.

    Maybe somebody of LS devs can tell whats going on? Why it takes so long for screen to open even if all the data is transfered in 3 seconds? Is grid control really so slow? Is there anything we can do about it?

    Framework for building bussines apps which can handle only 50 records on the screen at the same time does not sound serious...


    Wednesday, August 31, 2011 7:35 PM

Answers

  • Kaspars, can you do a quick test for me? Take the screen that shows the problem and for the grid control on that screen disable Export to Excel feature. Then run the app again as a browser app and as a desktop app and let me know if you still see a big perf. difference between these two.

    Thanks! Karol

    • Marked as answer by Kaspars Ozols Friday, September 30, 2011 10:29 AM
    Tuesday, September 27, 2011 9:42 PM
    Moderator

All replies

  • Hi Kaspars,

    Could you please provide the following information about your application.

    1.  What the data model looks like for the data bound to the screen.

    2.  Which properties/fields are displayed on the screen.

    3.  What type of screen you are using?  List and Details, Grid, etc.

    Thanks

    Wednesday, August 31, 2011 9:37 PM
  • We have created simple product management application using LightSwitch. Unfortunately, one of requirements from our customer is that we disable paging or at least set it to display reasonable number (500) of rows in a single page. So, we tried to fullfil those requirements and removed paging for some queries.

    And the result was huge performance hit. Screen with 2600 records opened in 30 seconds. At first we thought that slowest part is SQL data retrieval, but when we looked in SQL Profiler, it showed that SQL request took only 1-2 seconds. During window opening there is network activity for approx. 3 seconds, then CPU usage goes up to 50 percent and stays there until screen fully loads. So, we think that the problem is with data serialization/deserialization or rendering.

    OK, you could say: If you request so many records, of course, you have to wait a little longer. Okay, I agree. But waiting for screen to load is just one problem, even fully loaded screen with more than 50 records is laggy and scrolling is really slow.

    Maybe somebody of LS devs can tell whats going on? Why it takes so long for screen to open even if all the data is transfered in 3 seconds? Is grid control really so slow? Is there anything we can do about it?

    Framework for building bussines apps which can handle only 50 records on the screen at the same time does not sound serious...


    I am finding a similar problem - clients do not like or want paging - I have  a table with about 1700 records now that is the basic unit of what is being viewed.  The startup screen is a search screen and this has no trouble with this many records - quite fast and I even have some column bindings for color highlighting based on value. 

    The bizarre thing is I can add a detail screen for a single record view of something different - then if I add the data item from the search screen to this screen as a completely unrelated entity with its own datagrid that looks just like the first screen it works fine with paging, but without it will hang seemingly forever (although there's lots of thread exiting in the trace) and eventually produce a garbled screen.  Watching memory (when run OOB) it will just keep growing in the the more than 1GB range while this is happening...  There is nothing about this screen that should make it different than the search screen other than it contains a single record of another entity....

    Wednesday, August 31, 2011 11:50 PM
  • smells like a stuck loop or something to me.  Wonder if you can Break All when this is happening and check the Threads debug window and sniff around.
    Thursday, September 01, 2011 12:53 AM
  • smells like a stuck loop or something to me.  Wonder if you can Break All when this is happening and check the Threads debug window and sniff around.

    I actually did do this but almost all of the threads had no info, the main thread was in its modal loop and the worker threads are all in socket operations - I assume talking to the RIA service...my next step is to change the inital search screen to look like the detail screen to see if the same thing happens - I have a feeling the problem has to do with the screen type - does anyone know if there are internal differences of how controls are populated based on screen type?

     

     

    Thursday, September 01, 2011 12:59 AM
  • We have used editable grid screen type for this aplication. Our customer wants to have only one screen for one operation. That is, to view and be able to edit all the data in the same screen for all the records at the same time. And we can do nothing about that. And I can understand the customer - as many others, they have used Excel prior their business had grown so much that they had to create a sepperate application. And it is hard to explain why previously in Excel they could view & edit thousands of records at the same time, but now using the new technology with fancy name LightSwitch he is limited to 50 or so.

    We did a little testing, our findings show that it does not matter what the data model really is if you use editable grid. We created dummy database and table with 6 columns, generated 6000 records and added it to LS as external data source. DB server is in our local network. We used SQL 2008 R2. It was 'flat' table without any references to other entities or calculated fields. Then we created screen based on Editable Grid template. We disabled paging for the query and once more screen became unusable. It took over 20 seconds to load and after that screen was so laggy that it was not possibe to use.

    For our particular application data model is little more complex. We have 18 columns - 15 DB fields, 3 computed fields, 5 relations to other entities. We could coupe with long loading time, but UI responsiveness is under any expectations. Maybe there are some way to speed it up?

    I guess LS team could go to Microsoft Dynamics AX team and ask how they have managed to create very fast grids with option to edit data as well. I know, they are not limited by SilverLight framework, but the know-how basicaly stays the same. :)

    Thursday, September 01, 2011 3:18 AM
  • This is highly unlikely. Screen templates control just the screen creation process, after that all screens are treated equally. Again, it would be helpful to describe the screen and the underlying data in more detail, so that we (LS team) can attempt to reproduce the problem.

    Thursday, September 01, 2011 3:25 AM
    Moderator
  • We have used editable grid screen type for this aplication. Our customer wants to have only one screen for one operation. That is, to view and be able to edit all the data in the same screen for all the records at the same time. And we can do nothing about that. And I can understand the customer - as many others, they have used Excel prior their business had grown so much that they had to create a sepperate application. And it is hard to explain why previously in Excel they could view & edit thousands of records at the same time, but now using the new technology with fancy name LightSwitch he is limited to 50 or so.


    I am curious, if you do not use LightSwitch, which technology will you use?

    Make Them Ask: That's a LightSwitch App?

    http://LightSwitchHelpWebsite.com

    Thursday, September 01, 2011 5:05 AM
  • Hey, don't get me wrong! I am not just complaining about things I don't like in LS and saying that it is bad and I won't use it. The idea of LS is really great and I really appreciate work of LS team and I REALLY WOULD LIKE TO USE IT! But, unfortunately, there are issues that are critical for my requirements. And I am posting in this forum hopefully to get those things fixed in V2.

     

    To Karol Zadara:

     

    I have prepared small sample to show the problem. It is based on AdventureWorks 2008R2 SR1 sample DB. There is not much in this app. I just added External DataSource and created default screens of type Editable Grid, Search and played a little bit with paging settings.

     

    Main performance issues:

     

    Loading time:

    Customer screen with 18000 records loads in ~60 seconds with DB on LAN. Sounds not too fast, but still reasonable until you take a look at resource monitor.

    NOTE: Look at the yellow line, it shows activity only from process sllauncher.exe.

     

     

    As you can see, it took only some seconds to retrieve all the data from DB on LAN. You can see that from network activity. What is going on for the rest of the time? Why CPU % is so high?

     

    Scrolling:

    * If I scroll up and down once the screen is opened, it works slowly, but is acceptable;

    * If I scroll grid horizontally to the right side, and then try to scroll down, scrolling is so slow that it is unusable;

     

    Once more, CPU usage is very high.

     

    Tabs:

    * Switching between tabs with screens with many records is painfully slow. CPU also is very high;

     

      

    May be somebody can shed a light on what happens under the hood during loading, scrolling and tab-switching?

     

    If you need, I can send you a working sample what illustrates the problem. But basically all I was doing was putting default screens on default AdventureWorks tables DimCustomers (~18000 records) and DimProducts (~18000 records). PM your email and I will send it to you.

     

    To ADefwebserver

     

    I try to use LigthSwitch. The fact, that currently there are not so many technologies to use in fast business application developement does not make my application to work better or my customers to be happier! Does it?

     

    Performace might be not as important factor if you develop appplications for yourself or internal company use, but when it comes to external customer, it is hard to explain why  application is sluggish.

    We have spent quite a bit of time to develop application for customer and now we are not sure if he won’t turn our created application down because of performace. If so, then we just have to stick with WinForms. It might not be the fastest development method, but certainly gives faster UI than LightSwitch.

     

     

     

     

    Thursday, September 01, 2011 10:04 AM
  • Just for comparing i put together simple form what loads data from the same DB in WinForms.

    Form opens fast, scrolling works perfectly. Unfortunately, a lots of manual coding...

     

    Thursday, September 01, 2011 10:42 AM
  • Hi Kaspars

    Analysis and solution of this case is very important, is a matter of performance optimization in which the Lightswitch team can help a lot, but it is required to enable you to make things easy.

     

    I think for everyone, the forum is important that these performance issues are properly addressed, you can help by allowing them to directly review the project to determine the cause.

     

    For that I suggest you put a copy of the project is presenting this serious flaw somewhere where they can download it, either Hotmail SkyDrive or directly Connect. And then put the link here.

     

    To make smaller the file size before compression, in LS do Build - CleanSolution.


    Jaime
    Thursday, September 01, 2011 1:14 PM
  • This is highly unlikely. Screen templates control just the screen creation process, after that all screens are treated equally. Again, it would be helpful to describe the screen and the underlying data in more detail, so that we (LS team) can attempt to reproduce the problem.

    I just tried a test :

    Create a detail screen for unrelated entity - remove that entity detail from the screen and the query for it as well.  Add the query and datagrid for exactly the same data on my initial search screen.

    Create a new search screen for the unrelated entity, delete the controls and query for that entity and add the ones like my initial search screen.

    Create 2 buttons on my initial search screen - one opens the detail screen, one opens the new search screen - both those screens have identical contents - the only difference is that the detail screen requires a parameter when called of an ID of the unrelated entity (passing in a hardcoded value that exists although it is never used on the screen any more).

    If I open the new search screen it is fine.  If I open the detail screen it hangs forever and will gobble up memory (I stopped after 1.6GB and started seeing OutOfMemory exceptions in the trace)

    Thursday, September 01, 2011 1:33 PM
  • This is highly unlikely. Screen templates control just the screen creation process, after that all screens are treated equally. Again, it would be helpful to describe the screen and the underlying data in more detail, so that we (LS team) can attempt to reproduce the problem.

    I just tried a test :

    Create a detail screen for unrelated entity - remove that entity detail from the screen and the query for it as well.  Add the query and datagrid for exactly the same data on my initial search screen.

    Create a new search screen for the unrelated entity, delete the controls and query for that entity and add the ones like my initial search screen.

    Create 2 buttons on my initial search screen - one opens the detail screen, one opens the new search screen - both those screens have identical contents - the only difference is that the detail screen requires a parameter when called of an ID of the unrelated entity (passing in a hardcoded value that exists although it is never used on the screen any more).

    If I open the new search screen it is fine.  If I open the detail screen it hangs forever and will gobble up memory (I stopped after 1.6GB and started seeing OutOfMemory exceptions in the trace)

    And to add to that I tried:

    Editable Grid screen with same procedure - works fine

    New Data screen with same procedure - same behaviour as detail screen. Hang with memory climbing out of control...

    There is definitely something under the covers on these screen which behave differently... I also put traces in my RIA service when each type of entity is queried, and during the hang nothing from that side is called....

     

     

    Thursday, September 01, 2011 1:50 PM
  • That is fair. I just wanted to know what options you are considering. If you were considering, for example, HTML5 I would remind you that JavaScript has unpredictable performance on the end-users machine.

    I think we must keep in mind that LightSwitch creates a Silverlight application and it is a "client side" technology (yes even when running "out of browser").

    That fact alone causes a trade off. However, I have been programing Silverlight for many years and it is the best client side technology ever.

    But, I always expect Windows Forms to provide better performance.

    Now, If I wanted to use LightSwitch but needed speed, I would use my own Silverlight custom control to display the data. I would also use my own WCF RIA Services to supply the data. In a professional application you need more control, this "control" is built into LightSwitch. Yes you will have to do more work, but you will still save a MASSIVE amount of time.

    This is the sort of thing we explore on the http://LightSwitchHelpWebsite.com . For example see this article:

    Using WCF RIA Services In LightSwitch To Simplify Your Application UI

    Also see:

    MVVM (4)
    Custom Controls (9)
    WCF RIA Service (4)


    Make Them Ask: That's a LightSwitch App?

    http://LightSwitchHelpWebsite.com

    Thursday, September 01, 2011 2:06 PM
  • To LS Team

    As I said earlier, my test project contains only default screens and entities generated on Adventure Works database. There is no customization at all, only thing what differs from detault settings is that paging for some test screen is disabled.

    Anyway, if it helps, I have uploaded my test project (see link below). To get it to work you have to change connection string of external DB to your DB with AdventureWorks DB. I don't know how to do that, but I think that for LS team it is no problem! :)

    https://skydrive.live.com/redir.aspx?cid=fae335a773d3b689&resid=FAE335A773D3B689!121

    To ADefwebserver

    At first I also thought that Silverlight is the thing to blame. In my understanding, it simply is not designed to host business applications. But my oppinion changed when I did some research on custom controls I could use in LightSwitch. I found this: http://demos.telerik.com/silverlight/#GridView/UIVirtualization and it completely changed my mind. Telerik's grid performs much better than LS default EditableGrid - I would say that on my PC Telerik's grid with one billion cells work even faster than LS grid with 10x50 = 500 cells. Of course, in this demo data is generated and not taken from DB what makes it much faster, but LS grid is slow even when all the data has been loaded.

    I don't expect LS to be as fast as Windows Forms application, I just expect it to be usable. I added screenshot from WinForm application just to show that there are no problems with network or whatsoever.

    Using custom controls would be fine if I needed some extra fancy window with custom functionality. Otherwise, it seems to be a overkill to create each window using custom control. I could make generic grid control, but it would be hard to add custom logic for it - I would have to do it in control level, because (as I understand it) custom control is treated as a black box and I have no control of it from LS screen. Also, I could not use benefits of standart LS (validation error handling, runtime designer, etc).

    I can't see how RIA services would help in this particular situation. There is no problem with data retrieval, it is quite fast and takes 2 seconds at most. At least, if I look at network activity I can see that transfer ends after 2 seconds or so. Rest of the time LS is just burning CPU.

    Did you take a look at graphs above? What is your opinion about problem? It seems that LS has hard time when it comes to DB response parsing and result displaying. Otherwise I can not explain high CPU usage. Does LightSwitch/Silverlight grid control use DirectX to render the grid?

    Thursday, September 01, 2011 2:38 PM
  • When I say use a custom control, that includes simply using a control from Telerik or installing a LightSwitch Control Extension :) This only takes a bit of money and about 10 minutes :)

    I bring up the WCF RIA Service because it is important to keep in mind that you have the ability to control how data flows into the application. For example, you could just use ADO.NET and call a stored procedure if you wanted to. When displaying data from multiple tables I find it easier to make my own WCF RIA Service that returns the data formatted exactly how it needs to be displayed.

    As far as the CPU usage, I would experiment with different Silverlight custom controls and note the differences.


    Make Them Ask: That's a LightSwitch App?

    http://LightSwitchHelpWebsite.com

    Thursday, September 01, 2011 3:39 PM
  • Telerik's grid performs much better than LS default EditableGrid - I would say that on my PC Telerik's grid with one billion cells work even faster than LS grid with 10x50 = 500 cells. Of course, in this demo data is generated and not taken from DB what makes it much faster, but LS grid is slow even when all the data has been loaded.

    I don't expect LS to be as fast as Windows Forms application, I just expect it to be usable.

     

    There is no reason why the LS Grid should be any slower than Telerik's or any other Control.

    LS out-of-the-box controls must support the best possible performance.  After all, LS is a Microsoft Visual Studio product.  It is our reasonable expectation that Microsoft should know .NET as well as anyone.

    Kaspars has done an exceptional job in presenting the problem to the community.

    Let's find out where the bottleneck is.   Karol, could you please give us some direction to create a controlled benchmark environment to track down where the bottleneck is?

     

    Thursday, September 01, 2011 4:42 PM
  • This is highly unlikely. Screen templates control just the screen creation process, after that all screens are treated equally. Again, it would be helpful to describe the screen and the underlying data in more detail, so that we (LS team) can attempt to reproduce the problem.

    I just tried a test :

    Create a detail screen for unrelated entity - remove that entity detail from the screen and the query for it as well.  Add the query and datagrid for exactly the same data on my initial search screen.

    Create a new search screen for the unrelated entity, delete the controls and query for that entity and add the ones like my initial search screen.

    Create 2 buttons on my initial search screen - one opens the detail screen, one opens the new search screen - both those screens have identical contents - the only difference is that the detail screen requires a parameter when called of an ID of the unrelated entity (passing in a hardcoded value that exists although it is never used on the screen any more).

    If I open the new search screen it is fine.  If I open the detail screen it hangs forever and will gobble up memory (I stopped after 1.6GB and started seeing OutOfMemory exceptions in the trace)

    And to add to that I tried:

    Editable Grid screen with same procedure - works fine

    New Data screen with same procedure - same behaviour as detail screen. Hang with memory climbing out of control...

    There is definitely something under the covers on these screen which behave differently... I also put traces in my RIA service when each type of entity is queried, and during the hang nothing from that side is called....

     

     

    I found the problem with this.  The very top "rows layout" on the screen has "Vertical scroll Enabled" checked on the Detail and New Data screens, and not on the others.

    If I uncheck on these versions of my screen they work fine.  If I check it on the Search and Editable Grid versions of my screen they stop working with unbounded memory...

    Basically, there is a bug with non-paged Data on a screen that has this "Vertical Scroll Enabled" checked.

    Thursday, September 01, 2011 5:51 PM

  • Basically, there is a bug with non-paged Data on a screen that has this "Vertical Scroll Enabled" checked.

    I just unchecked on a "List and Details" screen too (for the stuff in the right hand column) and HOLY SMOKES the performance is now where I would expect....

     

    Thursday, September 01, 2011 6:02 PM
  • SamyM,

    Thanks for your feedback and hinting at the source of the cause on this one.

    Kaspars, thanks also for raising this thread, Performance is an issue and perhaps the LS team can explain the processes that are causing these slow downs. I totally agree with Garth as regards LS screen and integral control performance, they should sing as good if not better than any third party offering and the framework should be optimised such that core screens and their control produce a user experiance that is acceptable.

    I await further comments and ideally feedback from the LS team, thanks, Keith


    E tenebris lux. ±
    Thursday, September 01, 2011 7:07 PM
  • One thing I noticed is that this slowness problem was componded with the RTM release.  Speed was better in the beta 2.  See this thread I presented some time ago.

    http://social.msdn.microsoft.com/Forums/en-US/lightswitch/thread/be8cd16a-0c00-47e0-b847-01d060a005a2

     

    I have been complaining of this for a month now and am glad to finally get some coraboration.  This is definitely something Microsoft should fix and soon.

    I checked my screens and none of them have the Verticle Scroll Enabled checked and yet I still get the unacceptable performance.  I don't have the measurements the Kaspers was able to present, but I do have users that complain daily about the speed issue (luckily they are internal users).

    Dave

    Thursday, September 01, 2011 8:47 PM
  • Hello, again!

    I have done some extra testing ang got new results. I did compare LS default grid control performance to performance of some third party grid control as ADefwebserver suggested. As the third party control I chose Telerik's grid control, because it has free trial and there was tutorial how to set it up. So I modified my test application and the results are very interesting.

    I did a test on DimCustomer table with contains ~18000 records and disabled paging at all.

    * Default grid control - screen opens in ~60 seconds

    * Telerik's grid without height defined - I waited more than 120 seconds, but it still continued to load;

    * Telerik's grid with height of 500px - screen opened in 10!!! seconds

    Resource monitor (NOTE: look at the yellow line because it shows data for process sllauncher.exe)

    As you can see everything is as you would expect - data loads, there is CPU % peak for data deserialiation and then it is done!

    My guess is that there might be a problem with layout engine (remember previous posts with UseVerticalScrollBar) because Telerik's control also didn't work normally when the height was not set. It might be that layout engine keeps on adjusting screen for every row created. It is just a guess. Anyway, something is clearly wrong there, it can not be possible that third party control gives 6!!! times better loading time and scrolling performance is near perfect!

    So, it would be interesting to hear something from LS team about this issue. Is there any way we could work around that? Maybe we just have to play around grid settings and disable all the dynamic sizing and aligning? It is late night here, so I probably will try it tomorrow. :) Feel free to express your opinions!

    I will keep you informed if I find something!

     


    Thursday, September 01, 2011 10:12 PM
  • This seems consistant with my comment about the Silverlight control being a factor.

    LightSwitch is not at fault because you can use ANY Silverlight control that works best for you. Silverlight is not perfect. Sometimes you have to tweek things a bit.


    Make Them Ask: That's a LightSwitch App?

    http://LightSwitchHelpWebsite.com

    Thursday, September 01, 2011 10:41 PM
  • "It might be that layout engine keeps on adjusting screen for every row created. It is just a guess"

    I think your guess sounds exactly like the issue from the observed behavior. Maybe they need to so something like a SuspendLayout/ResumeLayout or the SL equiv.  I remember similar behavior in WinForms.


    Friday, September 02, 2011 12:35 AM
  • Thanks to everyone for good feedback, we appreciate it, perhaps even more if it is not as good as we'd hope.

    I spent some time looking at Kaspar's application today and thinking about Samy's report.

    The behavior with "Vertical Scrolling Enabled" set to true on top-level row layout used by the screen is "expected". What it does is basically tells the grid that it has unlimited vertical space and should fill it. Well, the grid does, rendering all the entities in the underlying data modell all at once. A few thousand rows may not seem like a lot, but if you count all the properties, all the propety value objects (strings and so on), all the related controls, and their DependencyProperties and WeakReferences, not to mention all the native resources that back these managed UI objects, it is a lot of stuff to create and display. Don't do this. It does not make sense given that the grid has already its own vertical scrollbar.

    With Kaspars' application I do not see exactly what he observed. On my machine (it is a VM, the underlying physical hardware is quad-core 2.4 GHz machine with 4 GB of RAM) there is relatively little difference whether the screen fetches just a page of data or all 18 450 rows from DimCustomer table in terms of the time it takes to open the screen. It is slower without paging, but not excessively so, taking about 10 seconds to open this screen. Once the screen is opened, the amount of data on the screen affects the scrolling performance very little. What matters is how much new data is displayed as part of a single "scroll unit". For example, if I press just the down arrow, the scroll unit is one row. This works OK in terms of performance, not great, but OK. On the other hand if I press and hold Page Down key (scroll unit=one screen of data), the scrolling becomes quite slow and jerky. It gets better if I resize the app window so that less rows are displayed at once (smaller scroll unit), but it is not as fast as scrolling one row at a time.

    This is all with the standard LS grid; I haven't tried the grid control from Telerik.

    Let us know if this is what you see too, or if there is something else going on. Thanks!

    Friday, September 02, 2011 4:03 AM
    Moderator
  • It sounds strange that on your PC there is almost no difference whether the screen fetches just a page of data or all records. On my setup difference is HUGE. Also, load time you mention (10 seconds for all records) is 5-6!!! smaller than on my system.

    I made a video to show you how this TestApp performs on my PC so you can compare it to what you see on your side.

    https://skydrive.live.com/?cid=FAE335A773D3B689&id=FAE335A773D3B689%21123#

    In this video I tried to illustrate following problems:

    1) I have an average PC you could expect customer to have;

    2) Opening screen with Telerik's grid takes ~10 seconds. You can clearly see data requesting time (network activity) and deserialization/rendering time (CPU%). Grid scrolls a bit slow, but is still OK both vertically and horizontally. On sorting Telerik's grid does not make any extra DB request which is fine in scenario when there is no paging;

    3) Opening screen with default LS grid takes ~60 seconds. Data request takes really small part of the total time. Rest of the time LS is just burning CPU. Have no idea what happens for rest of the time. When I try to scroll vertically or horizontally then it starts to lag enormously. Sorting cause extra request to DB what feels wrong when paging is disabled because grid already has all the data needed;

    4) Switching between screens works very slowly. What makes it so slow? What happens under the hood?

    5) If I enable paging (45 records per page) grid still does not feel fast. Scrolling is slow and paging also could be faster;

    6) Closing screens with a lot of data also takes time...

     

    Please, take a look and I wait for you comment!

     

     


    Friday, September 02, 2011 10:52 AM
  • Yes, thank you for informative video, Kaspars. It indeed looks like grid rendering takes most of the time in the video. I see the glitches you see but they are not nearly as bad on my system. Hmm... :-(

    I would really like to understand what makes us work so hard, especially in the grid rendering scenario. Would you be willing to do me a favor and collect a trace when the screen is being loaded? If you agree, I would send you detailed instruction. The tool to collect the trace is a standard Windows SDK tool and it would not collect any personally identifiable information. My email address is KarolZ at (you know where).

    In the mean time I will try to do some poking around too--I will try to limit the VM usage to just one CPU and throttle it down. That might expose the problem in my environment.

    The query-after-sort behavior you see is expected. We do this to maintain maximum fidelity between the data source and the UI. We did consider client-side sorting, but never got to implementing it. Looks like Telerik control does it. Do they sort locally even if paging is enabled?

    Saturday, September 03, 2011 4:06 AM
    Moderator
  • There is no reason why the LS Grid should be any slower than Telerik's or any other Control.

    LS out-of-the-box controls must support the best possible performance.  After all, LS is a Microsoft Visual Studio product.  It is our reasonable expectation that Microsoft should know .NET as well as anyone.

    With all due respect Garth, I disagree. First off, MS were in a race against time to get v1 of LS released, and it wouldn't surprise me if they didn't have time to optimise everything. We have all seen issues around LS, and I'm sure a lot of these will be addressed in the next version.

    It's a sad fact that MS have often put out substandard first versions of software, including Windows itself. That's not to say that LS is substandard (I think it's superb, despite the issues), but it's nowhere near what I would expect (say) version five or six to be like. I know IT professionals who refuse to touch any new MS OS until service pack one is released.

    Also, you are assuming that MS programmers are the best in the business. Sorry to disappoint you, but they are generally no better or worse than you'd find in other big companies. I've seen some awful code come out of MS. That's not to say they are all bad, but they aren't all top-notch either.

    On the other hand, Telerik (for example, as they came up in this thread)  are working on a much smaller code base than MS, a much smaller product range, with a much higher and more direct sales-to-development ratio, so can more easily see the benefit of investing in this sort of work. MS on the other hand have to contend with a huge number of issues when deciding how to allocate resources, especially on a brand new product. It's not just LS either, programmers move around between teams, and their resource balancing will involve weighing LS priorities against those of other products, as well as priorities just for LS itself.

    So, I wouldn't be surprised if a third-party could beat a first release MS control. That doesn't mean to say that MS won't improve, but they aren't necessarily going to be the very best all the time.

    Regards


    If you're really bored, you could read about my experiments with .NET and some of Microsoft's newer technologies at http://dotnetwhatnot.pixata.co.uk/
    Sunday, September 04, 2011 2:47 PM
  • There is no reason why the LS Grid should be any slower than Telerik's or any other Control.

    LS out-of-the-box controls must support the best possible performance.  After all, LS is a Microsoft Visual Studio product.  It is our reasonable expectation that Microsoft should know .NET as well as anyone.

    With all due respect Garth, I disagree. First off, MS were in a race against time to get v1 of LS released, and it wouldn't surprise me if they didn't have time to optimise everything. We have all seen issues around LS, and I'm sure a lot of these will be addressed in the next version.

    It's a sad fact that MS have often put out substandard first versions of software, including Windows itself. That's not to say that LS is substandard (I think it's superb, despite the issues), but it's nowhere near what I would expect (say) version five or six to be like. I know IT professionals who refuse to touch any new MS OS until service pack one is released.

    Also, you are assuming that MS programmers are the best in the business. Sorry to disappoint you, but they are generally no better or worse than you'd find in other big companies. I've seen some awful code come out of MS. That's not to say they are all bad, but they aren't all top-notch either.

    On the other hand, Telerik (for example, as they came up in this thread)  are working on a much smaller code base than MS, a much smaller product range, with a much higher and more direct sales-to-development ratio, so can more easily see the benefit of investing in this sort of work. MS on the other hand have to contend with a huge number of issues when deciding how to allocate resources, especially on a brand new product. It's not just LS either, programmers move around between teams, and their resource balancing will involve weighing LS priorities against those of other products, as well as priorities just for LS itself.

    So, I wouldn't be surprised if a third-party could beat a first release MS control. That doesn't mean to say that MS won't improve, but they aren't necessarily going to be the very best all the time.

    Regards


    If you're really bored, you could read about my experiments with .NET and some of Microsoft's newer technologies at http://dotnetwhatnot.pixata.co.uk/


    In a bit of a defense for the Silverlight group since I watched the development of all these controls since Silverlight 1.0 (back then we did not have any data controls), there is always a trade-off between features and functionality.

    For example you could make the data grid simply show everything at one time. It would move a lot faster... until you had a lot of data. You put in a feature that only shows one page at a time, but then that causes problems when the grid has a certain sizing or certain features enabled. You can allow the developer to set certain "performance hints" but now the grid is hard for developers to understand and use.

    While the LightSwitch team is concerned with getting the "most common situation" working, if you are a pro dev, you will need to get your hands dirty and learn to optimize things.

    It is like a car, you cannot expect a simple low maintenance car to perform like a sports car.  


    Make Them Ask: That's a LightSwitch App?

    http://LightSwitchHelpWebsite.com

    Sunday, September 04, 2011 3:53 PM
  • In a bit of a defense for the Silverlight group since I watched the development of all these controls since Silverlight 1.0 (back then we did not have any data controls), there is always a trade-off between features and functionality.

    I wasn't criticising them, merely giving an opinion on why I thought Garth's suggestion that the MS controls should be faster than any third-party ones might not be right.

    While the LightSwitch team is concerned with getting the "most common situation" working, if you are a pro dev, you will need to get your hands dirty and learn to optimize things.

    It is like a car, you cannot expect a simple low maintenance car to perform like a sports car. 

    Agreed, although I expect to see future versions of LS improve things a lot, so we may end up with the best of both worlds.


    If you're really bored, you could read about my experiments with .NET and some of Microsoft's newer technologies at http://dotnetwhatnot.pixata.co.uk/
    Sunday, September 04, 2011 4:44 PM
  • Agreed, although I expect to see future versions of LS improve things a lot, so we may end up with the best of both worlds.

    If you're really bored, you could read about my experiments with .NET and some of Microsoft's newer technologies at http://dotnetwhatnot.pixata.co.uk/


    I could be wrong but I see this as a Silverlight control issue. Yes the LightSwitch team can do some things to optimize how some of the LightSwitch controls set things when you use the defaults, but in the end it is still just Silverlight controls on the screen so it is very much a Silverlight Core issue.

    When you are making Silverlight apps without LightSwitch you are expected to figure all this stuff out yourself :)

    But yeah you're right, the LightSwitch team may figure out how to tweek things to address issues that are not fully in their control (they are "customers using Silverlight") to make things easier.


    Make Them Ask: That's a LightSwitch App?

    http://LightSwitchHelpWebsite.com

    Sunday, September 04, 2011 5:37 PM

  • I could be wrong but I see this as a Silverlight control issue. Yes the LightSwitch team can do some things to optimize how some of the LightSwitch controls set things when you use the defaults, but in the end it is still just Silverlight controls on the screen so it is very much a Silverlight Core issue.

    Which begs an interesting question, are Lightswitch controls just Silverlight controls wrapped up for LS, or are they actually new controls? If they are wrapped up, then why doesn't the LS team provide for a neater way of wrapping other controls?

    My guess is that you would have something to say on this Michael, as you seem to be one of the experts on LS controls! What do you say?


    If you're really bored, you could read about my experiments with .NET and some of Microsoft's newer technologies at http://dotnetwhatnot.pixata.co.uk/
    Sunday, September 04, 2011 6:17 PM
  • Which begs an interesting question, are Lightswitch controls just Silverlight controls wrapped up for LS, or are they actually new controls? If they are wrapped up, then why doesn't the LS team provide for a neater way of wrapping other controls?

     

    My understanding is that the standard "LightSwitch Controls" are just nomal "Silverlight Controls" with a "LightSwitch Wrapper" using these methods:


    Make Them Ask: That's a LightSwitch App?

    http://LightSwitchHelpWebsite.com



    Sunday, September 04, 2011 6:23 PM
  • OK, that makes sense. I knew you'd have an idea!
    If you're really bored, you could read about my experiments with .NET and some of Microsoft's newer technologies at http://dotnetwhatnot.pixata.co.uk/
    Sunday, September 04, 2011 8:03 PM
  • To Karol

    Of course, I am willing to help to track this issue down! I sent you my contact information. Hope I spelled you company name correctly! :)

    To ADefwebserver

    Yes, indeed it migt be a problem with Silverlight grid control, but you never know. Maybe grid control is OK, but it is not correctly integrated with LS or it does not use any optimization. About "features vs speed". Telerik's grid as much more features than LS standart grid (grouping, filtering on columns, etc). So, we know that it is possible to make such controls. I hope better LS/SL controls will be created in next releases of LS and this is just a matter of time.

     

    Sunday, September 04, 2011 8:27 PM
  • Karol
    Do not miss this good chance to score a hit, finding that generates such a low performance and obviously the solution. I think everyone in the forum are aware of this.

    Jaime
    Sunday, September 04, 2011 9:32 PM
  • I published a tutorial that shows how you can put in your own DataGrid into a LightSwitch application:

    A Random Walk Through The LightSwitch Data Model

    I hope you will see that the data loads very fast. In my opinion, as fast as a Windows Forms application.

    Therefore, in my opinion, performance problems are related to the Silverlight Control and they can be fixed by creating the Silverlight Control manually.


    Make Them Ask: That's a LightSwitch App?

    http://LightSwitchHelpWebsite.com

    Saturday, September 10, 2011 4:47 PM
  • Very interesting work William. Vote +1
    Jaime If you found this post helpful, please: VoteAsHelpful. If it answered your question, remember to: MarkAsAnswer.
    Saturday, September 10, 2011 4:52 PM
  • We need a simple benchmark app that generates a million (ok, maybe 10,000 to start) stand alone records.

    2 Screens:  One with a Grid with paging turned on.  One with a Grid with paging turned off.

    If both screens perform equally well, then we add in more related tables and what happens.

    Eventually, we will replicate the problems that Kaspars has run into.

    I have priority tasks and cannot do this now.  However, if no one else gets to it, I will do this when I have time. 

    It is very important for us to have a benchmarking app to help demonstrate/educate our clients and prospects about LS technology.

    Jaime, I think this sort of project might be of interest to you.  Don't hesitate to email/skype me if you might be interested in doing this and would like my assistance.

    Of course, the benchmark app would expand to have more than grid detail benchmarks.

    I have been writing data conversion programs for my new LS systems that are reading multiple databases and doing a lot of interactive lookups/validations, etc.   The Server side processing is phenominal.  The use of LS 2011 with VS 2010 is incredible.

    We all thank Michael for his continuing work to dig into more LS technology details and share the results with all of us.  I also rely daily on William's expanding http://ourbizforward.com/LightSwitchTips as an LS reference guide. 

    LS is all about using .NET and Visual Studio to build and maintain RAD LOB and ERP solutions.  Most of our initial work will, logically, be to integrate and expand the functionality of legacy systems - although there are several of us that are building complete accounting and operational management solutions with LS.  . . . because it is so much faster than any other development solution and very affordable (only $199 for a while, eh?).



    Saturday, September 10, 2011 5:52 PM
  • Garth and others,

    just FYI, we are still looking at this problem here at Microsoft. During v1 product cycle we did look at the performance of the grid-no paging scenario of course. At that time we thought we have identified all the bottlenecks and that the performance was acceptable. The report from Kaspars shows that we probably did not look deep enough. So we are doing it right now, but it will take some time. I do not have any definitive answers yet.

    A benchmark app is a good idea. 10-20k of records is a good number--this much should be usable even with no paging. Please let us know what you find!

    Even more interesting and valuable would be to hear from you guys about typical performance-sensitive scenarios that you encounter in the apps you build. Then we could compare them with the set that LS team has identified and see if we have any gaps or wrong assumptions.

    Cheers! Karol

    Sunday, September 11, 2011 5:12 PM
    Moderator
  • Kaspars, Karol and Others,

    I am volunteering to produce a benchmark app & database derived from data from our (UK) Ordnance Survey site. If Kaspars, Garth .... or anyone has some design criteria to add, please let me know.

    And obviously, I don't want to duplicate effort so if someone has an alternative suggestion or wants to take the lead, please feel free else I'm open for collaboration/guidance.

    Keith

    Extract from site regarding use:-

    Changes to the OS OpenData Licence

    We now incorporate the new Open Government Licence into our OS OpenData Licence.

    The Open Government Licence, developed by The National Archives, is designed to provide a single set of terms and conditions for anyone wishing to use freely available government information. Importantly, this does not alter what users of OS OpenData can do. All the data available through OS OpenData is still open for commercial or non-commercial use without restriction.

    As before, we ask for little more than an acknowledgement of the source of the data.

    Site link: http://www.ordnancesurvey.co.uk/oswebsite/products/os-opendata.html

     FYI - There's 1.7 million unique postcodes to play with. That's what I've used in the past to test LS's lookup performance.

     


    E tenebris lux. ±
    • Edited by PlusOrMinus Sunday, September 11, 2011 6:41 PM
    Sunday, September 11, 2011 6:33 PM
  • Good on you Keith!

    Yann

    (plus ça change, plus c'est la même chose!)

    If you found this post helpful, please "Vote as Helpful". If it actually answered your question, remember to "Mark as Answer".

    This will help people find the answers that they're looking for more quickly.

    Sunday, September 11, 2011 10:38 PM
    Moderator
  • Yes, thank you for your offer, Keith! Much appreciated. I have a few questions/suggestions.

    1. Before you set out to build the app, could you maybe describe in a bit more detail what will the app test and what data it will be operating on?
      1. I assume it will focus on the data retrieval performance (as opposed to data update or startup time)?

     

    1. Even if we focus just on data retrieval, there are many things that could be subject to :
      1. Data set size(s)
      1. Number of properties for the main entity of each data set
      2. Types of properties (numbers, string, dates, addresses, pictures)
      1. Presence/absence of related data
      1. Presence/absence of calculated properties
      2. Overall screen UI complexity
      3. Usage of specific controls: standard LS controls (grid, list) vs. custom controls
      4. What is measured
        1. Initial data load
        2. Paging
        3. Sorting
        4. Searching
        5. Scrolling: using arrow keys, using PgUp/PgDown keys, by clicking on the scrollbar, by dragging the scroll position indicator

     

    1. Given the matrix of possibilities above I suggest to choose something that mimics closely an application that you (or someone) might want to build and use in his/her work, using the data set you want to leverage. Here at Microsoft, for "big" Visual Studio performance testing we use several real-world, large applications (stripped from proprietary information, of course), that our customers have shared with us. This way we test and optimize for very realistic scenarios, as opposed to focus on a synthetic benchmark that may reflect real-world needs only  to some extent.
    2. Finally I assume we are not focusing on how well LS is dealing with high-latency, low-bandwidth data sources, so I would like to be able to set up and run the data source locally. In this respect Kaspars did a great job by using a standard Microsoft sample database. It makes it very easy to reproduce the setup elsewhere.

     

    Hope that helps! Karol 

    Sunday, September 11, 2011 11:52 PM
    Moderator
  • Hi, all!

    At first, thanks to everyone!

    A lot of things have been going on for the past week and unfortunatelly I was not able to collect data for Karol. I will try to get on this ASAP. I have one more interesting finding about my previous test application. If I run application as a Desktop application, the startup is very slow (~60 seconds) as you can see in the previous video. If I change that to In-Browser application, startup times are significantly better and are closer to the times Karol mentions (~10 seconds).

    I hope to send in trace data during today. I will include both scenarious - Desktop and In-Browser application.

     

    During this week we had to finish finish our current application, we had look how to overcome following problems:

    * performance - display several thousands of rows on the same time without startup lag and be able to scroll through them;

    * show some information from related entities;

    * allow sort on EVERY!!! visible column.

    The workaround we found to work the best is to use DB views. We created DB view for every Search screen and made them to be Read-Only. It reduced UI complexity a great deal and scrolling works better. Then we added Add&Edit&Details screens for every entity and connected screens together with buttons. Still, the performance is not brilliant, but it will be good enought for our current application.

    Monday, September 12, 2011 2:34 AM
  • First of all, I really apologize for such a late reply, but past weeks have been like hell for me. I am working on my master thesis and it really takes the most of my free time. :)

    Finally I have managed to do the trace for scenarious with slow performance.

    I have found out that problem mainly is visible when I run LS project as Desktop application (it takes ~60 seconds for form to open). If I run it as a Web application, form opens ~5 times faster (~10 seconds).
    I did trace for both scenarios - in first trace application type is Desktop, in second - Web. SkyDrive file size limit is 100mb, so instead I used DropBox.

    Here are trace files:

    http://dl.dropbox.com/u/43223864/1501828_Desktop.etl
    http://dl.dropbox.com/u/43223864/2286890_IE.etl


    There is one more thing. From the time of first measurements I have upgraded my PC. Now I have quad core I5 with 16GB of RAM. At first, I thought that on the new PC my LS projects will run smoothly, but, guess what, I was wrong. If I run LS application as Desktop application on the new PC the same form opens in ~25 seconds. Still slow and this is above-mediocre PC. If I change application type to Web, it takes just ~5 seconds. And again it is 5x difference.

    For simple data viewing application I would expect that the bottleneck is database and connection speed & latency. For both tests (new & old pc) DB and connection was the same, but it gave very different results. Increase of processing speed decreased form loading time more than 2 times. Still, for Desktop application, even better PC didn't help and loading time of forms is still bad.

    I hope Karol & rest LS team will find a time to address this issue and trace files I created will be examined and it will help to find where the bottleneck is.

     

    Tuesday, September 27, 2011 9:14 AM
  • Kaspars,

    Pleased I held back and waited for your input. I think there's enough bedtime reading in your trace files for the LS team. I'm sure there's some clues as to why there is a difference in performance.

    Karol,

    Thanks for the reply and suggestions. As I initially said that I do not intend to duplicate effort for a benchmark app, I note that you hint that MS already has some already designed for their own testing. Would in not be easier if these were made available to the community? I understand it would be easier to use Northwinds et al, as they are generally available, and easier to access.

    Thanks, Keith


    E tenebris lux. ±
    Tuesday, September 27, 2011 10:03 AM
  • Kaspars, thanks much for sharing the traces, I will start looking at them today and compare them to what I have gathered in house. Hopefully they will provide some good clues.

    Keith, yes, we do have some benchmark apps here at Microsoft, but they tend to depend on other pieces of LightSwitch product testing infrastructure and are kind of hard to share because of that. I can certainly describe what these apps/scenarios do if that would be helpful.

    Ultimately it is Microsoft job to make sure that LightSwitch performs well in common scenarios. This thread has been very helpful in the sense that it brought to our attention something that we have, frankly, missed in v1 release. Thank you, guys! I will let you guys know what I find.

    Karol

    Tuesday, September 27, 2011 9:26 PM
    Moderator
  • Kaspars, can you do a quick test for me? Take the screen that shows the problem and for the grid control on that screen disable Export to Excel feature. Then run the app again as a browser app and as a desktop app and let me know if you still see a big perf. difference between these two.

    Thanks! Karol

    • Marked as answer by Kaspars Ozols Friday, September 30, 2011 10:29 AM
    Tuesday, September 27, 2011 9:42 PM
    Moderator
  • I did the test you requested and it gives results as expected - form load time on my old PC is 10 sec, on the new PC - 5 seconds. Graphs in Resource monitor now look normal - firstly there is a peak at network activity, then memory & CPU usage go up for small amount of time.

    So, now we know that "Export to Excel" feature is causing slowdown. It would be great if that gets fixed soon as one of the main requirements for our application was ability to export grid contents to Excel. Actually, that is why we have chosen to use Desktop application.

    Althought things got better, still I would not call LightSwitch blazing fast. Even on my new PC what is better-than-average performance is just acceptable.  Customers that want to save on software won't spend a lot on hardware too, so LS should work smoothly even on mediocre machines. I hope that LS team will find a way to optimize everything. I hope to see performance increase in UI in the next version of LightSwitch.

     

     

    Wednesday, September 28, 2011 7:21 AM
  • Please, is the summary of this post that "Export to Excel" in V1 will slow down:

    1) DataGrids with paging on

    2) DataGrids with paging off

    3) Both

     

    Saturday, October 01, 2011 10:59 PM
  • To Garth

    It seems that this slowdown ir proportional to record count. So, it does not matter if you use paging or not (3rd option in your list). Function "Export to Excel" actually works only on records that are on the current page (imagine what happens if I want to export 1000 records with page size set to 45!!! This implementation does not seem to be right, but this is a sepparate issue). So, if page size is set to large number, you can see the slowdown. In small page size scenario slowdown is barely noticable.

    To Karol

    Dropbox sent me an email saying that links to my trace files have been blocked due to high traffic. Did you manage to get those files? Or you don't need them anymore? I can put them in some other place if you need them...

    Also, any response about when this issue will be fixed? Are there any other speed optimizations planned is LS V2 (for example, GPU based rendering, transfer compression, etc)?


    Monday, October 03, 2011 11:10 AM
  • Kaspars, I do not need the trace files anymore, you can delete them from DropBox. Thanks again for posting them.

    As you say, the slowdown caused by Export to Excel is not affected by paging. It should be directly proportional to the number of rows visible in the grid and affects loading data primarily. There is no easy workaround  (at least I cannot think of any) other than re-implementing exporting to Excel yourself, using OLE Automation support built into Silverlight. The current plan of record is to fix this problem in LS v.next.

    This thread has been helpful in highlighting the need to look again at performance of data loading and working with data. We are investigating ways of making scrolling faster. As to other performance optimizations (if/what/when), I cannot give you any details yet, sorry.

    Monday, October 03, 2011 4:38 PM
    Moderator
  • Ok, thank you for your reply! I hope you will find a way to increase the performance because otherwise it is hard to explain to outer customer why a new technology is so slow. :)

    Re-implementing exporting does not sound as easy solution. Only reasonable workaround I can think of is to enable feature "Export to Excel" just for some specific user group as this feature is required only for advanced users. I haven't tried to implement it yet, but it should be piece of cake, shouldn't it?

     

    Monday, October 03, 2011 4:48 PM
  • Hi Kaspars,

    Take a look at the Office Integration Pack, it's a free extension from Grid Logic that will support exporting to Excel plus a whole lot more. I wrote about it here:

    http://blogs.msdn.com/b/bethmassi/archive/2011/09/22/fun-with-the-office-integration-pack-extension-for-lightswitch.aspx

    Cheers,

    -Beth

     


    Senior Program Manager, Visual Studio Community http://msdn.com/lightswitch http://msdn.com/vbasic http://msdn.com/vsto http://www.bethmassi.com
    Friday, October 07, 2011 8:54 PM
    Owner
  • Hi Henderson,

    Am trying to integrated OfficeIntegration in my appilcation but am get errors how to do it plz tell me.

    Wednesday, December 14, 2011 12:39 PM