locked
How to use semantic zoom with a GridView that displays large amounts of data (ie. 5000 grouped records) when grouping doesn't support virtualization RRS feed

  • Question

  • This is a new post in response to a new issue that has come up from my post here: http://social.msdn.microsoft.com/Forums/en-US/winappswithcsharp/thread/3d2eb525-6d4c-4445-bc84-e1484369a4a9

    What I have gathered so far:

    - In order to load large amount of records into the GridView you need to use a virtualized panel like the WrapGrid or the VirtualizedStackPanel.
    - In order to properly use semantic zoom your data source must be grouped and you must link the two "views" by using a CollectionViewSource.
    - The main reason to use semantic zoom is when you have a large amount of data to display and you would like to provide a grouped "zoom out" view.
    - The GridView is not capable of supporting Virtualization when using the GroupStyle

    Hence, the main reason I need semantic zoom is because I have a large amount of data and semantic zoom requires grouping, however I am unable to load a large amount of data into my GridView because it will not support virtualization when using grouping. It does not seem like the use case here has been very though out. Considering I am preparing a project for official Microsoft use to be due by next week, I would greatly appreciate any help in what direction I can take here as this appears to be a massive limitation and roadblock to my design.


    • Edited by bccollin Thursday, July 5, 2012 7:49 PM
    Thursday, July 5, 2012 7:49 PM

Answers

  • Thanks for your replies, I ended up solving my issue another way. Basically I de-linked the two views from the CollectionViewSource and set them to their own datasource - Groups, and Items. I then display the items in the GridView with grouping disabled but I instead fake grouping by including the "groups" as items inside of the list and using a template selector to generate the group headers differently than the real items. Then when clicking on an item in the zoomed out semantic zoom I use the Visual Tree Helper to gain access to the ScrollViewer inside of my GridView and manipulate it to scroll to the "group" that was clicked.

    The only issue is there is a slight pause on the UI when doing this if I click a group that is off of the screen as I assume the virtualization engine is trying to redraw where I moved to the scrollbar too. I think this will have to be good enough for my purposes now and actually works pretty well. Thanks for all your help!

    • Marked as answer by bccollin Tuesday, July 10, 2012 1:37 AM
    Sunday, July 8, 2012 11:36 PM

All replies

  • Hi,

    Your explanation unfortunately demonstrates that coupling the "grouping concept" with the "semantic zoom concept" is only adequate for reasonable amounts of data. Both mechanisms have conflicting requirements that are magnified when important volumes of data are in play.

    Long-term, the solution is - I believe - to design an architecture that more loosely couples these 2 mechanisms, thereby granting the possibility to cope with large amounts of data when required. Unfortunately, this seems presently not to come out-of-the-box. I believe you will have to code your way around this: it is not impossible but does the result justify the effort ?

    Hope someone can provide you with more positive feedback.  


    • Edited by ForInfo Thursday, July 5, 2012 9:28 PM
    Thursday, July 5, 2012 9:13 PM
  • Thanks ForInfo,

    I was under the assumption that the entire point of semantic zoom was the ability to easily sort through large amounts of data. What is the purpose of this feature otherwise? I haven't had much time to play with WinJS but are you aware if it would have this same issue?

    Thursday, July 5, 2012 9:27 PM
  • Well,

    To put it in slightly less diplomatic terms, one should not perform grouping of large amounts of data (the grouping concept) with such a tight coupling into the presentation layer (the semantic zoom concept).

    This has nothing to do with the merits of the Windows8/WinRT framework but is a general given in the data processing world: why would you transport over the internet millions (say) of detail items just to aggregate them front-end into thousands of group items (ZoomOut) and display them aside a dozen of detail items (ZoomIn) ? 

    So, based on this reasoning, your work-around would be to loosen the dependence between a Collection of detailed items (that is necessarily virtualized) and a Collection of grouped items (that is optionally virtualized) and find out whether it is possible to use semantic zoom on these separate collections. I have not looked into the feasibility of actually doing this.

    But _assuming you could this_, you seem to have proven that the Collection of detailed items will not be virtualized in the context of a SemanticZoom.GridView : so there you would hit the roadblock again. A work-around could be forced around this obstacle ... but then again we would end up with an architecture that does not have its place in application code that should prove a point to the audience. 

    This is why I can only see a long-term improved solution ... unless someone else has a better idea.

      



    • Edited by ForInfo Thursday, July 5, 2012 9:54 PM
    Thursday, July 5, 2012 9:32 PM
  • Well another issue is, my data is provided by the user and is loaded in xml format. I then have many different methods that the user can "pivot" the data. Due to it's open ended nature the user could theoretically load things in such a way that there is a lot of data. I have no way of knowing how much data is too much data, until the grid view itself crashes and throws an unhandled exception. If the grid is just going to crash out of nowhere without me knowing what limit it can even handle, I have no way of protecting the user from trying to feed in too many data entries. I would have to set some arbitrary limit? The GridView can clearly support virtualizing, and can handle all these items, I really don't understand why it has to be disabled just because I want to do some grouping.
    Friday, July 6, 2012 1:08 AM
  • Generally speaking, I am afraid we are now trying to architect an application w/o too much relevance with the purpose of this forum. Your merit is to have to have shown that the tools at hand are probably not designed to cope with the situation at hand, i.e. large amounts of data. Hence it is necessary to address the problem with another design architecture in mind.

    I will look somewhat deeper into whether the GridView limitations - when used in the realm of SemanticZoom - can be worked around nevertheless, not knowing very much about your objectives. But I cannot promise the moon.

    Maybe someone else has the bright idea that is needed ...

    Friday, July 6, 2012 5:01 AM
  • Hereafter are my experimental findings. I managed to avoid the "exception case" (positive) but the UI experience is not optimal.

                //
                // ItemTemplate/ItemContainerStyle : small
                //
                //                                                          Speed  : ZoomIn state   Zoom In/out   ZoomOut state
                // Total Items Items/Category Categories Gesture: Hori Scroll                            Vertical Scroll
                // 10000        50                    200                        Very slow(*)     average         perfect
                // 10000        25                    400                        Little faster(*)  average         perfect
                //
                // 100000      50                   2000                       idem                average         perfect
                // 100000      25                   4000                       idem                average         perfect
                //
                // 1000000     50                  20000                     idem                average         perfect
                // 1000000     25                  40000                     idem                average         perfect
                //
                // (*) means : did not get an exception but usage is not appropriate for an end-user (practically freezing)
                //
                // These results were :
                // - marginally optimized when using some own-made panels (taking the existing constraints of non-virtualizing into account)
                // - however after using them, I eliminated the exception case (most unsure however what the reason for that is though)
                // - extremely dependent on the complexity of the ItemTemplate/ItemContainerStyle
                //

    The last point is quite important and would directly influence the complexity of the Detail Items you wish to display. All in all I believe it would quite a bit of work to reach an acceptable result at this stage (Preview) in the product cycle. At least from what I could find out as an outsider.



    • Edited by ForInfo Friday, July 6, 2012 10:25 AM
    Friday, July 6, 2012 7:53 AM
  • Thanks for your help and time ForInfo. I'd just like to point out that the Windows 8 "People" application seems very capable of displaying a large of amount of grouped items while supporting semantic zoom. Here is another case where the programmer couldn't know how many items a particular user may have in the list, yet the app seems to handle the load fine. I'd love to know how the coded that screen because it's exactly what I'm looking for.
    • Edited by bccollin Friday, July 6, 2012 2:42 PM
    Friday, July 6, 2012 2:39 PM
  • I had a similar problem, I used a slightly different approach. I had more than 100 groups of data, each group having up to 4000 records (These records within the group based on  priority).

    1. I started off with the out of box Grid template application.

    2. In the Zoomed In View, I displayed all the Groups and Group Items. But instead of displaying all the 4000 items of a group, I have only displayed the top 100 (Group Items - Each Item in the group has priority associated with it). So we have 100 groups with 100 items in each group. This itself is a lot of panning for the if the user wanted to go to 75 group.

    3. The Zoomed Out View would only display the Group Names (as expected). So user can automatically pan to the appropriate Group Data.

    4. In the Zoomed In View, if the user clicks on the Group Header, I am navigating to another page which has a grid view with Virtualized panel . This page displays all the 5000 items belonging to that group. I have also implemented "Incremental Loading" in this page to make loading of these 5000 items much smoother.

    Now I found a way of further sub - grouping these 5000 into smaller groups. I will follow the same steps to display these sub groupings as well.

    When I first started, I was trying to display 5000 items under the group. As mentioned by you performance is slow. However I went ahead and did some usability testing with my friends. I  showed them the view after the 5000 items were loaded - not while it is loading. They  weren't aware of the loading time, they only saw what was rendered. They felt that looking at 5000 items under each group wasn't such a good experience. Even though I am implemented Semantic Zoom, it would only help in going to the appropriate group, once you are that group, they still had to go through 5000 items. That's when I moved to above mentioned design of displaying only top 100 items in each group.

     

    Friday, July 6, 2012 7:36 PM
  • Point 2. looks as a very reasonable work-around, if it fits bccollin's scenario. The question remains though whether Windows 8 RTM/GA will lift the present limitation or not about virtualizing ... possibly making bccollin's work superfluous in a few weeks time
    Saturday, July 7, 2012 7:11 AM
  • Thanks for your replies, I ended up solving my issue another way. Basically I de-linked the two views from the CollectionViewSource and set them to their own datasource - Groups, and Items. I then display the items in the GridView with grouping disabled but I instead fake grouping by including the "groups" as items inside of the list and using a template selector to generate the group headers differently than the real items. Then when clicking on an item in the zoomed out semantic zoom I use the Visual Tree Helper to gain access to the ScrollViewer inside of my GridView and manipulate it to scroll to the "group" that was clicked.

    The only issue is there is a slight pause on the UI when doing this if I click a group that is off of the screen as I assume the virtualization engine is trying to redraw where I moved to the scrollbar too. I think this will have to be good enough for my purposes now and actually works pretty well. Thanks for all your help!

    • Marked as answer by bccollin Tuesday, July 10, 2012 1:37 AM
    Sunday, July 8, 2012 11:36 PM
  • Thanks for your replies, I ended up solving my issue another way. Basically I de-linked the two views from the CollectionViewSource and set them to their own datasource - Groups, and Items. I then display the items in the GridView with grouping disabled but I instead fake grouping by including the "groups" as items inside of the list and using a template selector to generate the group headers differently than the real items. Then when clicking on an item in the zoomed out semantic zoom I use the Visual Tree Helper to gain access to the ScrollViewer inside of my GridView and manipulate it to scroll to the "group" that was clicked.

    The only issue is there is a slight pause on the UI when doing this if I click a group that is off of the screen as I assume the virtualization engine is trying to redraw where I moved to the scrollbar too. I think this will have to be good enough for my purposes now and actually works pretty well. Thanks for all your help!

    Thanks for sharing the concept. I wonder, would it be possible for you to share the code for your implementation? I'm sure there's many battling out with the GridView performance. 

    Especially the part about using template selector to create the group headers. Are you able to create group headers which look identical to the default implementation?

    Edit: Here's a blog post which describes how to do this. The group headers don't look similar to the built-in ones but the performance is good.
    • Edited by MikaelKoskinen Friday, November 9, 2012 6:53 AM Added a link
    Monday, October 29, 2012 7:09 PM
  • To put it in slightly less diplomatic terms, one should not perform grouping of large amounts of data (the grouping concept) with such a tight coupling into the presentation layer (the semantic zoom concept).

    From Guidelines for Semantic Zoom

    Semantic Zoom is a touch-optimized technique used by Windows Store apps in Windows 8 for presenting and navigating large sets of related data or content within a single view (such as a photo album, app list, or address book).

    http://msdn.microsoft.com/en-us/library/windows/apps/hh465319.aspx

    To put it in slightly less diplomatic terms - your wrong. Thanks.


    Sunday, March 24, 2013 4:34 AM