none
PivotViewer memory issue - when multiple Items have the same image

    Question

  • Hi

    I've written some large data sets (10000+) using the excellent pauthorlib from codeplex.

    When I view these datasets it seems that they work quite well except if multiple Item's share the same Img.

    If I assign default images to multiple Items then what seems to happen is that the thumbnails for these shared items is very slow to load and the browser memory skyrockets until the browser crashes

    • e.g. for a 16000 set with all items with their own image then memory use is approx 200MB
    • for a 16000 set with 75% of items have shared images then memory use rockets up to 1.5GB before BOOM.

    Has anyone else seen this behaviour? Other than creating unique images for every single item, is there any fix/workaround available?

    I'll try to post some links soon - but need to get permission to share the data first...

    Stuart

    P.S Yes, I know the official recommendation is 3000 max for collection sizes :) 

    Sunday, November 21, 2010 3:47 AM

All replies

  • Hi Stuart,

    How long it takes you to load the 16000 set in the silverlight browser control?

     

    - Kiri kamal

     

    Monday, November 22, 2010 1:13 AM
  • Glad to know that there are like-minded people trying to push the envelope... Cool

    Have you tried viewing your large collection using the [unsupported] standalone Pivot Viewer client?  Does it load up nice and smooth?

    I'm not able to confirm if this is the case, but it does appear to me that there might be some bug or issue in the Silverlight PivotViewer control whereby it does not reuse texture images that are repeated, and instead create new textures eating up all the memory.

    No such problem with the standalone Pivot Viewer client though.

    Will be great if anyone from the dev team could confirm this memory issue.


    Monday, November 22, 2010 1:42 AM
  • How long it takes you to load the 16000 set in the silverlight browser control?

    As long as your browser's PC has the RAM available, then the performance is good.

    The main delay is downloading the index file - it's quite large - ~20MB

    In total it takes about 50 seconds from typing in the URL to playing with the data.

    I did also try a 75000 record set - the delay was not too bad, but my available RAM disappeared and a crash followed.

    Monday, November 22, 2010 2:22 AM
  • Thanks jarrodwee

    Have you tried viewing your large collection using the [unsupported] standalone Pivot Viewer client?  Does it load up nice and smooth?

    Where can I get this client - every URL I've tried so far just leads me to one of the cover pages for "live labs is dead - long live bing labs"

    Monday, November 22, 2010 2:24 AM
  • Thanks - that link worked :)

    And yes - in that viewer I can open all 73000 objects in my pivot set without an initial crash - indeed total memory use stays at around 1GB.

    The pivot UI animation isn't at it's most smooth with that many objects - but the functionality works!

    Monday, November 22, 2010 3:16 AM
  • Thanks for the confirmation... Smile

    Now, how do we go about getting someone from the Silverlight PivotViewer dev team to look into this issue?

    To recap, the large collection with repeated image references works fine when using the standalone client.

    However, when viewing the same collection using the Silverlight PivotViewer, it chokes and croaks.


    Monday, November 22, 2010 3:30 AM
  • To recap:

    • the bug is for collections where <Item>s share img references.
    • if I load up one of these collections in the standalone viewer these work fine - the load is smooth and the memory use is expected.
    • if I load up the same collection in the silverlight pivotviewer then the items with shared images take a long time to display (they pop into the display one by one) and the memory use is higher - it increases with each  image loaded.

    You can just about see this effect if you watch the swimmer icons when you  first load up:

    • using standalone pivotviewer on - http://zoom.runsaturday.com/out.cxml
    • silverlight control in browser - http://zoom.runsaturday.com/

    The effect is really really noticable when you get to larger data sets (if ms techs contact me I'm sure I can get them a URL for testing - just can't post this data into the public domain!) 

    Monday, November 22, 2010 3:42 AM
  • I also experienced the exact same problem when I was building a prototype this weekend. Thank you for starting this thread and I am really looking forward on how to solve this issue.

    Wednesday, November 24, 2010 5:38 AM
  • Thanks Stuart,

    I've +1 vote and also +1 reproducible.

    Btw, check out http://twitter.com/#!/jenndox

    Looks like MS is looking to hire a QA Developer for Silverlight PivotViewer.


    Wednesday, December 01, 2010 11:09 PM
  • Not sure this will work in all situation, but I had this issue when I using just one image reference with the 1000 items, Pivot get issue when it sees same unique id in the CXML file
    i.e in Img attribue, so all Item elements have Img="#1" or most items have then it giving this issue

    <Items ImgBase="output/collection.dzc">
        <Item Id="1" Img="#1"


    So What I did was to use unique Img reference but to change the ImgBase to point to single or same image xml as below

    in my collection.dzc

    <Items>
        <I Id="1" N="1" Source="3130.xml">
         <Size Width="80" Height="60" />
        </I>
        <I Id="2" N="2" Source="3130.xml">
         <Size Width="80" Height="60" />
        </I>
        <I Id="3" N="3" Source="3130.xml">
         <Size Width="80" Height="60" />
        </I>


    All Id refer Source as "3130.xml" so what we have to do is do some intelligent generation of dzc file where manipulate or add new entries to dzc file for all Id but most of them pointing to same Source value

    Then in my case it loads within 2 seconds all the tiles, not sure this will help all in all scenarios, but hope this will give some workaround until this fixed in the control itself

    I already vote the microsoft support thread you opened and put the workaround as well.

     

     

    Tuesday, January 04, 2011 9:18 PM
  • I started doing the same as above recently and it looks to be fine. Some of our collections have a default tile that many items link to using the main DZC XML.

    Wednesday, January 05, 2011 5:20 AM
  • Hello Stuart, 

     Could you please tell me how did you solve this issue ?  I browse your  URL  ( http://zoom.runsaturday.com/) and i see  its working fine .  It ia faster .

     

    Regards,

    Phani

    Monday, March 26, 2012 1:40 AM
  • It is just as Rasika said above. The images ids must be unique in the cxml but in the dzc you can point them at a common dzi file.

    Monday, March 26, 2012 3:36 AM
  • Hello Expert

     I have done the same having different image ID but  mapping the  different ID tos the same  dzi image in  but it did not work out

    Any other suggestions 

    Sunday, May 27, 2012 6:37 AM
  • I solved the problem by modifying the class ParallelDeepZoomCreator of PauthorLib :

     public int Submit(PivotImage sourceImage)
            {
                String dziTargetPath = Path.GetFileNameWithoutExtension(sourceImage.SourcePath);
                dziTargetPath = dziTargetPath.Replace(' ', '_');
                dziTargetPath = dziTargetPath.Replace('{', '_');
                dziTargetPath = dziTargetPath.Replace('}', '_');
                dziTargetPath = dziTargetPath.Replace('[', '_');
                dziTargetPath = dziTargetPath.Replace(']', '_');
                dziTargetPath = String.Format(DziFileNameTemplate, dziTargetPath);
                dziTargetPath = Path.Combine(m_outputDirectory, dziTargetPath);
    
                int index = m_dziPaths.IndexOf(dziTargetPath);
                //if (index != -1) return index;
    
                m_dziPaths.Add(dziTargetPath);
    
                if (File.Exists(dziTargetPath) == false && index == -1)
                {
                    lock (m_workQueue)
                    {
                        lock (m_itemsRemaining)
                        {
                            m_workQueue.Enqueue(new Object[] { new PivotImage(sourceImage.SourcePath), dziTargetPath });
                            m_itemsRemaining[0]++;
                        }
                        Monitor.Pulse(m_workQueue);
                    }
                }
    
                return m_dziPaths.Count - 1;
            }

    It seems to work like a charm =)

    Wednesday, July 04, 2012 5:05 AM