none
Bing Maps Large data of pushpins RRS feed

  • Question

  • Hi,

    My Bing Map(using Clustering) is working fine with small number of Pushpins like 5000. But when I try to Push a large number of Pushpins like 100k. The Browser(Google Chrome) is taking a lot of to show the Map. The issue is with the pins.push using that in a for loop is hitting the performance a lot.

    Is there any other way to push the locations for a large data?

    Thanks

    function loadMapScenario(){
            if (dataSource.data().length != 0 )
            {
                GetDataSourceLocations();
    
                map = new Microsoft.Maps.Map(document.getElementById('Map'), {
                    credentials: '',
                    enableClickableLogo: false
                });
    
                infobox = new Microsoft.Maps.Infobox(map.getCenter(), { visible: false });
                infobox.setMap(map);
    
                Microsoft.Maps.loadModule('Microsoft.Maps.Clustering', function () {
                        clusterLayer = new Microsoft.Maps.ClusterLayer(pins, { 
                        gridSize: 100,
                        clusteredPinCallback: customizeClusteredPin,
                        
                    });
                    map.layers.insert(clusterLayer);
                });
                
                var bounds = Microsoft.Maps.LocationRect.fromLocations(locs);
                map.setView({bounds:bounds, padding: 100});
            }
        }
    
    function GetDataSourceLocations(){
            var grid = $("#SearchGrid").data("kendoGrid");
            var dataSource = grid.dataSource;
            pins = []; locs =[];
            for(var i =0; i < dataSource.data().length; i++){
                var longitude = dataSource.data()[i].des_Longitude;
                var latitude = dataSource.data()[i].des_Latitude;
                if(longitude != null && latitude != null){
                    pins.push(new Microsoft.Maps.Pushpin(new Microsoft.Maps.Location(latitude, longitude)));
                    pins[i].metadata = { ProductId : dataSource.data()[i].Id};
                    Microsoft.Maps.Events.addHandler(pins[i], 'click', pushpinClicked);
                    locs.push(new Microsoft.Maps.Location(latitude, longitude));
                }
                else{
                    pins.push(new Microsoft.Maps.Pushpin(new Microsoft.Maps.Location(0, 0)));
                    pins[i].metadata = { ProductId: dataSource.data()[i].Id};
                }
            }
            return;
        }

     
    Thursday, September 14, 2017 2:30 PM

Answers

  • A 100K pushpins will be slow, period. When you have that much data you should consider a server side solution for clustering. Server side clustering would be faster and allow you to support millions of pushpins. 

    That said, here are some things you can make your code a bit more efficient:

    • Add the events to the layer, not individual pushpins. This will use a lot less memory and allow events to work a lot faster. 
    • Put the map into lite mode to remove vector labels. This will reduce that calculations done when moving the map. To do this add liteMode:true to the map options when loading the map.
    • Consider moving to a custom rendering overlay like this: http://bingmapsv8samples.azurewebsites.net/#Canvas%20Layer

    [Blog] [twitter] [LinkedIn]

    Thursday, September 14, 2017 4:51 PM
  • There are a number of different ways to do this and it really depends on how your data is stored and how often it is updated.

    If the data is stored in a database and only updated a couple of times a day at most, then you could calculate clusters ahead of time and store the data in database table. You likely would only need to cluster zoom levels 1 through 12 and then could have your code switch to client side clustering or go to level 14 or 15 with server side clustering and then for lower zoom levels return all pushpins that are within the map view.

    If your data is stored in a database or any other file format or your data is going to be updated constantly then you might want to consider calculating the clusters periodically on the server and storing the results in memory. 

    For all solutions typically the only information stored for a cluster is its location, zoom level, and its size (# of pushpins it represents). Optionally you might want to store the bounding box of the data a cluster represents, this would allow you to add a click event to the cluster and zoom into it easily if you wanted to. If storing this data in a database table you will want to create an index based on the zoom level.

    Each time the map zoom level changes you will want to retrieve the clusters. If you have a lot of cluster data you may also want to do this when the map is panned and limit the clusters that are returned to those that would be in view. By doing this you would only be downloading maybe 20 clusters at most per map move which would be very small and fast to download and render.

    There are two common algorithms that are used for breaking data up into clusters; one is grid based and very fast, the other is point based and looks better but a little bit slower. Since you would be doing this server side, I recommend using a point based approach. This algorithm takes the first pushpin and finds all other pushpins that are within a certain pixel distance and creates a cluster out of it. The next pushpin that hasn't been clustered then does the same thing and so one with all pushpins until all pushpins have been clustered. I wrote a blog post about these algorithms years ago here: https://rbrundritt.wordpress.com/2012/01/04/client-side-point-based-clustering/

    As for code, its been a while since anyone has asked about server side clustering, and most of the samples I'm familiar with are 8+ years old and rendered on old map controls that don't exist anymore, although wiring up the client side of things is fairly easy, some simply are dead links now. I have both of these algorithms on their own available in JavaScript and C# form which would likely be a good starting point:

    http://mapstoolbox.codeplex.com/SourceControl/latest#Microsoft.Maps.Spatialtoolbox/Source/Microsoft.Maps.SpatialToolbox.Win8/Bing/Clustering/ClusteringLayer.cs

    http://bingmapsv7modules.codeplex.com/SourceControl/latest#BMv7Plugins/BMv7.PointBasedClustering/scripts/PointBasedClustering.js

    https://code.msdn.microsoft.com/windowsapps/Point-Based-Clustering-42c2e9c5

    Looking around, here are some useful resources that are a bit more specific to server side clustering:

    http://fatdevz.blogspot.com.au/2013/06/spatial-data-clustering-with-c.html

    https://github.com/zenobase/geocluster-facet

    http://rtsoftwaregroup.io/server-side-google-map-markers-clustering/

    https://github.com/kunukn/Google-Maps-Clustering-CSharp

    I've added a server side clustering example to my list of samples to do when I get a chance. Hopefully I'll be able to put one together sometime in the next few months. In the meantime, the above resources should give you a good starting point.


    [Blog] [twitter] [LinkedIn]

    Friday, September 15, 2017 4:23 PM

All replies

  • A 100K pushpins will be slow, period. When you have that much data you should consider a server side solution for clustering. Server side clustering would be faster and allow you to support millions of pushpins. 

    That said, here are some things you can make your code a bit more efficient:

    • Add the events to the layer, not individual pushpins. This will use a lot less memory and allow events to work a lot faster. 
    • Put the map into lite mode to remove vector labels. This will reduce that calculations done when moving the map. To do this add liteMode:true to the map options when loading the map.
    • Consider moving to a custom rendering overlay like this: http://bingmapsv8samples.azurewebsites.net/#Canvas%20Layer

    [Blog] [twitter] [LinkedIn]

    Thursday, September 14, 2017 4:51 PM
  • Thanks Ricky for the Response. I'd prefer to use Server side clustering by AJAX to get the Cluster layer.

    I'm unable to use Microsoft.Maps.Pushpin. Is there any sample code for server side clustering?

     

    Thursday, September 14, 2017 11:22 PM
  • There are a number of different ways to do this and it really depends on how your data is stored and how often it is updated.

    If the data is stored in a database and only updated a couple of times a day at most, then you could calculate clusters ahead of time and store the data in database table. You likely would only need to cluster zoom levels 1 through 12 and then could have your code switch to client side clustering or go to level 14 or 15 with server side clustering and then for lower zoom levels return all pushpins that are within the map view.

    If your data is stored in a database or any other file format or your data is going to be updated constantly then you might want to consider calculating the clusters periodically on the server and storing the results in memory. 

    For all solutions typically the only information stored for a cluster is its location, zoom level, and its size (# of pushpins it represents). Optionally you might want to store the bounding box of the data a cluster represents, this would allow you to add a click event to the cluster and zoom into it easily if you wanted to. If storing this data in a database table you will want to create an index based on the zoom level.

    Each time the map zoom level changes you will want to retrieve the clusters. If you have a lot of cluster data you may also want to do this when the map is panned and limit the clusters that are returned to those that would be in view. By doing this you would only be downloading maybe 20 clusters at most per map move which would be very small and fast to download and render.

    There are two common algorithms that are used for breaking data up into clusters; one is grid based and very fast, the other is point based and looks better but a little bit slower. Since you would be doing this server side, I recommend using a point based approach. This algorithm takes the first pushpin and finds all other pushpins that are within a certain pixel distance and creates a cluster out of it. The next pushpin that hasn't been clustered then does the same thing and so one with all pushpins until all pushpins have been clustered. I wrote a blog post about these algorithms years ago here: https://rbrundritt.wordpress.com/2012/01/04/client-side-point-based-clustering/

    As for code, its been a while since anyone has asked about server side clustering, and most of the samples I'm familiar with are 8+ years old and rendered on old map controls that don't exist anymore, although wiring up the client side of things is fairly easy, some simply are dead links now. I have both of these algorithms on their own available in JavaScript and C# form which would likely be a good starting point:

    http://mapstoolbox.codeplex.com/SourceControl/latest#Microsoft.Maps.Spatialtoolbox/Source/Microsoft.Maps.SpatialToolbox.Win8/Bing/Clustering/ClusteringLayer.cs

    http://bingmapsv7modules.codeplex.com/SourceControl/latest#BMv7Plugins/BMv7.PointBasedClustering/scripts/PointBasedClustering.js

    https://code.msdn.microsoft.com/windowsapps/Point-Based-Clustering-42c2e9c5

    Looking around, here are some useful resources that are a bit more specific to server side clustering:

    http://fatdevz.blogspot.com.au/2013/06/spatial-data-clustering-with-c.html

    https://github.com/zenobase/geocluster-facet

    http://rtsoftwaregroup.io/server-side-google-map-markers-clustering/

    https://github.com/kunukn/Google-Maps-Clustering-CSharp

    I've added a server side clustering example to my list of samples to do when I get a chance. Hopefully I'll be able to put one together sometime in the next few months. In the meantime, the above resources should give you a good starting point.


    [Blog] [twitter] [LinkedIn]

    Friday, September 15, 2017 4:23 PM
  • We successfully implemented clustering in the back end for our client specific requirements. Some of our clients have upwards of 50 layers (essentially backend tables). I think some of our larger tables are in the 10's of millions records. Client requirements pushed us to implement our own database backed dynamic tile cache where we are providing views into multi gigabytes of spatial data alone.

    Specific needs such as clustering being able to break into individual pushpins on a higher level than the default clustering but is dependent upon data density and configurable in the back end. We break it into individual pushpins somewhere close to 8000 or below because we found above that the performance begins to be degrade.

    Unfortunately I cannot share the code but it can be done.

    Saturday, September 23, 2017 1:18 PM