Answered by:
Database Read (Break up of Data)

Question
-
User-2017823710 posted
Hi,
I have the following components:
UI Page:
WebService:
SQLServerDB:
I am calling a web method in my web service which in turn calls a SP which return 30k rows. Unfortunately there is a 7MB limit on the data being passed from the WebService to the UI.
I would like to split the data 30k into chunks of 5k and send to UI. Is there a way to accomplish this? Any other ideas?
Thanks,
Sid
Monday, September 9, 2013 11:32 AM
Answers
-
User-821857111 posted
You can use Row_Number to page the data from the database and provide paging links in the UI. That way you will only retrieve 20 rows (for example) at a time. If the user really wants to read all 30,000 rows of data, they can go through the paging links.
- Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
Monday, September 9, 2013 2:42 PM -
User697462465 posted
Hi sid,
According to your description, my understanding is that you would like to page the data.
If so, please try to refer to the following code:
First of all the big magic here is the SQL Query which makes it easy to handle the amount of data to be fetch from database. I used the NorthWind database's Orders table which has over 800 records. Here Index and Size
are two integer variables which are comes from querystring .
- Index is the current page number
- Size is the no of records to be shown on a page
select * from (SELECT ROW_NUMBER() OVER (ORDER BY CustomerID asc) as row,* FROM Orders) tblTemp WHERE row between (" + index + " - 1) * " + size + " + 1 and " + index + "*" + size + " "; SQL += " select COUNT(*) from Orders
First time when the page loads it has no querystring
values so the default Index
is 1 and Size
is 10. You can latter change the page size, there are options for 10 records per page and 15 records per page. I tried to explain all necessary points within the code. <!--?xml:namespace prefix = "o" ns = "urn:schemas-microsoft-com:office:office" /--><o:p></o:p>
void BindGrid(int size, int index) { //page url--get from Web.config file--------change it as your page URL string url = ConfigurationManager.AppSettings["URL"].ToString(); //string templates for links string link = "<a href='" + url + "?Index=##Index##&Size=##Size##'><span class='page-numbers'>##Text##</span></a>"; string link_pre = "<a href='" + url + "?Index=##Index##&Size=##Size##'><span class='page-numbers prev'>##Text##</span></a>"; string link_next = "<a href='" + url + "?Index=##Index##&Size=##Size##'><span class='page-numbers next'>##Text##</span></a>"; try { //the connectionstring from Web.config file--------change it as per your database settings String ConnStr = ConfigurationManager.ConnectionStrings["myConnection"].ToString(); //the sql query with paging logics //here table name is "Orders" change this name as your requirement. //"CustomerID" is the column by which you can sort the records, change it as per your requirement String SQL = @"select * from (SELECT ROW_NUMBER() OVER (ORDER BY CustomerID asc) as row,* FROM Orders) tblTemp WHERE row between (" + index + " - 1) * " + size + " + 1 and " + index + "*" + size + " "; SQL += " select COUNT(*) from Orders"; //fetching data from database suing SqlDataAdapter Fill method to bind the Gridview SqlDataAdapter ad = new SqlDataAdapter(SQL, ConnStr); DataSet ds = new DataSet(); ad.Fill(ds); //bind the grid with the ist data table---------remember that this dataset consist of two data tables this.gvPaging.DataSource = ds.Tables[0]; this.gvPaging.DataBind(); ////////get the n number of record/////////// Double n = Convert.ToDouble(Convert.ToInt32(ds.Tables[1].Rows[0][0]) / size); /////////setting page numbers with links if (index != 1) lblpre.Text = link_pre.Replace("##Size##", size.ToString()).Replace( "##Index##", (index - 1).ToString()).Replace("##Text##", "prev"); else lblpre.Text = "<span class='page-numbers prev'>prev</span>"; if (index != Convert.ToInt32(n)) lblnext.Text = link_next.Replace("##Size##", size.ToString()).Replace( "##Index##", (index + 1).ToString()).Replace("##Text##", "next"); else lblnext.Text = "<span class='page-numbers next'>next</span>"; //generate dynamic paging int start; if (index <= 5) start = 1; else start = index - 4; for (int i = start; i < start + 7; i++) { if (i > n) continue; //create dynamic HyperLinks HyperLink lnk = new HyperLink(); lnk.ID = "lnk_" + i.ToString(); if (i == index)//current page { lnk.CssClass = "page-numbers current"; lnk.Text = i.ToString(); } else { lnk.Text = i.ToString(); lnk.CssClass = "page-numbers"; lnk.NavigateUrl = url + "?Index=" + i + "&Size=" + size + ""; } //add links to page this.pl.Controls.Add(lnk); } //------------------------------------------------------------------ //set up the ist page and the last page if (n > 7) { if (index <= Convert.ToInt32(n / 2)) { lblLast.Visible = true; lblIst.Visible = false; lblLast.Text = link.Replace("##Index##", n.ToString()).Replace( "##Size##", size.ToString()).Replace("##Text##", n.ToString()); spDot2.Visible = true; spDot1.Visible = false; } else { lblLast.Visible = false; lblIst.Visible = true; lblIst.Text = link.Replace("##Index##", (n - n + 1).ToString()).Replace( "##Size##", size.ToString()).Replace("##Text##", (n - n + 1).ToString()); spDot2.Visible = false; spDot1.Visible = true; } } } catch (Exception ee) { //catch the exception } }
In the above method you have to change the Config settings like the Connectionstring
and page url which can be edited in the web.config file.<!--?xml:namespace prefix = "o" ns = "urn:schemas-microsoft-com:office:office" /--><o:p></o:p>
On the aspx page there are six labels:<o:p></o:p>
- previous page link
- ist page link <o:p></o:p>
- dot before ist page link <o:p></o:p>
- next page link<o:p></o:p>
- last page link <o:p></o:p>
- dot before ist page link<o:p></o:p>
and a PlaceHolder
for other page numbers <o:p></o:p>
<div class="pager fl"> <asp:Label runat="server" ID="lblpre"></asp:Label> <asp:Label runat="server" ID="lblIst" Visible="false"></asp:Label> <asp:Label runat="server" ID="spDot1" CssClass="page-numbers prev" Visible="false" Text="..."></asp:Label> <asp:PlaceHolder ID="pl" runat="server"></asp:PlaceHolder> <asp:Label runat="server" ID="spDot2" Visible="false" CssClass="page-numbers prev" Text="..."></asp:Label> <asp:Label runat="server" ID="lblLast" Visible="false"></asp:Label> <asp:Label runat="server" ID="lblnext"></asp:Label> </div> <div class="pager f2"> <a id="A1" href="javascript:void(0);" runat="server" class="page-numbers">10</a> <a href="javascript:void(0);" id="A2" runat="server" class="page-numbers">15</a><span class="page-numbers desc">per page</span> </div>
HyperLinks
are dynamically created and set their Links and css and add in the placeholder at page load. <!--?xml:namespace prefix = "o" ns = "urn:schemas-microsoft-com:office:office" /--><o:p></o:p>
Another important point is to replace the Table name in the sql Query and the column name in the order by statement. <o:p></o:p>
To change the page size there are two options(html anchor tags) on the right side of the page. You can set 10 records per page or 15 records per page. Using JQuery.
I capture the click events of these two html anchors and set the currently set page size in a HiddenField
and reload the current page by passing Index and Size in querystring.<o:p></o:p>
jquery code:<o:p></o:p>
<input type="hidden" runat="server" id="hdSize" /> <script type="text/javascript"> $(document).ready(function () { $('#A1').click(function () { $('#hdSize').val('10'); Loadpage(); }); $('#A2').click(function () { $('#hdSize').val('15'); Loadpage(); }); }); function Loadpage() { try { //reload page using new page size var url = $(location).attr('href'); var index = "Index=1"; if (url.indexOf('Index') > 0) { index = url.split('?')[1]; index = index.split('&')[0]; } url = url.split('?')[0] + "?" + index + "&Size=" + $('#hdSize').val() + ""; window.location.href = url; } catch (e) { alert(e); } } </script>
CSS: CSS plays an important role in this paging. Although all works done but these all looks very poor untill a good css effect is given. <!--?xml:namespace prefix = "o" ns = "urn:schemas-microsoft-com:office:office" /--><o:p></o:p>
td { color: #55ff00; } .text { color: #74aacc; } .fl { float: left; } .f2 { float: right; padding-right: 200px; } .pager { margin-top: 20px; margin-bottom: 20px; } .page-numbers { font-family: 'Helvetica Neue' ,Helvetica,Arial,sans-serif; border: 1px solid #CCC; color: #808185; display: block; float: left; font-size: 130%; margin-right: 3px; padding: 4px 4px 3px; text-decoration: none; } .page-numbers.current { background-color: #808185; border: 1px solid #808185; color: white; font-weight: bold; } .page-numbers.next, .page-numbers.prev { border: 1px solid white; } .page-numbers.desc { border: none; margin-bottom: 10px; }
Hope it can help you.
Best Regards,
Terry Guo
- Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
Friday, September 13, 2013 9:25 AM
All replies
-
User-821857111 posted
You can use Row_Number to page the data from the database and provide paging links in the UI. That way you will only retrieve 20 rows (for example) at a time. If the user really wants to read all 30,000 rows of data, they can go through the paging links.
- Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
Monday, September 9, 2013 2:42 PM -
User-2017823710 posted
Hi,
Thanks for the reply. Yes I incorporated the Row_Count logic in my SP. I was wondering if there was any example for the grid to incorporate the paging?
Thanks,
Sid
Tuesday, September 10, 2013 9:53 AM -
User-821857111 posted
Put this in Google (or Bing) 'gridview efficient paging row_number'
Tuesday, September 10, 2013 3:14 PM -
User697462465 posted
Hi sid,
According to your description, my understanding is that you would like to page the data.
If so, please try to refer to the following code:
First of all the big magic here is the SQL Query which makes it easy to handle the amount of data to be fetch from database. I used the NorthWind database's Orders table which has over 800 records. Here Index and Size
are two integer variables which are comes from querystring .
- Index is the current page number
- Size is the no of records to be shown on a page
select * from (SELECT ROW_NUMBER() OVER (ORDER BY CustomerID asc) as row,* FROM Orders) tblTemp WHERE row between (" + index + " - 1) * " + size + " + 1 and " + index + "*" + size + " "; SQL += " select COUNT(*) from Orders
First time when the page loads it has no querystring
values so the default Index
is 1 and Size
is 10. You can latter change the page size, there are options for 10 records per page and 15 records per page. I tried to explain all necessary points within the code. <!--?xml:namespace prefix = "o" ns = "urn:schemas-microsoft-com:office:office" /--><o:p></o:p>
void BindGrid(int size, int index) { //page url--get from Web.config file--------change it as your page URL string url = ConfigurationManager.AppSettings["URL"].ToString(); //string templates for links string link = "<a href='" + url + "?Index=##Index##&Size=##Size##'><span class='page-numbers'>##Text##</span></a>"; string link_pre = "<a href='" + url + "?Index=##Index##&Size=##Size##'><span class='page-numbers prev'>##Text##</span></a>"; string link_next = "<a href='" + url + "?Index=##Index##&Size=##Size##'><span class='page-numbers next'>##Text##</span></a>"; try { //the connectionstring from Web.config file--------change it as per your database settings String ConnStr = ConfigurationManager.ConnectionStrings["myConnection"].ToString(); //the sql query with paging logics //here table name is "Orders" change this name as your requirement. //"CustomerID" is the column by which you can sort the records, change it as per your requirement String SQL = @"select * from (SELECT ROW_NUMBER() OVER (ORDER BY CustomerID asc) as row,* FROM Orders) tblTemp WHERE row between (" + index + " - 1) * " + size + " + 1 and " + index + "*" + size + " "; SQL += " select COUNT(*) from Orders"; //fetching data from database suing SqlDataAdapter Fill method to bind the Gridview SqlDataAdapter ad = new SqlDataAdapter(SQL, ConnStr); DataSet ds = new DataSet(); ad.Fill(ds); //bind the grid with the ist data table---------remember that this dataset consist of two data tables this.gvPaging.DataSource = ds.Tables[0]; this.gvPaging.DataBind(); ////////get the n number of record/////////// Double n = Convert.ToDouble(Convert.ToInt32(ds.Tables[1].Rows[0][0]) / size); /////////setting page numbers with links if (index != 1) lblpre.Text = link_pre.Replace("##Size##", size.ToString()).Replace( "##Index##", (index - 1).ToString()).Replace("##Text##", "prev"); else lblpre.Text = "<span class='page-numbers prev'>prev</span>"; if (index != Convert.ToInt32(n)) lblnext.Text = link_next.Replace("##Size##", size.ToString()).Replace( "##Index##", (index + 1).ToString()).Replace("##Text##", "next"); else lblnext.Text = "<span class='page-numbers next'>next</span>"; //generate dynamic paging int start; if (index <= 5) start = 1; else start = index - 4; for (int i = start; i < start + 7; i++) { if (i > n) continue; //create dynamic HyperLinks HyperLink lnk = new HyperLink(); lnk.ID = "lnk_" + i.ToString(); if (i == index)//current page { lnk.CssClass = "page-numbers current"; lnk.Text = i.ToString(); } else { lnk.Text = i.ToString(); lnk.CssClass = "page-numbers"; lnk.NavigateUrl = url + "?Index=" + i + "&Size=" + size + ""; } //add links to page this.pl.Controls.Add(lnk); } //------------------------------------------------------------------ //set up the ist page and the last page if (n > 7) { if (index <= Convert.ToInt32(n / 2)) { lblLast.Visible = true; lblIst.Visible = false; lblLast.Text = link.Replace("##Index##", n.ToString()).Replace( "##Size##", size.ToString()).Replace("##Text##", n.ToString()); spDot2.Visible = true; spDot1.Visible = false; } else { lblLast.Visible = false; lblIst.Visible = true; lblIst.Text = link.Replace("##Index##", (n - n + 1).ToString()).Replace( "##Size##", size.ToString()).Replace("##Text##", (n - n + 1).ToString()); spDot2.Visible = false; spDot1.Visible = true; } } } catch (Exception ee) { //catch the exception } }
In the above method you have to change the Config settings like the Connectionstring
and page url which can be edited in the web.config file.<!--?xml:namespace prefix = "o" ns = "urn:schemas-microsoft-com:office:office" /--><o:p></o:p>
On the aspx page there are six labels:<o:p></o:p>
- previous page link
- ist page link <o:p></o:p>
- dot before ist page link <o:p></o:p>
- next page link<o:p></o:p>
- last page link <o:p></o:p>
- dot before ist page link<o:p></o:p>
and a PlaceHolder
for other page numbers <o:p></o:p>
<div class="pager fl"> <asp:Label runat="server" ID="lblpre"></asp:Label> <asp:Label runat="server" ID="lblIst" Visible="false"></asp:Label> <asp:Label runat="server" ID="spDot1" CssClass="page-numbers prev" Visible="false" Text="..."></asp:Label> <asp:PlaceHolder ID="pl" runat="server"></asp:PlaceHolder> <asp:Label runat="server" ID="spDot2" Visible="false" CssClass="page-numbers prev" Text="..."></asp:Label> <asp:Label runat="server" ID="lblLast" Visible="false"></asp:Label> <asp:Label runat="server" ID="lblnext"></asp:Label> </div> <div class="pager f2"> <a id="A1" href="javascript:void(0);" runat="server" class="page-numbers">10</a> <a href="javascript:void(0);" id="A2" runat="server" class="page-numbers">15</a><span class="page-numbers desc">per page</span> </div>
HyperLinks
are dynamically created and set their Links and css and add in the placeholder at page load. <!--?xml:namespace prefix = "o" ns = "urn:schemas-microsoft-com:office:office" /--><o:p></o:p>
Another important point is to replace the Table name in the sql Query and the column name in the order by statement. <o:p></o:p>
To change the page size there are two options(html anchor tags) on the right side of the page. You can set 10 records per page or 15 records per page. Using JQuery.
I capture the click events of these two html anchors and set the currently set page size in a HiddenField
and reload the current page by passing Index and Size in querystring.<o:p></o:p>
jquery code:<o:p></o:p>
<input type="hidden" runat="server" id="hdSize" /> <script type="text/javascript"> $(document).ready(function () { $('#A1').click(function () { $('#hdSize').val('10'); Loadpage(); }); $('#A2').click(function () { $('#hdSize').val('15'); Loadpage(); }); }); function Loadpage() { try { //reload page using new page size var url = $(location).attr('href'); var index = "Index=1"; if (url.indexOf('Index') > 0) { index = url.split('?')[1]; index = index.split('&')[0]; } url = url.split('?')[0] + "?" + index + "&Size=" + $('#hdSize').val() + ""; window.location.href = url; } catch (e) { alert(e); } } </script>
CSS: CSS plays an important role in this paging. Although all works done but these all looks very poor untill a good css effect is given. <!--?xml:namespace prefix = "o" ns = "urn:schemas-microsoft-com:office:office" /--><o:p></o:p>
td { color: #55ff00; } .text { color: #74aacc; } .fl { float: left; } .f2 { float: right; padding-right: 200px; } .pager { margin-top: 20px; margin-bottom: 20px; } .page-numbers { font-family: 'Helvetica Neue' ,Helvetica,Arial,sans-serif; border: 1px solid #CCC; color: #808185; display: block; float: left; font-size: 130%; margin-right: 3px; padding: 4px 4px 3px; text-decoration: none; } .page-numbers.current { background-color: #808185; border: 1px solid #808185; color: white; font-weight: bold; } .page-numbers.next, .page-numbers.prev { border: 1px solid white; } .page-numbers.desc { border: none; margin-bottom: 10px; }
Hope it can help you.
Best Regards,
Terry Guo
- Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
Friday, September 13, 2013 9:25 AM -
User-2017823710 posted
Thanks for your help Terry. I have incorporated using the same. I appreciate your help.
Regards,
Siddharth
Monday, September 16, 2013 12:10 PM -
User-2017823710 posted
Thanks Mike for your help.
Regards,
Sid
Monday, September 16, 2013 12:11 PM