Answered by:
What is the best efficient way to create optimized query with lots of fields in this scenario

Question
-
Hi,
I have about 10 tables in my Database. One of the table has about 70 fields in it and rest has about 10 to 30 fields.
Now, I am writing a query to get all the fields, also with so many different calculations depending upon different fields, so basically while returning from that Query, I have about 160 fields. I need to return everything from my Stored Procedure itself as I have lots of calculations as per my system's functionality and I want to have all these calculations returned from my stored procedure itself so that I can use the same stored procedure all over my application and that's what is working perfect except just one problem.
Now, the problem is that with about 2000 rows in my Table1, this stored procedure is taking about 2-3 seconds to execute. So I am looking for some efficient way to execute my stored procedure with this query.
In my stored procedure, first I am executing a big query with about 150 fields and inserting into a temptable. Then again running a query with calculation fields from that temptable and putting it into another temptable. And finally executing query from that second temptable. The reason I am doing this way is because of my calculations.
So what I am looking for is, I will need to return all the columns with calculations as it is but I am looking for the best efficient way to write query inside this stored procedure. At present, as you see, first I am calcuating something and putting everything in a temp table. Than again depending upon the values of these temp table, I am calculating and putting into another temptable and finally I am quering that second temptable with filter criteria.
At present, with about 2000 rows, this query is taking about 2-3 seconds to execute which will be problem once my records reach around 10000 rows so I want to optimize this stored procedure.
Will appreciate if someone can help me to find out the best approach to optimize this stored procedure.
Herewith I am providing my Stored procedure Stored Procedure Script : ( modified to fit here, added Table1.*, instead of all the fields one by one just to paste here, actually I have all the fields there ):
DECLARE @FilterExpression VARCHAR(MAX) BEGIN IF OBJECT_ID('tempdb..#tempTable1') IS NOT NULL BEGIN DROP TABLE #tempTable1; END IF OBJECT_ID('tempdb..#tempTableForCalculation') IS NOT NULL BEGIN DROP TABLE #tempTableForCalculation; END SELECT CASE WHEN ( ISNULL (Table3.Under5000Email, '' ) <> '' ) AND ( ISNULL ( Table2.Under5000ApprovedORRejected, 0 ) <> @ApprovedStatusID ) THEN Table3.Under5000Email WHEN ( ISNULL (Table3.Under5000Email, '' ) <> '' ) AND ( ISNULL ( Table2.Under5000ApprovedORRejected, 0 ) <> @ApprovedStatusID ) AND ( ISNULL ( Table2.Under5000AssignedForApproval , 0 ) = 1 ) THEN Table3.Under5000Email ELSE '' END AS NextApprovalRequiredFromEmailAddress , Table1.* , Table2.Under5000ApprovedORRejected, Table2.Over5000ApprovedORRejected, Table2.FinanceApprovedORRejected , Table2.Over25000CFOApprovedORRejected, Table2.Over25000COOApprovedORRejected, Table2.Over25000CEOApprovedORRejected, CASE WHEN ( ISNULL ( Table2.Over25000CEOApprovedORRejected , 0 ) = @ApprovedStatusID ) THEN 'Approved' WHEN ( ISNULL ( Table2.Over25000CEOApprovedORRejected , 0 ) = @RejectedStatusID ) THEN 'Rejected' ELSE '' END AS Over25000CEOApprovedORRejected_Text, Under5000User.UserDisplayName AS Under5000_ApprovedORRejectedBy_UserDisplayName , Over5000User.UserDisplayName AS Over5000_ApprovedORRejectedBy_UserDisplayName, FinanceUser.UserDisplayName AS Finance_ApprovedORRejectedBy_UserDisplayName, Over25000CFOUser.UserDisplayName AS Over25000CFO_ApprovedORRejectedBy_UserDisplayName, Over25000COOUser.UserDisplayName AS Over25000COO_ApprovedORRejectedBy_UserDisplayName, Over25000CEOUser.UserDisplayName AS Over25000CEO_ApprovedORRejectedBy_UserDisplayName , CASE WHEN ( ( ISNULL ( Table1.Completed , 0 ) = 1 OR ISNULL ( Table1.Cancelled , 0 ) = 1 ) AND ISNULL ( Table2.Over25000CFOApprovedORRejected , 0 ) > 0 AND Over25000CFOUser.EmailAddress <> Table3_History_For_RequestInfo.Over25000CFOEmail AND ISNULL ( Over25000CFOUser.EmailAddress, '' ) <> '' AND ISNULL ( Table3_History_For_RequestInfo.Over25000CFOEmail, '' ) = '' ) THEN Over25000CFOUser.UserDisplayName ELSE '' END AS Over25000CFO_ApprovedORRejectedBy_UserDisplayName_Calculated , CASE WHEN ( ISNULL ( Table1.Completed , 0 ) = 0 AND ISNULL ( Table1.Cancelled , 0 ) = 0 AND ISNULL ( Table2.Over25000CEOApprovedORRejected , 0 ) > 0 AND Over25000CEOUser.EmailAddress <> Table3.Over25000CEOEmail AND ISNULL ( Over25000CEOUser.EmailAddress, '' ) <> '' AND ISNULL ( Table3.Over25000CEOEmail, '' ) = '' ) THEN Over25000CEOUser.UserDisplayName ELSE '' END AS Over25000CEO_ApprovedORRejectedBy_UserDisplayName_Calculated , ISNULL ( Table1.IsSubmitted , 0 ) AS isSubmitted , CASE WHEN ISNULL ( Table1.IsSubmitted , 0 ) = 1 THEN 'Submitted' ELSE 'Not Submitted' END AS isSubmitted_Status_Display_Text, Table1.SubmittedDate INTO #tempTable1 FROM Table1 INNER JOIN Table2 ON Table2.RequestInfoID = Table1.RequestInfoID LEFT OUTER JOIN Table3 ON Table3.UserID = Table1.UserID LEFT OUTER JOIN Table3_History_For_RequestInfo ON Table3_History_For_RequestInfo.UserID = Table1.UserID AND Table3_History_For_RequestInfo.RequestInfoID = Table1.RequestInfoID LEFT OUTER JOIN tblUserInfo ON tblUserInfo.UserID = Table1.UserID LEFT OUTER JOIN tblUserInfo Under5000User ON Under5000User.UserID = Table2.Under5000ApprovedORRejectedByUserID LEFT OUTER JOIN tblUserInfo Over5000User ON Over5000User.UserID = Table2.Over5000ApprovedORRejectedByUserID LEFT OUTER JOIN tblUserInfo FinanceUser ON FinanceUser.UserID = Table2.FinanceApprovedORRejectedByUserID LEFT OUTER JOIN tblUserInfo Over25000CFOUser ON Over25000CFOUser.UserID = Table2.Over25000CFOApprovedORRejectedByUserID LEFT OUTER JOIN tblUserInfo Over25000COOUser ON Over25000COOUser.UserID = Table2.Over25000COOApprovedORRejectedByUserID LEFT OUTER JOIN tblUserInfo Over25000CEOUser ON Over25000CEOUser.UserID = Table2.Over25000CEOApprovedORRejectedByUserID LEFT OUTER JOIN tblPurchaseCategory ON tblPurchaseCategory.PurchaseCategoryID = Table1.PurchaseCategoryID LEFT OUTER JOIN tblCurrency ON tblCurrency.CurrencyID = Table1.CurrencyID LEFT OUTER JOIN tblCompany ON tblCompany.CompanyID = Table1.CompanyID LEFT OUTER JOIN tblDepartment ON tblDepartment.DepartmentID = Table1.DepartmentID LEFT OUTER JOIN tblRegion ON tblRegion.RegionID = Table1.RegionID LEFT OUTER JOIN tblUserInfo Table_VerifiedBy_FinanceUser ON Table_VerifiedBy_FinanceUser.UserID = Table1.ReviewedByFinanceUserID SELECT #tempTable1.* , CASE WHEN ( ISNULL ( TotalAmount , 0 ) < 5000 ) THEN '' ELSE Under5000ApprovedORRejected_Text + ' by ' + Under5000_ApprovedORRejectedBy_UserDisplayName_Calculated + ' on ' + CONVERT(varchar(12) , Under5000DateApprovedORRejected , 101 ) END AS Display_Under5000ApprovedORRejected_Full_Text , INTO #tempTableForCalculation FROM #tempTable1 DECLARE @strSQL VARCHAR(MAX); SET @strSQL = ' SELECT * FROM #tempTableForCalculation ' ; IF ( ISNULL ( @FilterExpression , '' ) <> '' ) BEGIN SET @strSQL = @strSQL + ' WHERE ' + @FilterExpression ; END EXEC ( @strSQL ) ; IF OBJECT_ID('tempdb..#tempTable1') IS NOT NULL BEGIN DROP TABLE #tempTable1; END IF OBJECT_ID('tempdb..#tempTableForCalculation') IS NOT NULL BEGIN DROP TABLE #tempTableForCalculation; END END
Thanks in advance,
- Edited by MS User 1977 Friday, May 27, 2011 2:42 PM Formatted SQL properly and simplified question
Wednesday, May 25, 2011 2:10 PM
Answers
-
You are doing it wrong. :)
Seriously, you are thinking in imperative design style (Do this, then that, then the other) rather than in declarative design style (describe the set of results you need). You are also trying to get one procedure to do everything. That is not how SQL Server works. Your code is good for C# or VB style programming, but would be a nightmare for SQL Server. It would have an unstable query plan (sometimes good, sometimes bad) that was highly dependent on the actual parameters and the parameters the cached version used.
Write more procedures but simpler ones. Don't try and have one procedure do every possible data and parameter combination.
As Kent noted, you are asking for something that is likely beyond what we can do effectively in a forum post. Thanks for asking, but don't expect miracles.
Geoff N. Hiten Principal Consultant Microsoft SQL Server MVP- Proposed as answer by Naomi N Wednesday, May 25, 2011 3:56 PM
- Marked as answer by MS User 1977 Friday, May 27, 2011 2:10 PM
Wednesday, May 25, 2011 3:18 PM
All replies
-
Hello Nirav,
What your post describes is a project. Projects can be identified in a number of ways including the fact that the post could be broken down into multiple questions, the complexity of the question, or the size of the code that is posted. Projects are not within the scope of posts intended to be serviced by this forum.
There are a couple of options available. First, consider getting help from a consultant to help you with your project. Another alternative is to break your project into a number of smaller posts that can be addressed as single questions. If you decide to break this post into a sequence of individual questions, remember to fully describe your requirements for each single post.
Wednesday, May 25, 2011 2:18 PM -
You are doing it wrong. :)
Seriously, you are thinking in imperative design style (Do this, then that, then the other) rather than in declarative design style (describe the set of results you need). You are also trying to get one procedure to do everything. That is not how SQL Server works. Your code is good for C# or VB style programming, but would be a nightmare for SQL Server. It would have an unstable query plan (sometimes good, sometimes bad) that was highly dependent on the actual parameters and the parameters the cached version used.
Write more procedures but simpler ones. Don't try and have one procedure do every possible data and parameter combination.
As Kent noted, you are asking for something that is likely beyond what we can do effectively in a forum post. Thanks for asking, but don't expect miracles.
Geoff N. Hiten Principal Consultant Microsoft SQL Server MVP- Proposed as answer by Naomi N Wednesday, May 25, 2011 3:56 PM
- Marked as answer by MS User 1977 Friday, May 27, 2011 2:10 PM
Wednesday, May 25, 2011 3:18 PM -
Thanks Naomi,
Yes, I got your point, so I will break down in multiple stored procedures.
Actually my intention was not to have any logic or anything, I just wanted to know what is the correct way to achieve what I have at present and your answer is perfect to understand what approach I should take in this scenario.
I just pasted my stored procedure to let know how it is at present, as I also mentioned in my original post, I was just looking for a proper approach to achieve something like this and you gave me perfect idea.
Thanks again,
NiravFriday, May 27, 2011 2:10 PM -
Thanks Geoff, you meant to say :) I was just agreeing with Geoff and decided I can not post it better than him.
For every expert, there is an equal and opposite expert. - Becker's Law
My blogFriday, May 27, 2011 2:13 PM