Poly base script

Question

I am using DWU 100 unit in our SQL data warehouse and facing issue while loading data from blob storage to stage environment for few tables. Poly base script is failing sometimes and we are getting below error:

 

“Execution fail against sql server. Sql error number: 35386. Error Message: Unable to allocate 393851 KB for columnstore compression because it exceeds the remaining memory from total allocated for current resource class and DWU. Please rerun query at a higher resource class, and also consider increasing DWU. See ... for assistance.”

 

Same Polybase script is working fine if we change resource_class to “xlargerc”.

Answer 1

Hi Ashu101,

Thanks for reaching out. From the error message is states that there is insufficient memory available for the executing query due to the current DW SLO level and executing user concurrency slot allocation. 

I have also noticed that you have filed a SR for deeper investigation on this issue. A support engineer will provide 1:1 assistance to identify the root cause and provide a resolution. Once the issue is resolved by support engineer, please do consider to share the resolution details here with the community, that way it will be beneficial for other members of the community who reads this thread. 

 

---

Thank you

If a post helps to resolve your issue, please click the "Mark as Answer" of that post and/or click "Vote as helpful" button of that post. By marking a post as Answered and/or Helpful, you help others find the answer faster.

Answer 2

Update: From the Support Ticket, it is recommended that if you are running CCI rebuild/create then please use largeRC or more. 

Helpful docs:

• https://docs.microsoft.com/azure/synapse-analytics/sql-data-warehouse/memory-concurrency-limits#concurrency-maximums-for-resource-classes
• https://docs.microsoft.com/azure/synapse-analytics/sql-data-warehouse/sql-data-warehouse-memory-optimizations-for-columnstore-compression#how-to-estimate-memory-requirements
  
  
  

---

Thank you

If a post helps to resolve your issue, please click the "Mark as Answer" of that post and/or click "Vote as helpful" button of that post. By marking a post as Answered and/or Helpful, you help others find the answer faster.