locked
Copy Data from Mongo DB to Azure SQL DW using Polybase and Staging RRS feed

  • Question

  • Hi,

    i'm trying to copy data from MongoDB using the V2 connector to an Azure SQL DW and if I enable polybase and staging via an storage account, all properties that are not in the root are writen as empty strings in the DW.  IE: user.username.  

    If I disable staging and polybase the data is copied correctly, but at a much slower pace.

    Is there some configuration that would allow this or is this not yet supported?

    Saturday, July 20, 2019 11:18 PM

All replies

  • Hi Jose Vanderhorst OpSec,

    Below are the considerations if there is a slow perf when copy data into SQL DW

    Use Polybase: 

    1. If source is Azure Blob or ADLS and the source data format is polybase compatible, then Direct Copy using Polybase to have best performance. 
    2. If source is other binary data source (e.g. S3, SFTP) and the source data format is polybase compatible, then Staged Copy using Polybase to have best performance. 
    3.  Else, Direct Copy without Polybase.

    SQL DW tier:

    1.        The performance would be slow if the tier is not high. E.g. the tier is 100 DWU. (Please check on this)

    Additional info:

    Please refer to this doc for more info about direct copy using polybase.

    Please refer to this doc to know more about the copy performance.

     

    Hope this helps.


    [If a post helps to resolve your issue, please click the "Mark as Answer" of that post and/or click Answered "Vote as helpful" button of that post. By marking a post as Answered and/or Helpful, you help others find the answer faster. ]

    Tuesday, July 23, 2019 1:25 AM
  • Hi Jose Vanderhorst OpSec,

    Just checking see if the above suggestion was helpful. If this answers your query, please click “Mark as Answer” and "Up-Vote" as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

    Thank you.


    [If a post helps to resolve your issue, please click the "Mark as Answer" of that post and/or click Answered "Vote as helpful" button of that post. By marking a post as Answered and/or Helpful, you help others find the answer faster. ]

    Tuesday, July 23, 2019 7:20 PM
  • Hi Jose Vanderhorst OpSec,

    Following up to see if the above suggestion was helpful. And, if you have any further query do let us know.

    Thank you.


    [If a post helps to resolve your issue, please click the "Mark as Answer" of that post and/or click Answered "Vote as helpful" button of that post. By marking a post as Answered and/or Helpful, you help others find the answer faster. ]

    Wednesday, July 24, 2019 8:39 PM