none
Alternative to Polybase with extended ASCII RRS feed

  • Question

  • Hello,

    Im trying to populate a DW with the fastest technique and according to the documentation is Polybase but i live in a country were we use extended ASCII (ñ, á, é, í, etc).

    I've achieved this using ADF and copying from an Azure SQL (my staging) onto my DW.

    There is a better way to achieve this? or what is the fastest way to do it?

    Thanks!

    Wednesday, August 14, 2019 9:21 PM

All replies

  • Hi Clizana,

    Please see the following MSDN post with regard to the same issue and the solution that worked: Polybase / External Table and Data Factory failing for Rows with character " é " (link)

    Can you perform a small exercise wher eyou export your data using bcp to UTF-16 encoding (bcp uses UTF-16 by default) and then run the following Powershell command on the exported .rtf file: 

    Get-ChildItem -rec -fi *.rtf | %{(get-content $_ -enc string) | set-content -enc utf8 $_.fullname}

    This will convert the .rtf file from UTF-16 to UTF-8, which is compatible with Polybase. The idea here is to export a small sample of your data (that contains the undesired characters) using bcp, and then convert this to UTF-8 to see if the characters have been reformatted. 

    There is additional information here from a 3rd party blog: Loading Data into Azure SQL Data Warehouse

    Please let us know if you have additional questions.

    Regards,

    Mike

    Wednesday, August 14, 2019 11:27 PM
    Moderator
  • Thank you!
    Friday, August 16, 2019 10:33 PM