none
copy activity -> keep source data column names and data type RRS feed

  • Question

  • I am trying to copy data from csv on my local drive to Azure blob storage. To do so I created ADF pipeline. I need one to one copy without any modifications, so the copied file on blob should be exactly the same as my original one. 

    The original file has 124 columns some of them strings, some of them numbers etc. Two question to this:

    1) How to keep the same column naming schema? I do not want Prop_0, Prop_1 etc. I do not want to manually change all the 124 columns too. I would like this to happen dynamically

    2) I would expect the program dynamically recognizes data types for both source and sink data, but in fact it doesn't. This is an inefficient job to do this manually. How to do this?


    Tuesday, January 22, 2019 10:30 AM

Answers

  • Hi,

    How about just uploading the local file to your blob storage?

    If you want to go the Azure Data Factory way, first make sure that in the Linked Service for the Source, in the connection tab you have checked the option - "Column names in the first row". Here's a figure showing the same :

    Next, in the copy activity, in the Mapping tab, make sure you don't import the schema as you have already imported the schema in the source Linked Service. If you have imported the schema, press "Clear" to remove the mapping. This will ensure that the copy behavior is 1 to 1 and there's no new column mapping.

    Let us know if this helps. Else, we can gladly dig deeper.


    MSDN

    Tuesday, January 22, 2019 12:27 PM
    Owner

All replies

  • Hi,

    How about just uploading the local file to your blob storage?

    If you want to go the Azure Data Factory way, first make sure that in the Linked Service for the Source, in the connection tab you have checked the option - "Column names in the first row". Here's a figure showing the same :

    Next, in the copy activity, in the Mapping tab, make sure you don't import the schema as you have already imported the schema in the source Linked Service. If you have imported the schema, press "Clear" to remove the mapping. This will ensure that the copy behavior is 1 to 1 and there's no new column mapping.

    Let us know if this helps. Else, we can gladly dig deeper.


    MSDN

    Tuesday, January 22, 2019 12:27 PM
    Owner
  • You're right. Clearing schema from mapping and destination file solved the issue. Thanks a lot!
    Tuesday, January 22, 2019 4:03 PM
  • Happy to help !

    MSDN

    Wednesday, January 23, 2019 8:35 AM
    Owner