none
Copy activity logs show long strings from dataset cannot be truncated.

    Question

  • I have a copy activity from blob storage to sql server.  I kept running a pipeline, but not all the rows were being picked up. I set the settings to "skip and log incompatible rows" and looked at the log file. The column 'short description' seems to be the problem as the rows with longer text here have a message that says "string or binary data would be truncated".  My question is, if I have the source data field as a "string", how long can the string be? I am guessing it is defaulted to varchar(50) based on the length of text.  I am only using the Azure Data Factory UI.  Also, how can I fix this? I do not have access to the dataset to remove characters and was hoping Azure could handle this simple step.


    josh

    Wednesday, October 31, 2018 5:24 PM

All replies

  • Hi Josh,

    For the 'short_description' column, how is this declared in the target database? You are going to need access to the dataset to scrub the data you don't want or increase the data definition to allow all data to pass through. Azure Data Factory cannot perform anything you don't tell it how and what to do. 

    Please let us know if you have additional question. ~Mike

    Wednesday, October 31, 2018 6:08 PM
    Moderator
  • Hi Mike, thanks for responding.

    short description is a field in an excel document that is sent via email. I have the pipeline keeping the attachment and saving in blob. ADF picks up the file as the source. So, I don't define it initially at all except through the source dataset settings. There doesn't seem to be much flexibility in Azure. What do you recommend?


    josh

    Wednesday, October 31, 2018 6:14 PM