locked
Validation Activity issues RRS feed

  • Question

  • Hello,

    Was trying to use the validation activity to only continue pipeline if the dataset was availlable and was greater than 10 Bytes. But there were numerous problems with the activity. First of all the settings doesn't contain the option for setting minimum filesize eventhough the documentation says it is an option:

    

    Then had to manually amended the JSON to include the minimum filesize parameter.

    When I tried to do a debug run it failed with the following error; for this test I also selected Child Items = True as I wanted it to block execution if no files in folder (of dataset):

    {
        "errorCode": "3500",
        "message": "Field 'childItems' failed with error: 'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Cannot navigate through the provided data source. Make sure the provided data source is a directory that can do navigation.,Source=Microsoft.DataTransfer.ClientLibrary,'.",
        "failureType": "UserError",
        "target": "Validation1",
        "details": ""
    }


    Then when I selected Child Items = Ignore, it succeeded, but I expected it to fail as the actual file size was 0B and I put the minimum filesize as 10.

    I used to Get Metadata activity to check the file size, and it was showing as 0B, but the validation activity succeeded in 4s, when I was expecting it to timeout after 1 minute.

     Thia validation activity doesn't appear to work, and the documentation is not clear: https://docs.microsoft.com/en-gb/azure/data-factory/control-flow-validation-activity

    Monday, December 30, 2019 11:32 AM

All replies

  • It worked for me. For a parquet file having 440B data, I set min size as 1000. Triggered the pipeline and it went in Time out mode. 

    My dataset directly points to the parquet file. Ensure the variable in your dataset is pointing to correct file. 


    If the response helped, do "Mark as answer" and upvote it
    - Vaibhav

    Monday, December 30, 2019 1:36 PM