How to handle schema changes on relational data that is being pushed to ADLS


  • We are working on an implementation where relational data from Azure SQL is being pushed to ADLS via Azure Data Factory. The incremental data push will be handled through watermarks stored in the DB. I want to understand how are schema changes on Azure SQL handled when copying. As I understand, the data is stored on ADLS as a file and old data and new data would vary in schema ? Since this is schema on read, does it have to be handled only when reading/accessing this data ?
    Wednesday, October 4, 2017 10:45 AM


  • I'd be happy to help. 

    I'm assuming that you're using U-SQL, if so, yes you will handle this when doing the extraction, and you can also look at outer union to unify disparate schema.

    Thank you,


    • Marked as answer by Ankita Sanghvi Thursday, October 5, 2017 5:32 AM
    Wednesday, October 4, 2017 10:20 PM