locked
Performance - azure data factory RRS feed

  • Question

  • Can someone help me with this doubts:?

    1-It is more performatic create one pipeline per each database or
    Just one pipeline to all databaes?

    2-what is benefits of use the storage account, for example, I will copy data from
    My data source oracle to sql database, it is better create a storage account, like stage area, or
    Create a direct connection source and destionation?
    Storage works like BCP in sql server?

    What is the best practice with data factory?

    Tks

    Tuesday, January 10, 2017 3:52 PM

Answers

  • hello,

    1- For this questions, its depend the situation that u have, for me for example, in same pipeline i have diferent dataset that use diferents database connections and its work well, however depend that rows u process, for example for different size its more efficente build differente pipeline, one pipeline per each database its more efficent when u have heavy buffers.

    2- the benefits to use a storage account its that u have a extra stage area, this can fix problem with time out, bad loads or loss of information u can put in this stage some controls, and when u use this area u have DF connect with Azure this is so fast because the connection depends for azure services. However when u have a direct query with personal DB as source or destination u depend for this "personal" connection. make sense what i said?.

    Storage dont works like a BCP, the reason for most effectivity its all source and destination depends for azure services, all its cloud.

    i hope help you and sorry for my language.

    Wednesday, January 11, 2017 9:41 AM