This site uses cookies for analytics, personalized content and ads. By continuing to browse this site, you agree to this use. Learn more
Microsoft Logo
Gray Pipe
Developer Network Developer Network Developer
:CreateViewProfileText: Sign in
Subscriber portal
Get tools
  • Downloads
    • Visual Studio
    • SDKs
    • Trial software
      • Free downloads
      • Office resources
  • Programs
    • Subscriptions
      • Overview
      • Administrators
    • Students
      • Microsoft Imagine
      • Microsoft Student Partners
    • ISV
    • Startups
    • Events
  • Community
    • Magazine
    • Forums
    • Blogs
    • Channel 9
  • Documentation
    • APIs and reference
    • Dev centers
    • Samples
    • Retired content
search clear
We’re sorry. The content you requested has been removed. You’ll be auto redirected in 1 second.
Ask a question
Quick access
  • Forums home
  • Browse forums users
  • FAQ
Announcement: 2

Azure Data Factory announcement

  • Link
    Azure Databricks forum is available on the MSDN forum now.
    CHEEKATLAPRADEEP-MSFT Friday, October 25, 2019 9:29 AM

    We have noticed customers post their questions on Azure Databricks on incorrect forums on MSDN, since there is a new home for Azure Databricks.

    Forum Link: https://social.msdn.microsoft.com/Forums/en-US/home?forum=AzureDatabricks

    

  • Link
    Welcome to Azure Data Factory MSDN Forum
    Gaurav Malhotra-MSFT Tuesday, October 28, 2014 12:22 AM

    Azure Data Factory allows you to manage the production of trusted information by offering an easy way to create, orchestrate, and monitor data pipelines over the Hadoop ecosystem using structured, semi-structures and unstructured data sources.

    You can connect to your on-premises SQL Server, Azure database, tables or blobs and create data pipelines that will process the data with Hive and Pig scripting, or custom C# processing. The service offers a holistic monitoring and management experience over these pipelines, including a view of their data production and data lineage down to the source systems.

    The outcome of Data Factory is the transformation of raw data assets into trusted information that can be shared broadly with BI and analytics tools.

    The MSDN forum will be used for general discussions for Getting Started, Development, Management, and Troubleshooting using Azure Data Factory.

RRS feed

  • Remove From My Forums

Forums [ view all ]

Selected forums

Clear
Azure Data Factory
x
Filter : All threads
All threads
Answered
Unanswered
Proposed answers
General discussion

No replies
Helpful
Has code

All languages
Sort : Most recent post
  • Most recent post
  • Most recent thread
  • Total votes
  • Total replies
  • large check mark
    0 Votes

    Event Trigger is not working

    Microsoft Azure
     > 
    Azure Data Factory
    Hi, The event trigger is not working in Data Factory. We have created data factory event trigger on the creation of blob. The trigger works in the lower environment but it is not ...
    Unanswered | 4 Replies | 210 Views | Created by Ajinkya2210 - Wednesday, December 11, 2019 9:40 AM | Last reply by MartinJaffer-MSFT - Friday, December 13, 2019 1:30 AM
  • large check mark
    0 Votes

    Fault Tolerance not working

    Microsoft Azure
     > 
    Azure Data Factory
    Hi, The Fault Tolerance option in the Copy Data component is not working. We had our pipelines running fine till 8-Dec-2019 but since that it is failing. We have selected the Skip ...
    Proposed | 3 Replies | 250 Views | Created by Ajinkya2210 - Tuesday, December 10, 2019 11:58 AM | Last reply by HimanshuSinha-msft - Friday, December 13, 2019 12:26 AM
  • large check mark
    0 Votes

    ADF Expression Split?

    Microsoft Azure
     > 
    Azure Data Factory
    Hi again I am stumbling on an expression syntax here which is critical to our App The original source file can ...
    Proposed | 2 Replies | 218 Views | Created by EMKISER - Friday, December 6, 2019 4:58 PM | Last reply by HimanshuSinha-msft - Thursday, December 12, 2019 8:08 PM
  • large check mark
    2 Votes

    Event Trigger - Error when getting container list

    Microsoft Azure
     > 
    Azure Data Factory
    Hi I have a working Event Trigger against our test blob storage (regular blob storage v2 for our test environment), but when I try to create a new trigger against out Production ...
    Answered | 13 Replies | 419 Views | Created by Stranded Software - Thursday, December 5, 2019 11:18 AM | Last reply by Cesar Berard - Thursday, December 12, 2019 7:27 PM
  • large check mark
    0 Votes

    Data type mapping for SAP BW - how to handle Null

    Microsoft Azure
     > 
    Azure Data Factory
    We are bringing SAP Bex query data into ADLS. There is a default data type mapping as per document BW data type mapping. However, we occasionally get the ...
    Proposed | 3 Replies | 162 Views | Created by SAP to ADLS - Tuesday, December 10, 2019 6:10 PM | Last reply by SAP to ADLS - Thursday, December 12, 2019 5:48 PM
  • large check mark
    0 Votes

    How to acess the trigger property(frequency,interval) to pass dynamically to adf pipeline

    Microsoft Azure
     > 
    Azure Data Factory
    "type": "ScheduleTrigger", "typeProperties": { "recurrence": { "frequency": ...
    Proposed | 2 Replies | 149 Views | Created by SubhadipRoy - Thursday, December 12, 2019 12:04 AM | Last reply by SubhadipRoy - Thursday, December 12, 2019 2:31 PM
  • large check mark
    0 Votes

    Parameter to dynamically retrieve 'Activity Name' of a pipeline

    Microsoft Azure
     > 
    Azure Data Factory
    Hi Everyone, I am trying to store details of Pipeline Name,Pipeline Id and Activity Name in SQL Table by executing SP in pipeline which is followed by an ...
    Unanswered | 1 Replies | 74 Views | Created by sai_09 - Thursday, December 12, 2019 11:51 AM | Last reply by Zekaryah - Thursday, December 12, 2019 2:03 PM
  • large check mark
    0 Votes

    DataFlow Can't Connect To SQL Source Dataset

    Microsoft Azure
     > 
    Azure Data Factory
    I am working in Azure Data factory (v2) on moving data from CSV files to Azure SQL Database. The Copy data is working fine for moving data, but I need to do some modification on the data ...
    Unanswered | 15 Replies | 565 Views | Created by Joe Tannouri - Monday, December 2, 2019 8:51 AM | Last reply by Joe Tannouri - Thursday, December 12, 2019 1:05 PM
  • large check mark
    1 Votes

    Developments tools other than Author/monitor portal in ADF

    Microsoft Azure
     > 
    Azure Data Factory
    Hi, I am exploring Azure Data Factory, i see Azure is providing Author/monitor portal but I am looking for any other development tools such as Visual studio, Visual code or any ...
    Proposed | 3 Replies | 139 Views | Created by Cloud learner - Wednesday, December 11, 2019 12:07 PM | Last reply by Zekaryah - Wednesday, December 11, 2019 4:58 PM
  • large check mark
    0 Votes

    How to write the activity status to a parameter

    Microsoft Azure
     > 
    Azure Data Factory
    I need to capture the system variables (Pipeline Name, etc. ) in addition to some other pieces of information about an activity (Status, Name, Rows Read) and pass it from the Copy task to a Stored ...
    Proposed | 3 Replies | 607 Views | Created by Shannon Royer - Wednesday, June 13, 2018 3:21 PM | Last reply by Andries van der Walt - Wednesday, December 11, 2019 12:17 PM
  • large check mark
    0 Votes

    Importing Daily snapshots of data to Data Lake Storage Gen 2

    Microsoft Azure
     > 
    Azure Data Factory
    I have a source table in a SQL Database that contains daily snapshots of whats going on. What I want to do is get daily files of each snapshot out into a Data Lake Gen 2 Data store ...
    Proposed | 1 Replies | 104 Views | Created by Debbie Edwards - Wednesday, December 11, 2019 9:25 AM | Last reply by Vaibhav-Chaudhari - Wednesday, December 11, 2019 10:07 AM
  • large check mark
    0 Votes

    Order in foreach

    Microsoft Azure
     > 
    Azure Data Factory
    I've made some pipelines; Fill staging tablesExecute all stored proceduresRefresh Power BI datasets (in progress)Master pipeline which first executes the fill staging ...
    Proposed | 4 Replies | 270 Views | Created by MiKeHendriks - Friday, December 6, 2019 8:49 PM | Last reply by MiKeHendriks - Wednesday, December 11, 2019 8:24 AM
  • large check mark
    0 Votes

    How to process match and Nomatch data in ADF data flow?

    Microsoft Azure
     > 
    Azure Data Factory
    Hi, I am working on to transfer data from one database to another using ADF dataflow (Mapping dataflow). There are some scenario where I need to look for lookup ...
    Proposed | 6 Replies | 319 Views | Created by AmitBDesai - Friday, December 6, 2019 1:20 AM | Last reply by AmitBDesai - Wednesday, December 11, 2019 3:47 AM
  • large check mark
    0 Votes

    Issues in self hosted integration runtime connectivity

    Microsoft Azure
     > 
    Azure Data Factory
    Hi Team ,  I have created a self hosted IR and I am trying to connect to on-prem oracle database . However I am facing below error ...
    Unanswered | 2 Replies | 266 Views | Created by Lopamudra Kanungo - Wednesday, November 27, 2019 4:49 PM | Last reply by HimanshuSinha-msft - Tuesday, December 10, 2019 7:37 PM
  • large check mark
    0 Votes

    Failed activity does not fail the pipeline

    Microsoft Azure
     > 
    Azure Data Factory
    Hello, I have a pipeline with few dependent activities and one of them is a Stored Procedure activity. When this SP activity failes (for some reasons) the activity is marked as ...
    Proposed | 8 Replies | 390 Views | Created by paljon - Thursday, November 21, 2019 11:58 AM | Last reply by HimanshuSinha-msft - Tuesday, December 10, 2019 7:35 PM
  • large check mark
    0 Votes

    Gen2 to SQL Database Table adding derived column during Copy Activity

    Microsoft Azure
     > 
    Azure Data Factory
    How to Add derived column using the copy activity from source Gen2 to target SQL Database Table. I come across option to define explicit schema in configuration table but in my ...
    Proposed | 3 Replies | 215 Views | Created by Sam cit - Wednesday, December 4, 2019 6:41 PM | Last reply by MartinJaffer-MSFT - Tuesday, December 10, 2019 5:50 PM
  • large check mark
    0 Votes

    Your Azure-SSIS IR has failed to start due to custom setup failure.

    Microsoft Azure
     > 
    Azure Data Factory
    Hi, I am working on the Azure SSIS IR. Where i need to deploy the SSIS package on the Azure so i am creating IR(Integration Runtime) in the Azure Data Factory but i am facing below ...
    Answered | 3 Replies | 758 Views | Created by Sam D'Souza - Tuesday, September 4, 2018 6:51 AM | Last reply by TariqMAwan - Tuesday, December 10, 2019 12:28 PM
  • large check mark
    2 Votes

    Azure Data Factory Pagination with offset & limit paramters

    Microsoft Azure
     > 
    Azure Data Factory
    Hello, I have a Rest API that I want to copy data from into a data lake however the api uses offset and limit query parameters for Pagination. I was wonder how I would do this with Pagination ...
    Answered | 2 Replies | 132 Views | Created by BryBry02 - Monday, December 9, 2019 7:33 PM | Last reply by ChiragMishra-MSFT - Tuesday, December 10, 2019 9:41 AM
  • large check mark
    0 Votes

    if exists

    Microsoft Azure
     > 
    Azure Data Factory
    In my Azure Data Factory pipeline, there is a copy activity with an input parameter. which copies data from on prem sql server to Azure sql server... The destination (Sink) table ...
    Proposed | 2 Replies | 240 Views | Created by arkiboys - Saturday, December 7, 2019 3:01 PM | Last reply by ChiragMishra-MSFT - Tuesday, December 10, 2019 9:16 AM
  • large check mark
    1 Votes

    Connecting to Oracle DB which is in On Premise Server

    Microsoft Azure
     > 
    Azure Data Factory
    Hi All, I have a requirement where I need to get data from Oracle DB which is on premise server. I have got the solution of Self Hosted ...
    Proposed | 5 Replies | 317 Views | Created by Pankaj Mahra - Thursday, November 28, 2019 1:14 PM | Last reply by ChiragMishra-MSFT - Tuesday, December 10, 2019 9:09 AM
  • < Previous Items 21 to 40 of 3135 Next ›
Announcement: 2

Azure Data Factory announcement

  • Link
    Azure Databricks forum is available on the MSDN forum now.
    CHEEKATLAPRADEEP-MSFT Friday, October 25, 2019 9:29 AM

    We have noticed customers post their questions on Azure Databricks on incorrect forums on MSDN, since there is a new home for Azure Databricks.

    Forum Link: https://social.msdn.microsoft.com/Forums/en-US/home?forum=AzureDatabricks

    

  • Link
    Welcome to Azure Data Factory MSDN Forum
    Gaurav Malhotra-MSFT Tuesday, October 28, 2014 12:22 AM

    Azure Data Factory allows you to manage the production of trusted information by offering an easy way to create, orchestrate, and monitor data pipelines over the Hadoop ecosystem using structured, semi-structures and unstructured data sources.

    You can connect to your on-premises SQL Server, Azure database, tables or blobs and create data pipelines that will process the data with Hive and Pig scripting, or custom C# processing. The service offers a holistic monitoring and management experience over these pipelines, including a view of their data production and data lineage down to the source systems.

    The outcome of Data Factory is the transformation of raw data assets into trusted information that can be shared broadly with BI and analytics tools.

    The MSDN forum will be used for general discussions for Getting Started, Development, Management, and Troubleshooting using Azure Data Factory.

Help us improve MSDN.
Visit our UserVoice Page to submit and vote on ideas!
Make a suggestion

Dev centers

  • Windows
  • Office
  • Visual Studio
  • Microsoft Azure
  • More...

Learning resources

  • Microsoft Virtual Academy
  • Channel 9
  • MSDN Magazine

Community

  • Forums
  • Blogs
  • Codeplex

Support

  • Self support

Programs

  • BizSpark (for startups)
  • Microsoft Imagine (for students)
United States (English)
  • Newsletter
  • Privacy & cookies
  • Terms of use
  • Trademarks
logo © 2019 Microsoft