Azure Data Factory announcement
-
Link
We have noticed customers post their questions on Azure Databricks on incorrect forums on MSDN, since there is a new home for Azure Databricks.
Forum Link: https://social.msdn.microsoft.com/Forums/en-US/home?forum=AzureDatabricks
-
Link
Azure Data Factory allows you to manage the production of trusted information by offering an easy way to create, orchestrate, and monitor data pipelines over the Hadoop ecosystem using structured, semi-structures and unstructured data sources.
You can connect to your on-premises SQL Server, Azure database, tables or blobs and create data pipelines that will process the data with Hive and Pig scripting, or custom C# processing. The service offers a holistic monitoring and management experience over these pipelines, including a view of their data production and data lineage down to the source systems.
The outcome of Data Factory is the transformation of raw data assets into trusted information that can be shared broadly with BI and analytics tools.
The MSDN forum will be used for general discussions for Getting Started, Development, Management, and Troubleshooting using Azure Data Factory.
-
0 Votes[Announcement] Microsoft Q&A to replace all English Azure MSDN forums
Azure Data Factory forum will be migrating to a new home on Microsoft Q&A! We’ve listened to your feedback on ...
-
2 Votes
Why does my azure managed postgresql server terminates the connection prematurely in ADF Copy activity
Hello I have an Azure managed PostgreSQL Server. I am copying data from a table in a database on this server to an Azure managed MySQL Server database. The size if the table can be ...Answered | 6 Replies | 3841 Views | Created by samkg - Tuesday, December 3, 2019 7:20 PM | Last reply by Sundeep001 - Tuesday, June 9, 2020 9:55 PM -
0 Votes
GetCommandOutputAsync failed with
I'm getting this error when attempting data preview. No other information, just "GetCommandOutputAsync failed with". Occasionally I'm getting GetCommandOutputAsync failed with ...Unanswered | 10 Replies | 1779 Views | Created by richbeedf - Monday, March 23, 2020 2:34 PM | Last reply by Danimal_RockStar - Tuesday, June 9, 2020 6:32 PM -
0 Votes
Release ADF via DevOps
Which DevOps release task is the best task to release ADF. I see a lot of Azure resource group deployment tasks, but the deployment mode Complete empties the entire resource group and the ...Unanswered | 1 Replies | 1763 Views | Created by SSISJoost - Thursday, May 14, 2020 2:27 PM | Last reply by MartinJaffer-MSFT - Tuesday, June 9, 2020 6:14 PM -
0 Votes
ADF v2 copy activity long queue time
I have create a pipeline for copy data to DB from on-premise file (Azure VM), and finally move the file to backup folder. I have create a GetMetadata activity and CopyData activity in the pipeline, ...Proposed | 13 Replies | 5856 Views | Created by ChrisLau3636 - Friday, July 12, 2019 8:20 AM | Last reply by Ash2319 - Tuesday, June 9, 2020 8:55 AM -
0 Votes
ADF event-based trigger starts working with delay
Hi All We have faced with the issue when event-based trigger in Azure Data Factory doesn't start work when we expect. We have a ...Unanswered | 4 Replies | 2352 Views | Created by Artem_A - Monday, May 11, 2020 3:58 PM | Last reply by Artem_A - Tuesday, June 9, 2020 8:21 AM -
0 Votes
Linked Service changes 'Account selection Method' when deploying using Azure Devops pipeline
Hi, I am deploying my pipelines from 'Dev' to 'Prod' environment. I am using the 'arm-template-parameters-definition.json' to get custom parameters for the linked ...Unanswered | 4 Replies | 1794 Views | Created by Raj_Hillman - Wednesday, May 6, 2020 7:52 AM | Last reply by MartinJaffer-MSFT - Tuesday, June 9, 2020 12:36 AM -
0 Votes
Issue with SFTP connector
Hi Team, I am using SFTP connector in ADF Copy activity for source. It was working correctly getting the following issue from 14th May. Using Azure IR, Test ...Proposed | 11 Replies | 4989 Views | Created by RaviKrishnaJ - Wednesday, May 27, 2020 3:36 PM | Last reply by KranthiPakala-MSFT - Monday, June 8, 2020 11:01 PM -
0 Votes
How to save a string from returned HTTP results to use further in Pipeline
I have created a Copy item to retrieve data from an HTTP SOAP API. The returned results from this API are saved as binary data to my sink. Part of this returned data from the SOAP ...Proposed | 5 Replies | 1868 Views | Created by Mysticode - Thursday, May 28, 2020 9:28 PM | Last reply by Mysticode - Monday, June 8, 2020 10:40 PM -
0 Votes
copy activity additional headers issue
I am passing below @{concat('{','Content-Length: 0 Content-Type: application/x-www-form-urlencoded ','}')} as additional headers in copy activity to call ...Proposed | 3 Replies | 3134 Views | Created by pooja30 - Friday, May 29, 2020 10:24 AM | Last reply by ChiragMishra-MSFT - Monday, June 8, 2020 4:52 AM -
0 Votes
Exclude Azure-SSIS integration runtimes in devops release not working
Hello, I've got 3 ADF environments (dev,qa,test) that all use the same integration runtimes but with different server creds. I'm trying to exclude the IRs from my devops ...Unanswered | 2 Replies | 1864 Views | Created by Mitchell2012 - Wednesday, May 13, 2020 7:27 PM | Last reply by KranthiPakala-MSFT - Friday, June 5, 2020 10:04 PM -
0 Votes
How can I tell ADF to not overwrite the files that exists in both the containers, just write the files that are present in the source?
Hi can I use the control statements to not overwrite the files that exists in both the containers (Source and Destination) only copy the files which are not present in the destination? is there any ...Proposed | 3 Replies | 2500 Views | Created by anmol.gan - Friday, March 1, 2019 9:16 AM | Last reply by BodhuluruMonica - Friday, June 5, 2020 1:26 PM -
0 Votes
Data Factory PowerShell deployment: West Europe not available
Hi team, When will the western europe region become available again for powershell deployments for ADF? At the moment I am getting this error. Error Message: The ...Proposed | 3 Replies | 2025 Views | Created by Haassiej - Wednesday, May 13, 2020 6:45 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Friday, June 5, 2020 12:30 PM -
1 Votes
Use self dependency TWT to enforce max concurrency
When configuring a Tumbling Window Trigger I know I can set the max concurrency. But max concurrency doesn't apply to trigger reruns, and I want it to. I'm thinking that I can use a self dependency to ...Answered | 3 Replies | 1823 Views | Created by jader3rd - Thursday, May 28, 2020 5:19 PM | Last reply by MartinJaffer-MSFT - Thursday, June 4, 2020 8:52 PM -
0 Votes
Getting an error while deploying ARM template through C# code
Hi Geeks, I am trying to deploy ARM template for Data Factory in Azure through C# code. But every time I am getting below error "The request content was ...Proposed | 4 Replies | 1781 Views | Created by Ramandeep123 - Monday, June 1, 2020 5:54 AM | Last reply by Ramandeep123 - Thursday, June 4, 2020 6:39 AM -
0 Votes
Mapping Data Flow - Expression iifnull issue
Hello, Was using this iifNull function in a derived column, the documentation says I can specify 3 arguments where the 3rd argument was evaluated when condition was not ...Proposed | 2 Replies | 1691 Views | Created by SmoothOrange - Monday, May 25, 2020 3:55 PM | Last reply by HimanshuSinha-msft - Wednesday, June 3, 2020 9:35 PM -
0 Votes
Azure Key-Vault
Hi Team, I want to retrieve client id and secrets from Azure key-vault by using Python script.As per my knowledge i passed all relevant information which is required for ...Proposed | 6 Replies | 1792 Views | Created by pankaj92 - Monday, May 25, 2020 5:28 AM | Last reply by HimanshuSinha-msft - Wednesday, June 3, 2020 9:34 PM -
0 Votes
FOR Each Loop Help Require In ADF
hi all, I m new to ADF SO need help in For each loop. i have below MASTER pipeline. 1. file ...Unanswered | 4 Replies | 2113 Views | Created by Aminesh - Tuesday, May 26, 2020 10:24 AM | Last reply by HimanshuSinha-msft - Wednesday, June 3, 2020 9:26 PM -
0 Votes
How to push data from Azure Data lake to remote file server (network file folder)
XYZ or \\XYZ . Does i need to install IR run time there also if so where I have to install . Can you please help me in achieving this . Your Help is very much appreciated. and it ...Proposed | 3 Replies | 1741 Views | Created by fbmsdn511 - Monday, June 1, 2020 5:40 AM | Last reply by KranthiPakala-MSFT - Wednesday, June 3, 2020 8:44 PM -
0 Votes
Mapping Data Flows - replace column names issues
Hello, I was trying to replace all dots '.' with underscore '_' for the columns names using a rule based mapping in the select transform, but it only seems to replace the first ...Proposed | 3 Replies | 1964 Views | Created by SmoothOrange - Monday, May 18, 2020 4:51 PM | Last reply by KranthiPakala-MSFT - Wednesday, June 3, 2020 8:08 PM -
0 Votes
ARM template parameter definition for CDS for Apps in ADF
Hello, I'm building azure devops pipeline releases from our dev ADF to test ADF environment. Within my ARM .json templates, I am specifying the new connection ...Proposed | 1 Replies | 1654 Views | Created by Mitchell2012 - Wednesday, May 13, 2020 7:15 PM | Last reply by HimanshuSinha-msft - Wednesday, June 3, 2020 6:12 PM - Items 1 to 20 of 3766 Next ›
Azure Data Factory announcement
-
Link
We have noticed customers post their questions on Azure Databricks on incorrect forums on MSDN, since there is a new home for Azure Databricks.
Forum Link: https://social.msdn.microsoft.com/Forums/en-US/home?forum=AzureDatabricks
-
Link
Azure Data Factory allows you to manage the production of trusted information by offering an easy way to create, orchestrate, and monitor data pipelines over the Hadoop ecosystem using structured, semi-structures and unstructured data sources.
You can connect to your on-premises SQL Server, Azure database, tables or blobs and create data pipelines that will process the data with Hive and Pig scripting, or custom C# processing. The service offers a holistic monitoring and management experience over these pipelines, including a view of their data production and data lineage down to the source systems.
The outcome of Data Factory is the transformation of raw data assets into trusted information that can be shared broadly with BI and analytics tools.
The MSDN forum will be used for general discussions for Getting Started, Development, Management, and Troubleshooting using Azure Data Factory.
