Azure Data Factory announcement
-
Link
We have noticed customers post their questions on Azure Databricks on incorrect forums on MSDN, since there is a new home for Azure Databricks.
Forum Link: https://social.msdn.microsoft.com/Forums/en-US/home?forum=AzureDatabricks
-
Link
Azure Data Factory allows you to manage the production of trusted information by offering an easy way to create, orchestrate, and monitor data pipelines over the Hadoop ecosystem using structured, semi-structures and unstructured data sources.
You can connect to your on-premises SQL Server, Azure database, tables or blobs and create data pipelines that will process the data with Hive and Pig scripting, or custom C# processing. The service offers a holistic monitoring and management experience over these pipelines, including a view of their data production and data lineage down to the source systems.
The outcome of Data Factory is the transformation of raw data assets into trusted information that can be shared broadly with BI and analytics tools.
The MSDN forum will be used for general discussions for Getting Started, Development, Management, and Troubleshooting using Azure Data Factory.
-
1 VotesAzure Databricks forum is available on the MSDN forum
We have noticed customers post their questions on Azure Databricks on incorrect forums on MSDN, since there is a new home for Azure Databricks. MSDN link: ...Sticky | 0 Replies | 855 Views | Created by CHEEKATLAPRADEEP-MSFT - Friday, October 25, 2019 9:32 AM
-
0 Votes
How to perform a post request using a file as body in ADF
Hi all, What I want to: I want to read a parquet file in Data Lake (gen2) And I can make some changes on it (i.e. fill the value ...Proposed | 2 Replies | 138 Views | Created by Satellitef - Tuesday, December 24, 2019 8:37 PM | Last reply by Satellitef - 14 hours 41 minutes ago -
0 Votes
Json file issue
jsonfiles DF-SRC-006 - Missing column 'size - source( output( fruit as string, size as string, color as string ), store: 'blob', format: 'json', singleDocument: false, singleQuoted: false, ... -
0 Votes
Copy Activity Performance and Scalability in ADFV2
Hi Team, my Database (on-premise SQL Server) size is 50TB (terrabytes) and i want to copy the data from on-premise SQL Server To Azure Data Lake Store(ADLS) by using Copy ... -
0 Votes
Data consuming by Using REST API
Hi ALL, How can i consume the data in Azure data factory by using REST API(GET,POST,PUT) methods.Please guide me if anyone has idea on that and also can share the blogs,tutorial ... -
0 Votes
Pagination in Azure data Factory(By REST API)
Hi Team, What is Pagination in Azure data factory.?How can i perform this activity using of REST API.Could you please guide me on this? Can you please share the ... -
0 Votes
get the Azure pipeline start, end and duration time by using python/powershell scripts
Hi All, I am new to Azure Data factory. I should get the Start, End and duration time for the existing pipelines. It's running on daily basis. I should generate the report by using ...Unanswered | 0 Replies | 43 Views | Created by Arun Kumar Durairaj - Friday, December 27, 2019 1:31 PM -
0 Votes
Unable to Connect to Citus DB from Azure datafactory
While I am trying to create a connection in azure datafactory for azure hyperscale (Citus) DB , I encountered below error. The value of the property '' is ... -
0 Votes
Azure Data Factory failure alerts are not sending emails to Distribution lists
I have an Azure Data Factory (v2) setup with Azure Alerts for Failure Metrics. Email notification has been set for failure alerts to both distribution list and an individual email id. Emails are NOT ...Unanswered | 0 Replies | 61 Views | Created by SwaminathanVetri - Friday, December 27, 2019 10:05 AM -
0 Votes
Issue while copying data from http source to blob using parameterized ADF
Hi, Please find the error below and assist accordingly. { "errorCode": "2200", ...Proposed | 1 Replies | 141 Views | Created by BikashADF - Tuesday, December 24, 2019 1:44 PM | Last reply by SumanthMarigowda-MSFT - Friday, December 27, 2019 8:43 AM -
0 Votes
Send Email using Azure function without using third party tool
Hello Techies, I am trying to send email using Azure function and I wrote code (as below) for it and it is working fine locally. But when I am running function on ...Proposed | 6 Replies | 105 Views | Created by Ramandeep123 - Tuesday, December 24, 2019 7:03 AM | Last reply by Ramandeep123 - Thursday, December 26, 2019 10:51 AM -
0 Votes
Pulling data from Sharepoint list using ADF V2
Hello All, Can anyone help me understand how to connect to Sharepoint list using ADF V2. I have found couple of solution like using OData, ...Unanswered | 2 Replies | 117 Views | Created by Chayan Upadhyay27 - Monday, December 23, 2019 2:41 PM | Last reply by Chayan Upadhyay27 - Thursday, December 26, 2019 10:35 AM -
0 Votes
"errorCode": "2200" "message": "Failure happened on 'Source' side. path name display with "\\"
Dear I am trying to load data from SQL server to windows data lake , I got the following error ...Proposed | 1 Replies | 74 Views | Created by Ghassan Abdel jawad - Wednesday, December 25, 2019 9:49 PM | Last reply by ChiragMishra-MSFT - Thursday, December 26, 2019 8:26 AM -
0 Votes
how we read data from REST API Collecation in Azure Data Factory
Hi All, i Have try to read sample API-https://gorest.co.in/public-api/users inside this API we have collection- but using REST link service i create copy ...Proposed | 1 Replies | 64 Views | Created by manish0000 - Tuesday, December 24, 2019 11:59 AM | Last reply by ChiragMishra-MSFT - Thursday, December 26, 2019 6:15 AM -
0 Votes
Using wildcards in datasets and get metadata activities
I remember you could use wildcards in datasets, but now that functionality doesn't seem to work any more. The problem I have at the moment is my pipeline fails if some CSV files are ...Proposed | 4 Replies | 167 Views | Created by D2Otter - Wednesday, December 18, 2019 6:34 PM | Last reply by ChiragMishra-MSFT - Thursday, December 26, 2019 5:50 AM -
0 Votes
Inner Pipeline Activity Outputs
IF ==> Inner Pipeline 1 ==> Foreach ==> Inner Pipeline 2 ==> IF ===> Activities. So I need to get activity output information from ...Proposed | 4 Replies | 195 Views | Created by ipulsetv - Tuesday, December 17, 2019 7:20 PM | Last reply by ChiragMishra-MSFT - Thursday, December 26, 2019 5:35 AM -
3 Votes
ADFv2 - Custom Activity - pass parameter - Stored Procedure Activity
I am working on a pipeline in Azure Data Factory V2. I want to pass a string from a custom batch app activity to a stored procedure activity via the input parameter of the stored ...Answered | 10 Replies | 2126 Views | Created by Polly Herr - Thursday, April 26, 2018 9:15 PM | Last reply by ChiragMishra-MSFT - Thursday, December 26, 2019 4:14 AM -
0 Votes
Delete temporary table from SQL database using Stored procedure
Hi, Its rather a very simple question but I am not able to make it work. I am inserting data from a temporary table in SQL DB to a Main table in DB ...Unanswered | 4 Replies | 121 Views | Created by zzzsharepoint - Tuesday, December 24, 2019 1:49 PM | Last reply by Vaibhav-Chaudhari - Wednesday, December 25, 2019 6:17 PM -
0 Votes
Can't get lookup data passed to Oracle Database
TO_DATE(@activity('LookupLastDate').output.FirstRow.ActivityDate, 'YYYY-MM-DD') But I get the errormessage: Activity Copy from Orcale to Datalake failed: Failure ...Answered | 6 Replies | 380 Views | Created by jbazelmans - Monday, September 30, 2019 6:51 AM | Last reply by Vaibhav-Chaudhari - Tuesday, December 24, 2019 2:08 PM -
0 Votes
Running another data factory jobs?
How to trigger the another data factory jobs? Under one subscription we have multiple resources and each resource owned by different departments contain their own datafactory jobs. Now i want to build ...Unanswered | 6 Replies | 282 Views | Created by Baba9 - Friday, December 13, 2019 11:59 AM | Last reply by Baba9 - Tuesday, December 24, 2019 10:52 AM -
0 Votes
Azure Datafactory - Get Metadata activity is returned maximum size of metadata is 1 MB.
Hello , We have many large number of files are schedule to upload into azure blob storage and Get Metadata activity is used to fetch meta data and passed it to foreach ...Proposed | 3 Replies | 168 Views | Created by Nani S - Thursday, December 19, 2019 7:31 AM | Last reply by ChiragMishra-MSFT - Tuesday, December 24, 2019 8:22 AM - Items 1 to 20 of 3182 Next ›
Azure Data Factory announcement
-
Link
We have noticed customers post their questions on Azure Databricks on incorrect forums on MSDN, since there is a new home for Azure Databricks.
Forum Link: https://social.msdn.microsoft.com/Forums/en-US/home?forum=AzureDatabricks
-
Link
Azure Data Factory allows you to manage the production of trusted information by offering an easy way to create, orchestrate, and monitor data pipelines over the Hadoop ecosystem using structured, semi-structures and unstructured data sources.
You can connect to your on-premises SQL Server, Azure database, tables or blobs and create data pipelines that will process the data with Hive and Pig scripting, or custom C# processing. The service offers a holistic monitoring and management experience over these pipelines, including a view of their data production and data lineage down to the source systems.
The outcome of Data Factory is the transformation of raw data assets into trusted information that can be shared broadly with BI and analytics tools.
The MSDN forum will be used for general discussions for Getting Started, Development, Management, and Troubleshooting using Azure Data Factory.
