none
Triggering a Data Factory Pipeline through javascript

    Question

  • I have made this pipeline in Azure Data Factory that copies data from a Azure Storage Table to a Azure SQL database Table.

    The Azure storage table is given data from a Javascript chatbot that records answers and stores them in the table. I would like to trigger the CopyTabletoSQL through my javascript app once all of the answers have been recorded.

    If I look at the documentation it only mentions .net, Powershell, REST API and Python SDK but nothing for JavaScript

    Thank you kindly for any help.

    Thursday, July 12, 2018 12:46 PM

All replies

  • I think You could call Rest API with JavaScript.
    Thursday, July 12, 2018 1:25 PM
  • Hello, this seems to be the case, I am trying to use this guide https://docs.microsoft.com/en-us/azure/data-factory/concepts-pipeline-execution-triggers 

    I am very inexperienced with Rest API and JavaScript however I do have the corresponding JSON body and URL required for the Rest API, however I have no idea where to even begin to find out how to do the Rest API post through javascript.

    I've heard that I could do something like

    Return http.post(URL,BODY);

    I would love to learn more about the syntax on how to call the Rest API with JavaScript

    Edit: I have managed to use https://docs.microsoft.com/en-us/rest/api/datafactory/pipelines/createrun

    to create a run and try it and it was succesfull. 

    POST -URL-
    Authorization: Bearer -random symbols-
    Content-type: application/json

    How would I go about implementing this, or learning how to implement this, into my javascript app?

    many thanks!

    Edit 2:

    I found this :

    var url = "-myurl-";
    var method = "POST";
    var postData = "Some data";
    
    // You REALLY want shouldBeAsync = true.
    // Otherwise, it'll block ALL execution waiting for server response.
    var shouldBeAsync = true;
    var XMLHttpRequest = require("xmlhttprequest").XMLHttpRequest;
    var request = new XMLHttpRequest();
    
    // Before we send anything, we first have to say what we will do when the
    // server responds. This seems backwards (say how we'll respond before we send
    // the request? huh?), but that's how Javascript works.
    // This function attached to the XMLHttpRequest "onload" property specifies how
    // the HTTP response will be handled. 
    request.onload = function () {
    
       // Because of javascript's fabulous closure concept, the XMLHttpRequest "request"
       // object declared above is available in this function even though this function
       // executes long after the request is sent and long after this function is
       // instantiated. This fact is CRUCIAL to the workings of XHR in ordinary
       // applications.
    
       // You can get all kinds of information about the HTTP response.
       var status = request.status; // HTTP response status, e.g., 200 for "200 OK"
       var data = request.responseText; // Returned data, e.g., an HTML document.
    }
    
    request.open(method, url, shouldBeAsync);
    
    request.setRequestHeader("Content-Type", "application/json;charset=UTF-8");
    // Or... request.setRequestHeader("Content-Type", "text/plain;charset=UTF-8");
    // Or... whatever
    
    // Actually sends the request to the server.
    request.send(postData)

    Everything seems to be fine however, I am not sure what to write in 

    var postData = "Some data";

    I'm guessing it is somehow related to the JSON body?

    also would I need to add 

    Authorization: Bearer -random symbols-

    somehow?

    Many thanks!

    • Edited by bergbergen Friday, July 13, 2018 8:58 AM
    Friday, July 13, 2018 8:34 AM
  • 1. If you are familiar with Postman (A Chrome Extension), I would suggest you to use Postman to sent request manually first. It will help you understand what is required for the rest api, including url, body, token and headers

    2. In ADF V2 UI, you could press F12 and then click "Trigger". In network tab, you will find the create run api call.

     

    3. Then everything remain for your java script app is how to get the "authrization token".  You can't hard code it because it will expire in an hour. I saw you also asked this question in stack overflow and Jay has already replied you. Hope you could figure it out.


    Friday, July 13, 2018 9:39 AM
  • Hello, thank you for your reply! Very extensive!

    I have this so far: https://pastebin.com/urtabdvL , I replaced the request URL with "my-url", I'm not sure if I need to keep it secret or not.

    Jay on stackoverflow told me how to get the authorization token. What I'm wondering currently is what to add in the "Some data" variable. I'm guessing it is the JSON body however how would I convert that to add it to Javascript?

    var postData = "Some data";

    Also I am wondering how to implement the Authorization token once I have it? 

    I have no prior knowledge of javascript so am working mostly from the Microsoft documentation and whatever code examples / stackoverflow topics I can find/understand which means I have trouble with the syntax of using Rest API in javascript.


     Edit: I saw you replied to my Stackoverflow aswell Thanks :)
    • Edited by bergbergen Friday, July 13, 2018 11:03 AM
    Friday, July 13, 2018 10:36 AM
  • The "request payload" in my second screen shot should be the post data.
    Friday, July 13, 2018 12:31 PM
  • Seems there is no post data, maybe that is correct? It is only supposed to copy from a table to a SQL database


    I tried to get my auth token as Mr Jay Gong described on Stackoverflow using  ADAL nodejs sdk it seems I would need to register my app on a Azure Active Directory and I am working on a company azure account and would like to avoid registering anything on a Company Directory as I do not know what that entails.
    • Edited by bergbergen Friday, July 13, 2018 2:09 PM
    Friday, July 13, 2018 1:31 PM