locked
Missing columns, when copy data from ServiceNow using Azure Data Factory RRS feed

  • Question

  • Team, I'm new to ADF and I'm trying to pull the data from ServiceNow using the ServiceNow connector in ADF. I'm pulling the Display.Cost_Plan table, but I'm only getting one column and if I use the REST API and pull the data for the same table I'm getting all the columns.

    If I try to pull the Display.sys_user table using the ServiceNow connector, I'm getting all the columns.

    I don't understand why I'm not able to pull all the columns for cost_plan table but for sys_user table.

    Can someone please help me with the same?

    Thanks!

    Thursday, October 10, 2019 3:59 PM

All replies

  • Hello

    Thanks for the query and to my understanding the ask here is  why Display.Cost_Plan table is returning only one column. Can you let me know as to what happens when you try to query Actual.Cost_Plan?


    Thanks Himanshu

    Friday, October 11, 2019 11:01 PM
  • Yes I have noticed this as well, I believe the issue is to do with the ServiceNow Connector and this was built by Microsoft not by Servicenow. Your best bet is to use the REST Connector as like you said you can bring back both Actual and Display values. Also if you just use the Display Schema and you needed to filter on a table it will bring back all the results first then filter afterwards, this would take ages if you were looking at a big table.
    Sunday, October 13, 2019 6:07 PM
  • Hi @AVPowerBI, Thanks for your response. If I use REST API, I'm not able it pull the entire table in one go because there is a row limit right.
    Monday, October 14, 2019 5:01 AM
  • Hi @Himanshu, Thanks for your response.

    Same for both Actual.cost_plan and the Display.cost_plan tables. Both are pulling only one column.

    Monday, October 14, 2019 5:03 AM
  • Quick question : Are you using a Self-hosted IR , if yes , can you please send the logs at  AzCommunity@microsoft.com .

    How to collect driver logs

    1. Set up a self-hosted IR and enable logging on it:

     

    Change the content of all .ini files under path <IR Installation Path>\Shared\ODBC Drivers\Microsoft <Data Source> ODBC Driver\lib\ to below to enable logging:

    [Driver]

    ErrorMessagesPath=..\ErrorMessages

    LogLevel=6

    LogPath=<Log Path>

    The <Log Path> is an absolute path to a folder on the machine where IR is installed, for example: D:\log.

     

    Note: please do not use a path under C:\ drive since the IR may not have write permission.

     

    1. After logging is enabled, please reproduce the issue and IR logs will be created under <Log Path>.
    2. Disable IR logging by changing back the content of the .ini files.



    Thanks Himanshu

    Monday, October 14, 2019 6:31 PM
  • Hello , 

    We have not heard back from you on this . Please to share with the community if you have a resolution or please share the details which we have asked so we can take it forward .

    Thanks 

    Himanshu


    Thanks Himanshu

    Wednesday, October 23, 2019 11:10 PM