Bug with Salesforce "Get Records"

Question

It appears that Logic Apps is unable to correctly query the Salesforce object called "EntityDefinition" .  The row count always comes out as 200 regardless of the real count of rows as confirmed by querying Salesforce using the SOQL query, "SELECT COUNT() FROM EntityDefinition".

Can someone please try to replicate the below steps in Logic Apps?

1. Recurrence -> any values.
2. Get Records -> "Entity Definitions".
3. Click Run.
4. Expand "Get Records" -> Click to Download -> Save as JSON -> perform a row count using your tool of choice, e.g., Power BI. The result will always equal 200.

FYI, when trying to query this object in Data Factory it stalls the pipeline indefinitely.

I suspect both issues stem from the fact that the [Id] field in this object is not a primary key which, if Logic Apps and Data Factory use PK Chunking, may cause them to exhibit this behavior.

Answer 1

Hi Simon,

About an year ago, 200 or 250 was the max no. of records Logic App can return with Salesforce. There was a limitation of 2000 or 2500 with Salesforce rest API. If there are more records, Salesforce REST API will return a continuation URL in the response, which can then be used to send the subsequent set of requests to pull the remaining records.

Obviously you can have more records in the salesforce objects. You have to implement recursive calls to your Logic App or a loop to fetch all the records. The query should use skip and to ODATA filters along with sorting by any field. If you want help on how to implement this, let me know.

Note: If this answers your current question, mark it as an answer. I have seen you ignore on previous occasions.

Thanking you,

Mohamed Ibrahim

Answer 2

Woah that's nuts!  The action should absolutely be paginating on nextRecordsUrl by default.  Why haven't they implemented this functionality?

Moreover, the batchSize is currently 2,000 in Salesforce (API v43.0). Why is the connector using 200?

Anyway, yes, I'd like to hear details on how you handled pagination using Logic Apps.  Thanks!