Getting An Error IN ADL pointing to graph too large and unable to process U-SQL


  • The following message is received.  

    Is this an issue with ADL or the Visual Studio?  I pulled this issue from from the ADLA jobs summary.

    E_COSMOS_USER_GENERIC: User Error: Abort job because the execution graph is too large. Please try to separate the job into some small jobs or use the U-SQL system variable like 'SET @@ExtractGroupDefaultDataSize' in the script to reduce the graph

    Thursday, June 28, 2018 8:26 PM

All replies

  • This is a server message telling you that you exceeded the number of supported vertices in your job. There can be many reasons for that: too complex script or too much data. If the script is too complex and the amount of data is not to blame, then splitting the script into several scripts is the way forward. If you operate on too much data, then you can try to reduce the number of vertices by having each extract vertex operate on more data (default for example for extracting from large CSV files is 1GB per vertex. So setting the @@ExtractGroupDefaultDataSize value to 2GB would half the number of extract stage vertices.

    Michael Rys

    Friday, June 29, 2018 12:13 AM