none
__spark_conf__.zip: No such file or directory

    Question

  • I am getting error

    wasb://cluster-2018-04-20t13-11-42-670z@demo.blob.core.windows.net/user/livy/.sparkStaging/application_1524230454528_0057/__spark_conf__.zip: No such file or directory

    while trying to run a spark job from Azure Data factory on HDInsight cluster. Can someone help me figure out what the issue is? I am stuck on this and could not proceed. As per ADF documentation, I tried placing my files in ./files folder also in the container where my spark jar is present. But still facing the same issue



    • Edited by drawat72 Saturday, April 21, 2018 3:30 PM
    Saturday, April 21, 2018 3:29 PM

All replies

  • I assume you run spark jar program, what's your HDI cluster version and the version of Java used to develop your program, please make sure the versions are the same.
    BTW, HDI cluster 3.5+ require java 8, while the earlier versions require java 7.
    Tuesday, April 24, 2018 10:04 AM