none
Bash Script URL from repository? RRS feed

  • Question

  • I'm trying to use the python library "holidays" in my pyspark 3 kernel jupyter notebook in HDInsight.

    I'm following the instructions in the documentation to install custom packages using script actions.

    The example has a url for tensorflow, but I'm not sure how to find the url for my package, holidays. 

    I found the library in the maven repository, but I'm not sure what to put into the bash script url box. 

    Thanks in advance!

    Friday, August 9, 2019 7:13 PM

All replies

  • Hi Justin,

    Note: To install custom script actions, URI should be located in Azure Blob Storage, ADLS Gen1, Github and OneDrive.

    A script action is Bash script that runs on the nodes in an HDInsight cluster. Characteristics and features of script actions are as follows:

    Must be stored on a URI that's accessible from the HDInsight cluster. The following are possible storage locations:

    For regular clusters:

    ADLS Gen1: The service principal HDInsight uses to access Data Lake Storage must have read access to the script. The URI format for scripts stored in Data Lake Storage Gen1 isadl://DATALAKESTOREACCOUNTNAME.azuredatalakestore.net/path_to_file.

    A blob in an Azure Storage account that's either the primary or additional storage account for the HDInsight cluster. HDInsight is granted access to both of these types of storage accounts during cluster creation.

    A public file-sharing service accessible through http:// paths. Examples are Azure Blob, GitHub, OneDrive.

    For example URIs, see Example script action scripts.

    For clusters with ESP:

    The wasb:// or wasbs:// or http[s]:// URIs are supported.

    For more details, refer “Customize Azure HDInsight clusters by using script actions”.

    This article helps, how to use Script Actions to configure an Apache Spark cluster on HDInsight to use external, community-contributed python packages that are not included out-of-the-box in the cluster.

    Hope this helps.

    Monday, August 12, 2019 4:40 AM
    Moderator
  • Hi Justin,

    Just checking in to see if the above answer helped. If this answers your query, do click “Mark as Answer” and Up-Vote for the same. And, if you have any further query do let us know.

    Wednesday, August 14, 2019 6:26 AM
    Moderator
  • Hi Justin,

    Just checking in to see if the above answer helped. If this answers your query, do click “Mark as Answer” and Up-Vote for the same. And, if you have any further query do let us know.


    Wednesday, August 21, 2019 6:21 AM
    Moderator