How to customize the Jupyter notebook on HDInsight? RRS feed

  • Question

  • I want to use the spark-csv package to parse csv files in Spark Sql. This requires me to pass "--packages" to pyspark. It works fine if I SSH to the HDInsight cluster and launch the pyspark by myself. However, how can I get this working for the Jupyter notebook that installs with the HDInsight? I found that the IPython is not installed on the cluster. Where can I find the jupyter related settings?

    BTW: I am using the private preview Spark on Linux cluster.

    Sunday, December 20, 2015 7:47 AM

All replies