No announcements
-
0 VotesHow to install latest anaconda in the clusters?
Hi all, In my cluster, there is an old version of anaconda which works with python 2. Can I install new version of anaconda? ThanksUnanswered | 3 Replies | 443 Views | Created by Maryam_Lewen - Thursday, October 10, 2019 2:47 PM | Last reply by MartinJaffer-MSFT - Wednesday, October 16, 2019 6:30 PM
-
0 Votes
sc.parralelize() makes RDD but not sc.textFile('....txt') / Azure HDInsight
I am using pyspark on Azure HDInsight. I am able to make an RDD with parallelize and do action on it, but with <lines = sc.textFile("test.txt") > ...Proposed | 3 Replies | 426 Views | Created by Maryam_Lewen - Thursday, October 10, 2019 5:08 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Monday, October 14, 2019 8:25 AM -
0 Votes
HDInsight - Spark: Installing extra python dependencies using Conda hangs on: "Collecting package metadata (repodata.json)"
We are running a HDInsight 4.0 cluster on Azure. It's a test set-up with just 1 worker node (8 CPU, 56 GB ram) and 2 headnodes (4 CPU, 25 GB ram each). We need to add a few specific python ...Proposed | 6 Replies | 609 Views | Created by Tom Pauwaert - Thursday, September 26, 2019 10:16 AM | Last reply by Tom Pauwaert - Monday, October 7, 2019 7:50 PM -
0 Votes
API for check script action error message
[note: I changed the original question to make it more clear] Hi all, We are using the c# to execute script action(bash file) to ...Unanswered | 3 Replies | 458 Views | Created by davidyang2000 - Monday, September 30, 2019 10:30 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Friday, October 4, 2019 4:55 AM -
2 Votes
Failing while creating Hdinsinght
I am trying to spinup an Hdinsight using the powershell and passing a blob storage along with it ...Proposed | 7 Replies | 474 Views | Created by Anmol Alawadhi - Tuesday, September 24, 2019 9:02 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Tuesday, October 1, 2019 7:24 AM -
0 Votes
question about install packages on HDInsight spark cluster(linux)
Hi all, I am trying to build up spark cluster so that our ML team could use ADF to submit spark activity. I run into some configuration issue and hope any of ...Answered | 5 Replies | 538 Views | Created by davidyang2000 - Friday, September 6, 2019 5:49 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Tuesday, October 1, 2019 4:36 AM -
0 Votes
Problem connecting to RStudio Server through HDInsight cluster
Hello, I have a trial Azure subscription. I was able to successfully create an HDInsight ML Services (HDI 3.6) cluster. The cluster is shown as running and I can connect to its URL. ...Proposed | 5 Replies | 365 Views | Created by ccm22 - Sunday, September 22, 2019 5:16 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Friday, September 27, 2019 6:14 AM -
4 Votes
Dremio access to HDInsight cluster and Hive
We are exploring the option to access a hive table or underlying data in hdfs on HDInsight cluster on Azure from Dremio client. We are currently able to access the other resources like MSSQLServer, ...Answered | 7 Replies | 470 Views | Created by azdevad1 - Monday, September 9, 2019 3:04 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Wednesday, September 18, 2019 4:21 AM -
0 Votes
LLAP Interactive Query Cluster Slow Query Performance
Hi, We have HDInsight with interactive query running on our environment. The users reported that their queries running slow. We have tried to ...Answered | 8 Replies | 613 Views | Created by Freddy.Setiawan - Wednesday, August 14, 2019 2:21 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Tuesday, September 17, 2019 8:05 AM -
1 Votes
Query tables created in Hive using Spark in HDInsight
I have created a Hadoop cluster and loaded some tables to hive. Now I have created a spark cluster and wish to see if we can query the tables from there. Could you please let me know if there is a ...Proposed | 4 Replies | 418 Views | Created by azdevad1 - Monday, September 9, 2019 5:37 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Friday, September 13, 2019 9:49 AM -
0 Votes
Getting Operation timed out when trying to send event to Event Hub using Java client 3.0.0
Hi Team, I am trying to send an event to Event Hub using below code ...Proposed | 4 Replies | 430 Views | Created by Orion_123 - Tuesday, September 10, 2019 6:48 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Friday, September 13, 2019 9:49 AM -
0 Votes
Which Azure Data Factory resource to use with HDInsights Jupyter Notebook (Pyspark 3)
My team and I are trying to configure an azure data factory (ADF) pipeline that runs some code I wrote for an HDInsight job. The code runs just fine when I open the Jupyter Notebook in my HDInsight ...Proposed | 3 Replies | 356 Views | Created by Justin Cherveny - Tuesday, September 3, 2019 5:50 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Tuesday, September 10, 2019 4:23 AM -
0 Votes
JAVA API for getting information about a given HDInsight
az hdinsight show --resource-group <resource group> --name <cluster name> I am using Oracle java ...Proposed | 3 Replies | 319 Views | Created by Kiran_81 - Thursday, September 5, 2019 4:57 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Tuesday, September 10, 2019 4:21 AM -
0 Votes
How to run shell script in Azure datafactory V2?
Is there a way or work around to run a shell script in ADF v2?Proposed | 3 Replies | 577 Views | Created by nvbala.vignesh - Tuesday, September 3, 2019 9:05 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Friday, September 6, 2019 9:11 AM -
0 Votes
HDInsight with Datalake Gen2 storage
I have created HDInsight Hadoop cluster with underlying storage account as Blob storage and was able to successfully prove out a few concepts of creating tables, loading data from ...Proposed | 4 Replies | 373 Views | Created by azdevad1 - Tuesday, August 27, 2019 5:00 PM | Last reply by MartinJaffer-MSFT - Friday, August 30, 2019 5:29 PM -
0 Votes
Submitting pyspark job through livy using user defined parameters
Below is the folder structure that we follow on the wasb. main.py jobs.zip jobs ...Proposed | 5 Replies | 1232 Views | Created by learningmind - Monday, January 29, 2018 6:52 PM | Last reply by JelenaLazar - Wednesday, August 28, 2019 4:39 PM -
0 Votes
Question about HDInsight IO cache
Hi HDInsight team! We here evaluating running Spark on HDInsight with data cache, and I went across this great article introducing HDInsight IO cache ...Proposed | 4 Replies | 415 Views | Created by Chen71 - Thursday, August 15, 2019 10:26 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Wednesday, August 28, 2019 10:18 AM -
0 Votes
connecting via ssh client
ssh admin@Kafkaepam-ssh.azurehdinsight.net i'm getting access denied error while connecting to the HDInsight clusterAnswered | 4 Replies | 384 Views | Created by manjerekar rao - Tuesday, August 20, 2019 2:46 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Tuesday, August 27, 2019 8:26 AM -
0 Votes
Difference in HDInsight clusters - Insight requested
We have multiple options available for the HDInsight clusters. I am aware of the below documentation but still need some insights. For example, creating a Hadoop ...Answered | 1 Replies | 288 Views | Created by azdevad1 - Monday, August 26, 2019 4:46 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Tuesday, August 27, 2019 5:24 AM -
1 Votes
Access denied when connecting to HDIssight cluster using Putty
Hi, I have created a HBase cluster in HDInsight. When I tried to connect to cluster using putty from my windows machine, it gives access denied ...Answered | 5 Replies | 1673 Views | Created by Aswathy Raveendran - Friday, October 20, 2017 7:14 AM | Last reply by HimanshuSinha-msft - Monday, August 26, 2019 6:54 PM
No announcements
