No announcements
-
0 VotesHow to install latest anaconda in the clusters?
Hi all, In my cluster, there is an old version of anaconda which works with python 2. Can I install new version of anaconda? ThanksUnanswered | 3 Replies | 84 Views | Created by Maryam_Lewen - Thursday, October 10, 2019 2:47 PM | Last reply by MartinJaffer-MSFT - 4 hours 56 minutes ago
-
0 Votes
HDInsight ESP Spark cluster 3.6
Can anyone please confirm the role of LLAP in ESP Spark 3.6. Does it only provides methodology to enforce policies for security and does not actually cache ...Proposed | 4 Replies | 69 Views | Created by Rahul092 - Monday, October 14, 2019 2:56 PM | Last reply by Rahul092 - 10 hours 25 minutes ago -
0 Votes
sc.parralelize() makes RDD but not sc.textFile('....txt') / Azure HDInsight
I am using pyspark on Azure HDInsight. I am able to make an RDD with parallelize and do action on it, but with <lines = sc.textFile("test.txt") > ...Proposed | 3 Replies | 96 Views | Created by Maryam_Lewen - Thursday, October 10, 2019 5:08 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Monday, October 14, 2019 8:25 AM -
0 Votes
How to run shell commands from Jupyter Notebook?
Is there any way to run shell commands such as conda from Jupyter Notebook in Azure HDInsight? ThanksUnanswered | 5 Replies | 109 Views | Created by Maryam_Lewen - Thursday, October 10, 2019 8:23 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Monday, October 14, 2019 8:24 AM -
0 Votes
HDInsight - Spark: Installing extra python dependencies using Conda hangs on: "Collecting package metadata (repodata.json)"
We are running a HDInsight 4.0 cluster on Azure. It's a test set-up with just 1 worker node (8 CPU, 56 GB ram) and 2 headnodes (4 CPU, 25 GB ram each). We need to add a few specific python ...Proposed | 6 Replies | 251 Views | Created by Tom Pauwaert - Thursday, September 26, 2019 10:16 AM | Last reply by Tom Pauwaert - Monday, October 7, 2019 7:50 PM -
0 Votes
API for check script action error message
[note: I changed the original question to make it more clear] Hi all, We are using the c# to execute script action(bash file) to ...Unanswered | 3 Replies | 159 Views | Created by davidyang2000 - Monday, September 30, 2019 10:30 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Friday, October 4, 2019 4:55 AM -
0 Votes
Why is an empty file with the name of folder inside a Azure Blob storage container is created?
Hi All, I am running a Hive QL through HD Insight on demand cluster which does the following 1) spool the data from a hive view 2) Create a ...Proposed | 5 Replies | 828 Views | Created by Surya Seshagiri - Thursday, October 18, 2018 11:04 AM | Last reply by HimanshuSinha-msft - Wednesday, October 2, 2019 12:58 AM -
2 Votes
Failing while creating Hdinsinght
I am trying to spinup an Hdinsight using the powershell and passing a blob storage along with it ...Proposed | 7 Replies | 213 Views | Created by Anmol Alawadhi - Tuesday, September 24, 2019 9:02 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Tuesday, October 1, 2019 7:24 AM -
0 Votes
question about install packages on HDInsight spark cluster(linux)
Hi all, I am trying to build up spark cluster so that our ML team could use ADF to submit spark activity. I run into some configuration issue and hope any of ...Answered | 5 Replies | 309 Views | Created by davidyang2000 - Friday, September 6, 2019 5:49 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Tuesday, October 1, 2019 4:36 AM -
0 Votes
Problem connecting to RStudio Server through HDInsight cluster
Hello, I have a trial Azure subscription. I was able to successfully create an HDInsight ML Services (HDI 3.6) cluster. The cluster is shown as running and I can connect to its URL. ...Proposed | 5 Replies | 185 Views | Created by ccm22 - Sunday, September 22, 2019 5:16 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Friday, September 27, 2019 6:14 AM -
4 Votes
Dremio access to HDInsight cluster and Hive
We are exploring the option to access a hive table or underlying data in hdfs on HDInsight cluster on Azure from Dremio client. We are currently able to access the other resources like MSSQLServer, ...Answered | 7 Replies | 281 Views | Created by azdevad1 - Monday, September 9, 2019 3:04 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Wednesday, September 18, 2019 4:21 AM -
0 Votes
LLAP Interactive Query Cluster Slow Query Performance
Hi, We have HDInsight with interactive query running on our environment. The users reported that their queries running slow. We have tried to ...Answered | 8 Replies | 403 Views | Created by Freddy.Setiawan - Wednesday, August 14, 2019 2:21 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Tuesday, September 17, 2019 8:05 AM -
1 Votes
Query tables created in Hive using Spark in HDInsight
I have created a Hadoop cluster and loaded some tables to hive. Now I have created a spark cluster and wish to see if we can query the tables from there. Could you please let me know if there is a ...Proposed | 4 Replies | 230 Views | Created by azdevad1 - Monday, September 9, 2019 5:37 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Friday, September 13, 2019 9:49 AM -
0 Votes
Getting Operation timed out when trying to send event to Event Hub using Java client 3.0.0
Hi Team, I am trying to send an event to Event Hub using below code ...Proposed | 4 Replies | 232 Views | Created by Orion_123 - Tuesday, September 10, 2019 6:48 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Friday, September 13, 2019 9:49 AM -
0 Votes
Which Azure Data Factory resource to use with HDInsights Jupyter Notebook (Pyspark 3)
My team and I are trying to configure an azure data factory (ADF) pipeline that runs some code I wrote for an HDInsight job. The code runs just fine when I open the Jupyter Notebook in my HDInsight ...Proposed | 3 Replies | 209 Views | Created by Justin Cherveny - Tuesday, September 3, 2019 5:50 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Tuesday, September 10, 2019 4:23 AM -
0 Votes
JAVA API for getting information about a given HDInsight
az hdinsight show --resource-group <resource group> --name <cluster name> I am using Oracle java ...Proposed | 3 Replies | 202 Views | Created by Kiran_81 - Thursday, September 5, 2019 4:57 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Tuesday, September 10, 2019 4:21 AM -
0 Votes
How to run shell script in Azure datafactory V2?
Is there a way or work around to run a shell script in ADF v2?Proposed | 3 Replies | 266 Views | Created by nvbala.vignesh - Tuesday, September 3, 2019 9:05 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Friday, September 6, 2019 9:11 AM -
0 Votes
HDInsight with Datalake Gen2 storage
I have created HDInsight Hadoop cluster with underlying storage account as Blob storage and was able to successfully prove out a few concepts of creating tables, loading data from ...Proposed | 4 Replies | 264 Views | Created by azdevad1 - Tuesday, August 27, 2019 5:00 PM | Last reply by MartinJaffer-MSFT - Friday, August 30, 2019 5:29 PM -
0 Votes
Submitting pyspark job through livy using user defined parameters
Below is the folder structure that we follow on the wasb. main.py jobs.zip jobs ...Proposed | 5 Replies | 978 Views | Created by learningmind - Monday, January 29, 2018 6:52 PM | Last reply by JelenaLazar - Wednesday, August 28, 2019 4:39 PM -
0 Votes
Question about HDInsight IO cache
Hi HDInsight team! We here evaluating running Spark on HDInsight with data cache, and I went across this great article introducing HDInsight IO cache ...Proposed | 4 Replies | 315 Views | Created by Chen71 - Thursday, August 15, 2019 10:26 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Wednesday, August 28, 2019 10:18 AM - Items 1 to 20 of 1733 Next ›
No announcements
