No announcements
-
0 VotesAutomatization of Spark Job in Azure
I've a class which make some extract, transform an load to a dataset located in a different JSON files. This process work Ok. But, I've the necessity to process manually ...
-
0 Votes
HDInsight - How to add new users in Hue?
We have installed Hue in our HDInsight cluster using the Script actions. Now we want to create new users in this Hue ...Answered | 4 Replies | 303 Views | Created by Raven Peaks - Wednesday, December 4, 2019 12:02 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Tuesday, December 10, 2019 4:57 AM -
0 Votes
Not able to use pip in hdinsight spark cluster to install packages.
I am connecting through putty to spark cluster. when i enter $python it takes me to python shell, that means python environment is set. But i am unable to use pip to install packages.Proposed | 3 Replies | 258 Views | Created by satyamt1997 - Tuesday, December 3, 2019 10:56 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Friday, December 6, 2019 10:07 AM -
0 Votes
Error ocuuring during unzipping a file on Blob storage: BlockCountExceedsLimit.
Hi all, How can I unzip a big file in blob storage? I have HDInsight spark cluster with a Azure Storage on it. I need to unzip a file which is 1TB after unzipping using the ...Proposed | 6 Replies | 366 Views | Created by Maryam_Lewen - Wednesday, November 27, 2019 12:10 PM | Last reply by MartinJaffer-MSFT - Thursday, December 5, 2019 7:30 PM -
0 Votes
failing initial step while adding HDInsight cluster(services)
dEAR all . i already raised this issue but i did not get any response so please help me for below ...Answered | 7 Replies | 710 Views | Created by HadoopHelp - Thursday, October 17, 2019 3:51 PM | Last reply by HadoopHelp - Thursday, December 5, 2019 10:23 AM -
0 Votes
JDBC Connection error : Unable to connect to hive on HDP Sandbox which is on Azure
Hi, I am trying to connect to Hive from Mulesoft application (on my local machine) using a JDBC ...Proposed | 3 Replies | 285 Views | Created by SAUMYA2712 - Monday, November 25, 2019 9:01 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Thursday, December 5, 2019 9:25 AM -
0 Votes
Big Json file analysis in Azure HDInsight Spark Cluster
I would like to analyze a big data (0.9 TB after unzipping) in a cluster with 14 nodes and 39 cores (Azure HDInsight/Spark). But it's very slow. Here what I ...Proposed | 3 Replies | 341 Views | Created by Maryam_Lewen - Wednesday, November 27, 2019 8:14 AM | Last reply by MartinJaffer-MSFT - Monday, December 2, 2019 10:58 PM -
0 Votes
connect spark and Hive Hdinsight
I have a cluster of Hdinsigth I am creating tables from hive, but when I see the tables from pypark I can not see them could you help me pleaseProposed | 9 Replies | 390 Views | Created by alexgv12 - Tuesday, November 12, 2019 1:21 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Friday, November 22, 2019 10:24 AM -
0 Votes
HDInsight Spark Network
Hi all, i have questions about network for hdinsight spark. I have a 10.72.24.0/24 vnet subnetted as following: prod ...Answered | 6 Replies | 467 Views | Created by Emanuele86 - Monday, November 4, 2019 8:38 PM | Last reply by Emanuele86 - Wednesday, November 20, 2019 9:23 AM -
0 Votes
HDInsight Guice Dependency Version
Currently trying to run a jar file on an On-Demand HDInsight (version 3.6). After adding functionality for class injection (Java 8, Guice 4.0) I am getting the following ...Unanswered | 4 Replies | 339 Views | Created by bwong9 - Wednesday, November 6, 2019 9:52 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Wednesday, November 20, 2019 6:10 AM -
0 Votes
HDInsight ESP Spark cluster 3.6
Can anyone please confirm the role of LLAP in ESP Spark 3.6. Does it only provides methodology to enforce policies for security and does not actually cache ...Proposed | 8 Replies | 493 Views | Created by Rahul092 - Monday, October 14, 2019 2:56 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Tuesday, November 12, 2019 6:27 AM -
0 Votes
HDInsight importing custom py file. addPyFile() missing 1 required positional argument: 'self'
I'm trying to import my own custom .py file into my HDInsight cluster on Azure. I've looked through a few different how-to documents but I can't seem to find anyone with my problem. Here is the code ...Answered | 2 Replies | 349 Views | Created by Justin Cherveny - Monday, November 11, 2019 11:14 PM | Last reply by CHEEKATLAPRADEEP-MSFT - Tuesday, November 12, 2019 4:48 AM -
0 Votes
How to run shell commands from Jupyter Notebook?
Is there any way to run shell commands such as conda from Jupyter Notebook in Azure HDInsight? ThanksProposed | 6 Replies | 433 Views | Created by Maryam_Lewen - Thursday, October 10, 2019 8:23 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Tuesday, November 5, 2019 6:29 AM -
0 Votes
Why is an empty file with the name of folder inside a Azure Blob storage container is created?
Hi All, I am running a Hive QL through HD Insight on demand cluster which does the following 1) spool the data from a hive view 2) Create a ...Proposed | 7 Replies | 1198 Views | Created by Surya Seshagiri - Thursday, October 18, 2018 11:04 AM | Last reply by MartinJaffer-MSFT - Thursday, October 31, 2019 9:51 PM -
0 Votes
Spark application build in local server, how can we access Hive Warehouse located in HDInsight
Hello Techie, This is my first project on HDInsight I'm trying to connect to the Hive warehouse directory located in HDInsight by using Spark on ...Answered | 4 Replies | 336 Views | Created by Jamiechales - Friday, October 25, 2019 7:33 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Wednesday, October 30, 2019 8:52 AM -
0 Votes
HBase and Phoenix: the data queried for the first time is very slow
Hi! We have some readings in a table (20M reads) with 12 columns (most of them numbers, so it is not very big). The table is created with phoenix and its key is ...Unanswered | 4 Replies | 990 Views | Created by Pablo Castilla - Monday, February 27, 2017 10:04 AM | Last reply by KranthiPakala-MSFT - Friday, October 25, 2019 10:39 PM -
0 Votes
Failure creating HDInsight Cluster
I've been doing some testing using HDInsight clusters and have been able to create clusters until today, I keep on getting the following ...Proposed | 3 Replies | 376 Views | Created by Jenkins102 - Monday, October 21, 2019 4:38 PM | Last reply by MartinJaffer-MSFT - Wednesday, October 23, 2019 5:44 PM -
0 Votes
How to install latest anaconda in the clusters?
Hi all, In my cluster, there is an old version of anaconda which works with python 2. Can I install new version of anaconda? ThanksUnanswered | 3 Replies | 422 Views | Created by Maryam_Lewen - Thursday, October 10, 2019 2:47 PM | Last reply by MartinJaffer-MSFT - Wednesday, October 16, 2019 6:30 PM -
0 Votes
sc.parralelize() makes RDD but not sc.textFile('....txt') / Azure HDInsight
I am using pyspark on Azure HDInsight. I am able to make an RDD with parallelize and do action on it, but with <lines = sc.textFile("test.txt") > ...Proposed | 3 Replies | 406 Views | Created by Maryam_Lewen - Thursday, October 10, 2019 5:08 AM | Last reply by CHEEKATLAPRADEEP-MSFT - Monday, October 14, 2019 8:25 AM -
0 Votes
HDInsight - Spark: Installing extra python dependencies using Conda hangs on: "Collecting package metadata (repodata.json)"
We are running a HDInsight 4.0 cluster on Azure. It's a test set-up with just 1 worker node (8 CPU, 56 GB ram) and 2 headnodes (4 CPU, 25 GB ram each). We need to add a few specific python ...Proposed | 6 Replies | 590 Views | Created by Tom Pauwaert - Thursday, September 26, 2019 10:16 AM | Last reply by Tom Pauwaert - Monday, October 7, 2019 7:50 PM - Items 1 to 20 of 1746 Next ›
No announcements
