none
Unable to create Databricks cluster using free subscription account due to hidden clauses which are not clear from Azure team RRS feed

  • Question

  • I am unable to create Databricks cluster using free subscription account due to hidden clauses which are not clear from Azure team.

    If I follow the steps as stated here during cluster configuration I get an error saying i need to increase Quota.

    https://docs.microsoft.com/en-us/azure/azure-databricks/databricks-extract-load-sql-data-warehouse

    I have no idea on how to increase the  quota as Microsoft article says you can use free subscription account to accomplish this.


    Error details:

    TERMINATING
    ×
    Summary
    JSON
    Time
    2019-03-10 15:49:15 EDT
    Message
    Cluster terminated. Reason: Cloud Provider Launch Failure
    
    A cloud provider error was encountered while launching worker nodes. See the Databricks guide for more information.
    
    Azure error code: OperationNotAllowed
    
    Azure error message: Operation results in exceeding quota limits of Core. Maximum allowed: 4, Current in use: 4, Additional requested: 4. Please read more about quota increase at https://aka.ms/ProdportalCRP/?#create/Microsoft.Support/Parameters/{'subId':'dxxxxa-7xx5-4xx8-bxxb-axxxxxxxxxxx8','pesId':'15621','supportTopicId':'32447243'}.


     My ask is can we complete a Datbricks cluster as documented by Microsoft learning article.?
    Right now I am not able to achieve it .Will you provide me exact steps to configure as per the article ? 



    Monday, March 11, 2019 12:54 AM

All replies

  • Hello,

    Azure Free Trail has a limit of 4 cores, and you cannot use this tutorial using a Free Trial Subscription.

    To use this tutorial, you need to upgrade to Pay-As-You-Go subscription.

    Note: Free Trial subscriptions aren't eligible for limit or quota increases. If you have a Free Trial subscription, you can upgrade to a Pay-As-You-Go subscription.

    Hope this helps.

    Monday, March 11, 2019 4:55 AM
    Moderator
  • Thanks but looks like the learn article got amended 

    https://docs.microsoft.com/en-us/azure/azure-databricks/quickstart-create-databricks-workspace-portal.

    For evaluation purpose what would be the cost ? Say the requirement is as per the learning article and I want to know the cost .

    Also you need to ammend the article here -

    https://docs.microsoft.com/en-us/learn/modules/intro-to-azure-databricks/

    Introduction to Azure Databricks

    • 51 min
    • Module
    • 6 Units
    Intermediate
    Developer
    Solution Architect
    Azure
    Databricks

    Learn the basic fundamentals of Azure Databricks and Apache Spark notebooks.

    In this module, you will:

    • Create your own Azure Databricks workspace
    • Create a notebook inside your home folder in Databricks
    • Understand the fundamentals of Apache Spark notebook
    • Create, or attach to, a Spark cluster
    • Identify the types of tasks well suited to Apache Spark’s Unified Analytics Engine.

    Prerequisites:

    You will need an Azure subscription. If you don't have an Azure subscription, create a free account before you begin.



    Monday, March 11, 2019 6:25 PM
  • Hello,

    Thanks for the detailed feedback.

    From the same document, you can find the below note.

    To use a free account to create the Azure Databricks cluster, before creating the cluster, go to your profile and change your subscription to pay-as-you-go.

    And regarding the MS learn documentation, I’m working with the product team to make changes.

    Tuesday, March 12, 2019 9:15 AM
    Moderator
  • I don't find this answer to be clear at all. Right through MSLearn documentation, it mentions free learning paths. 

    https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RWtVsd mentions two free learning paths related to DataBricks. So they should be able to be done under the trial account. They both get stuck on the first module with this issue in not being able to create a cluster.

    This isn't a documentation bug, it is a bug in the inconsistency between available configurations and the trial quota. 


    • Edited by MelLondon Sunday, June 2, 2019 1:55 PM
    • Proposed as answer by Brian C Brown Tuesday, September 10, 2019 9:08 PM
    Sunday, June 2, 2019 1:55 PM
  • Are we to understand that, in order to use the free account, we must use some other account that is not free?
    Tuesday, September 10, 2019 9:08 PM
  • Hi Brian,

    Azure Free Trail has a limit of 4 cores, and you cannot use Azure Databricks using a Free Trial Subscription because to create spark cluster which requires more than 4 cores.

    If you have a free account, go to your profile and change your subscription to pay-as-you-go. For more information, see Azure free account. Then, remove the spending limit, and request a quota increase for vCPUs in your region. When you create your Azure Databricks workspace, you can select the Trial (Premium - 14-Days Free DBUs) pricing tier to give the workspace access to free Premium Azure Databricks DBUs for 14 days.

    Hope this helps.

    Wednesday, September 11, 2019 4:40 AM
    Moderator
  • How many workers would you recommend as a minimum for completing Microsoft's Azure Databricks training modules (docs.microsoft.com/en-us/learn/browse/?term=databricks)?  I would expect a cluster with only one worker to only consume 8 cores (including 4 for the driver) but you advised asking for a quota increase over the 10 cores we get with a pay-as-you-go subscription.




    • Proposed as answer by Brian C Brown Wednesday, September 11, 2019 7:04 PM
    • Edited by Brian C Brown Wednesday, September 11, 2019 7:05 PM
    Wednesday, September 11, 2019 6:35 PM