none
Azure ML Classic Web Services Endpoints limited to 5000 RRS feed

  • Question

  • Hello. 

    In Azure ML Studio, after develope a experiment, I deploy it as a Classic Web Service and generate a Predictive Experiment, in order to consume it for every client. I generate ilearners with the trained model of every client, that has to be consumed with an endpoint.

    I have no problem in create as many ilearners as clients there are, but when I create the endpoint I encounter that there´s a limitation of 5000 endpoints. 

    I have been looking for details or how to solve it, but hasn´t find any case.

    Is there any way to tune or quit this limitation?

    Thank you very much for your help.




    • Edited by ffuente Monday, August 26, 2019 11:48 AM
    Friday, August 23, 2019 1:21 PM

All replies

  • Hi ffuente,

    Thank you for reaching out, may I know are you using a free tier or standard tier Machine Learning Studio? Have you already created 5000 endpoints? Please check within your  Azure Machine Learning Web Service portal and refer to following document if you have any question for web service portal: https://docs.microsoft.com/en-us/azure/machine-learning/studio/create-endpoint#scaling

    Please let us know more details.

    Regards,

    Yutong

    Tuesday, August 27, 2019 3:19 AM
    Moderator
  • Thank you very much Yutong Tie for your answer.

    I am using a PaidStandard Workspace with a TIER S2 plan for the Web Service.

    Not necessarily I call 200 requests concurrently. I don´t think I call more than 10-15 at the same time. But in case I did it, I don´t think it could affect to the 5000 endpoint limit I encounter.

    I have more than 90.000 clients, and I have to create an ilearner and endpoint for every one. No limit with the ilearners (stored in a blob), but with endpoints.

    I run a runbook with a Powershell script to train the model with data of a client, and the ilearner is created. The script creates the ilearner and the endpoint. After ran, the logging message I get is this:

    Error: [403 (Forbidden)]: { "error": { "code": "MaximumEndpointCountReached", "message": "Maximum number of endpoints reached: 5000. Delete endpoints or see online help for options to increase the number of endpoints for your workspace type." } }
    On the Classic Web Services screen I can see all Web Services I have created. They all have 1 only endpoint, with the exception of the one I´m working in, that has 5000. 

    Thank you very much for your response.



    • Edited by ffuente Tuesday, August 27, 2019 8:21 AM
    Tuesday, August 27, 2019 7:59 AM
  • Hi ffuente,

    Thank you for your information, since you are in paidstandard tier, I would suggest you to open a support ticket from Help + Support channel to solve the problem. One support engineer will help you to check your endpoints from the backend. 

    Regards,

    Yutong

    Wednesday, August 28, 2019 5:59 PM
    Moderator
  • Thank you Yutong Tie for your response.

    I have a confirmation from one of your colleagues, that replies to the same issue asked by another user.

    Finally, the limitation of 5000 endpoints per Classic Web Service is real, so you can only deal with it managing the amount of endpoints created per Classic Web Service. As you can only create a CWS for every predictive experiment, for every 5000 endpoints you have to create a predictive experiment. So then you must manage the number of endpoints created in every Predictive Experiment.

    You can consult the issue in this GitHub issue.


    Wednesday, September 4, 2019 1:56 PM
  • Thank you for the details. I am also keeping my eyes on that thread and follow-up with that product manager. 

    Regards,

    Yutong

    Wednesday, September 4, 2019 6:22 PM
    Moderator