locked
Error in Storm spout reading from Service Bus Queue RRS feed

  • Question

  • Hi,

    We are running a Storm topology on a HDInsight Storm cluster. The spout is reading items from a Service bus (SB) queue & handing it over to a bolt which then writes in a HDFS location.

    Spout is able to successfully read from an Event Hub instance but, we are also trying to have this read from a SB queue.

    Writing to the queue is working fine but while reading from it, the following error is displayed:

    Caused by: com.microsoft.windowsazure.services.core.utils.ConnectionStringSyntaxException: The key 'SharedAccessKeyName' is not valid for this connection string

           at com.microsoft.windowsazure.services.core.utils.ParsedConnectionString.saveValue(ParsedConnectionString.java:296) ~[microsoft-windowsazure-api-0.4.6.jar:na]

           at com.microsoft.windowsazure.services.core.utils.ParsedConnectionString.matchKeyValuePair(ParsedConnectionString.java:137) ~[microsoft-windowsazure-api-0.4.6.jar:na]

           at com.microsoft.windowsazure.services.core.utils.ParsedConnectionString.matchAttribute(ParsedConnectionString.java:108) ~[microsoft-windowsazure-api-0.4.6.jar:na]

           at com.microsoft.windowsazure.services.core.utils.ParsedConnectionString.matchConnectionString(ParsedConnectionString.java:89) ~[microsoft-windowsazure-api-0.4.6.jar:na]

    (Part of the error response provided above) 

    Code snippet used in the spout for connecting to the queue is taken from this link - https://github.com/elastacloud/servicebusspout

    And the windows Azure library in use is the one specified there -

    <dependency>

    <groupId>com.microsoft.windowsazure</groupId>

           <artifactId>microsoft-windowsazure-api</artifactId>

           <version>0.4.6</version>

    </dependency>

     

    On searching up the error, it specified that SB namespace needs to have ACS authentication instead of SAS. And ACS authentication can be enabled through Powershell cmdlets.

    http://blogs.msdn.com/b/servicebus/archive/2014/09/03/change-to-azure-service-bus-portal-default-authentication-mechanism-for-service-bus-namespaces-now-sas.aspx

    http://blogs.msdn.com/b/cie/archive/2014/08/29/service-bus-namespace-creation-on-portal-no-longer-has-acs-connection-string.aspx

     

    But running the given command in Powershell to create a new SB namespace à New-AzureSBNamespace -Name whatsoneventhubjana -Location "Southeast Asia" -CreateACSNamespace $true -NamespaceType Messaging

    gives this exception - A parameter cannot be found that matches parameter name 'CreateACSNamespace'.

    Can anyone please throw light on how do we solve this issue. Please do let us know if any more information is required. 


    What's-On




    Tuesday, May 19, 2015 10:22 AM

Answers

  • Hi Ravi,

    We have been able to resolve the above issue of having a Storm spout read from a Service Bus queue.
    Broad steps followed are:

    1) The first issue of ACS authentication enabled SB queue was resolved by upgrading Azure Powershell version to 0.9. Previously it was 0.7. Doing this allowed us to run the following command to create a new SB namespace:
    New-AzureSBNamespace -Name whatsoneventhubjana -Location "Southeast Asia" -CreateACSNamespace $true -NamespaceType Messaging
    The ACS Authenticated connection string was then used in spout code to receive messages from Service Bus Queue.

    2) Added all missing dependencies including those related to Service Bus, as you have mentioned above, using link - https://github.com/Azure/azure-sdk-for-java/blob/master/serviceBus/pom.xml

    3) There are two enums which specify how a QueueClient can read from a queue:
    a. PEEK_LOCK – didn’t work for us & gave null result for messages received.
    b. RECEIVE_AND_DELETE - Able to receive messages but cannot de-serialize due to missing Jackson mapper library, which can be added by using below dependency code in POM file.

    <dependency>
    <groupId>org.codehaus.jackson</groupId>
    <artifactId>jackson-mapper-asl</artifactId>
    <version>1.9.13</version>
    </dependency>

    Appreciate your help & assistance. 

    Thanks & regards,
    Surjit

    • Edited by WhatsOnIndia Wednesday, May 27, 2015 10:08 AM
    • Marked as answer by WhatsOnIndia Wednesday, May 27, 2015 10:09 AM
    Wednesday, May 27, 2015 10:05 AM

All replies

  • In addition to what we have posted above, have also found reference to a new Azure library on Maven Repository from this link: http://blogs.msdn.com/b/cie/archive/2014/12/23/unable-to-create-topic-queues-within-service-bus-with-java-sdk.aspx

    Library link: http://mvnrepository.com/artifact/com.microsoft.azure/azure-servicebus/0.7.0

    This Azure library is supposedly compatible with SAS authentication.

    <dependency>

    <groupId>com.microsoft.azure</groupId>

    <artifactId>azure-servicebus</artifactId>

    <version>0.7.0</version>

    </dependency>


    After using the above library the previous exception i.e. “SharedAccessKeyName” has got resolved but, getting a new exception “Async loop died!”.

    Part of the exception stacktrace is given below:

    19657 [Thread-10-sb-queue-reader] ERROR backtype.storm.util - Async loop died!

    java.lang.TypeNotPresentException: Type org.apache.http.impl.client.HttpClientBuilder not present

           at sun.reflect.generics.factory.CoreReflectionFactory.makeNamedType(Unknown Source) ~[na:1.8.0_45]

           at sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Unknown Source) ~[na:1.8.0_45]

           at sun.reflect.generics.tree.ClassTypeSignature.accept(Unknown Source) ~[na:1.8.0_45]

           at sun.reflect.generics.visitor.Reifier.reifyTypeArguments(Unknown Source) ~[na:1.8.0_45]

           at sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Unknown Source) ~[na:1.8.0_45]

           at sun.reflect.generics.tree.ClassTypeSignature.accept(Unknown Source) ~[na:1.8.0_45]

           at sun.reflect.generics.repository.ClassRepository.getSuperInterfaces(Unknown Source) ~[na:1.8.0_45]

           at java.lang.Class.getGenericInterfaces(Unknown Source) ~[na:1.8.0_45]

           at com.microsoft.windowsazure.core.DefaultBuilder.add(DefaultBuilder.java:181) ~[azure-core-0.7.0.jar:na]

           at com.microsoft.windowsazure.core.pipeline.apache.Exports.register(Exports.java:58) ~[azure-core-0.7.0.jar:na]

           at com.microsoft.windowsazure.core.DefaultBuilder.create(DefaultBuilder.java:46) ~[azure-core-0.7.0.jar:na]

           at com.microsoft.windowsazure.Configuration.<init>(Configuration.java:67) ~[azure-core-0.7.0.jar:na]

           at com.whatsonindia.eventhub.finaleventhub.ServiceBusQueueConnection.connect(ServiceBusQueueConnection.java:53) ~[classes/:na]

           at com.whatsonindia.eventhub.finaleventhub.ServiceBusQueueSpout.open(ServiceBusQueueSpout.java:37) ~[classes/:na]

           at backtype.storm.daemon.executor$fn__3284$fn__3299.invoke(executor.clj:520) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]

           at backtype.storm.util$async_loop$fn__452.invoke(util.clj:429) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]

           at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]

           at java.lang.Thread.run(Unknown Source) [na:1.8.0_45]

    Can anyone please help us with this.

    What's-On









    Thursday, May 21, 2015 7:45 AM
  • Hi WhatsOnIndia

    Regarding the questions you asked:

    1. The first problem was primarily due to older Powershell cmdlets. The service bus related cmdlets recently went through few updates in recent Azure Powershell versions. You say have solved that problem, good to see that.

    Although you may already have this code, but we do have some automation created here for some Azure services that you can leverage: https://github.com/hdinsight/hdinsight-storm-examples/tree/master/scripts/azure/EventHubs

    Our official Github repo has end-to-end fully automated examples that you can use for reference towards coding topologies or Azure powershell automation.

    2. The second problem is more due to jar you are using. Unlike Hadoop/Hive, Storm does not use HDFS or a common jar location. So it is expected that the topology jar should include all your dependencies. From the error stack its clear that you are missing some classes due to which this fails. My suggestion would be to revisit your pom.xml, include all the dependencies that your topology needs including Service Bus and its dependencies too (as they are ones that seem to be missing). Then while submitting use the jar file that end with name : -jar-with-dependencies.jar

    Error: org.apache.http.impl.client.HttpClientBuilder not present

    Example: https://github.com/hdinsight/hdinsight-storm-examples/blob/master/EventCountExample/EventCountTopology/pom.xml

    See how we have also put a dependency on amqp jars which eventhub uses - similarly you need to include org.apache.http or any other dependencies.

    Another easier approach would to check the pom.xml for the ServiceBus jar to get the list of dependencies.

    Their dependency list: https://github.com/Azure/azure-sdk-for-java/blob/master/serviceBus/pom.xml

    These need to be present in your pom.xml as we need all classes in one single big jar.

    All our examples in official repo use jar-with-dependencies to avoid these issues.

    Why this occurs on Storm for Service Bus Only: Cause other jars like Kafka etc are already part of classpath. When you bring in third-party jars or dependencies outside what comes with Storm, you will have to do the above.

    Hope this helps.

    Friday, May 22, 2015 11:28 PM
  • Hi Ravi,


    Thank you very much for the detailed response.

    Yes, the first issue got resolved after we updated the Azure Service Bus library.

    Also, as you conveyed, to access & run service bus related cmdlets, Azure Powershell will need to be updated and we will do that. Our current Powershell ver. is 0.7.3.1

    Finally on the point of missing dependencies, the topology jar (named accordingly as -jar-with-dependencies.jar) includes all dependent libs as we have last seen.

    However, will get this re-checked & include dependencies that might be missing viz. those related to service bus, as you have pointed out.

    Will revert shortly with findings & steps taken for resolution.

    Appreciate your time & assistance!!


    What's-On




    • Edited by WhatsOnIndia Saturday, May 23, 2015 12:43 PM
    • Marked as answer by WhatsOnIndia Wednesday, May 27, 2015 10:05 AM
    • Unmarked as answer by WhatsOnIndia Wednesday, May 27, 2015 10:05 AM
    Saturday, May 23, 2015 12:40 PM
  • Hi Ravi,

    We have been able to resolve the above issue of having a Storm spout read from a Service Bus queue.
    Broad steps followed are:

    1) The first issue of ACS authentication enabled SB queue was resolved by upgrading Azure Powershell version to 0.9. Previously it was 0.7. Doing this allowed us to run the following command to create a new SB namespace:
    New-AzureSBNamespace -Name whatsoneventhubjana -Location "Southeast Asia" -CreateACSNamespace $true -NamespaceType Messaging
    The ACS Authenticated connection string was then used in spout code to receive messages from Service Bus Queue.

    2) Added all missing dependencies including those related to Service Bus, as you have mentioned above, using link - https://github.com/Azure/azure-sdk-for-java/blob/master/serviceBus/pom.xml

    3) There are two enums which specify how a QueueClient can read from a queue:
    a. PEEK_LOCK – didn’t work for us & gave null result for messages received.
    b. RECEIVE_AND_DELETE - Able to receive messages but cannot de-serialize due to missing Jackson mapper library, which can be added by using below dependency code in POM file.

    <dependency>
    <groupId>org.codehaus.jackson</groupId>
    <artifactId>jackson-mapper-asl</artifactId>
    <version>1.9.13</version>
    </dependency>

    Appreciate your help & assistance. 

    Thanks & regards,
    Surjit

    • Edited by WhatsOnIndia Wednesday, May 27, 2015 10:08 AM
    • Marked as answer by WhatsOnIndia Wednesday, May 27, 2015 10:09 AM
    Wednesday, May 27, 2015 10:05 AM
  • Very happy to know this Surjit! Do let us know if you hit any issues using HDInsight Storm.

    Also let us know at hdinsight-storm-examples what more examples or templates you want to see. A Azure Service Bus queue will definitely be a good addition.

    Friday, June 5, 2015 10:15 PM