locked
Push Notifications on new or updated blobs RRS feed

  • Question

  • Hi,

    I want to receive notification when a new blob gets added into a container or a existing blob gets changed. I don't want the pull principle: every 15 seconds ListBlobs(...) and compare....

    What is the proposed route for implementing a push principle/pattern?

    I tried to use queues, but await queue.getMessages() does return Nothing when no Messages are there. I thought it would "sleep" until messages arrive. So I cant use the await pattern with queues.

    (Do I have to write a WCF Service at Azure that observes blobs and their etags for example?)


    Thursday, March 10, 2016 1:54 PM

All replies

  • Two ways to do this:

    - Create a queue message when you create the blob, and use the QueueTrigger attribute on the function that processes the blob.

    OR

    - Use WebJobs SDK with Azure blob storage to get notified of new blobs or create a queue message when you create a blob. The WebJobs SDK scans log files to watch for new or changed blobs. This process is not real-time; a function might not get triggered until several minutes or longer after the blob is created. 

    See more details here:

    https://azure.microsoft.com/en-us/documentation/articles/websites-dotnet-webjobs-sdk-storage-blobs-how-to/

    • Proposed as answer by Pradnya K Friday, March 11, 2016 3:33 AM
    Friday, March 11, 2016 3:33 AM
  • Hi,

    You could try the WebJobs SDK. There's a [BlobInput] attribute that lets you specify a container to listen on, and it includes an efficient blob listener that will dispatch to the method when new blobs are detected.
    You could refer the following link for details:
    https://blogs.msdn.microsoft.com/jmstall/2014/02/18/azure-storage-bindings-part-1-blobs/

    and the triggering rules are explained in the link below:
    https://blogs.msdn.microsoft.com/jmstall/2014/01/28/trigger-bindings-and-route-parameters-in-azurejobs/

    Regards,
    Malar.

    Friday, March 11, 2016 7:02 AM
  • I created a queuemessage every blob I upload already, but then I gave it up, because I could only poll the queue at the client.

    Ok now with the new input, I have further questions:

        public static void Main()
        {
            JobHost host = new JobHost();
            host.RunAndBlock();
        }
    
        public static void ProcessQueueMessage([QueueTrigger("webjobsqueue")] string inputText, 
            [Blob("containername/blobname")]TextWriter writer)
        {
            writer.WriteLine(inputText);
        }

    I found this example code.
    (https://azure.microsoft.com/documentation/articles/websites-dotnet-webjobs-sdk/)

    That's all? I just have to start an (Azure...)JobHost and my Method "ProcessQueueMessage" gets executed? 
    How does the JobHost know which AzureAccount it has to use? ConnectionString?!

    - Create a queue message when you create the blob, and use the QueueTrigger attribute on the function that processes the blob.

    Could you please show a little example on how to do that?

    Friday, March 11, 2016 11:00 AM
  • I tried it in console app:

       Public Class Class1
          Public Shared Sub Main(args() As String)
             ' Dim host As New Microsoft.Azure.Jobs.JobHost()
             Dim host = New Microsoft.Azure.Jobs.JobHost(New Microsoft.Azure.Jobs.JobHostConfiguration() With {
                                     .StorageConnectionString = "aaaaaaaaaaa",
                                     .DashboardConnectionString = "aaaaaaaaaaaaa"
                                    })
             host.RunAndBlock()
          End Sub
    
          Public Shared Sub ProcessQueueMessage(<Microsoft.Azure.Jobs.QueueTrigger("testqueue")> inputText As String, <Microsoft.Azure.Jobs.Blob("test")> writer As IO.TextWriter)
             writer.WriteLine(inputText)
          End Sub
       End Class

    Microsoft.Azure.Jobs.IndexException wurde nicht behandelt.
      HResult=-2146233088
      Message=Index error on method 'ProcessQueueMessage': Object reference not set to an instance of an object.
      Source=Microsoft.Azure.Jobs.Host
      StackTrace:
           at Microsoft.Azure.Jobs.IndexException.NewMethod(String methodName, Exception inner)
           at Microsoft.Azure.Jobs.Host.Indexers.Indexer.CreateFunctionDefinition(MethodInfo method)
           at Microsoft.Azure.Jobs.Host.Indexers.Indexer.IndexType(Type type)
           at Microsoft.Azure.Jobs.Host.Indexers.FunctionStore..ctor(CloudStorageAccount storageAccount, String serviceBusConnectionString, IConfiguration config, IEnumerable`1 types)
           at Microsoft.Azure.Jobs.JobHostContext..ctor(String dashboardConnectionString, String storageConnectionString, String serviceBusConnectionString, ITypeLocator typeLocator, INameResolver nameResolver)
           at Microsoft.Azure.Jobs.JobHost..ctor(IServiceProvider serviceProvider)
           at Microsoft.Azure.Jobs.JobHost..ctor(JobHostConfiguration configuration)
           at ConsoleApplication1.myspace.Class1.Main(String[] args) in C:\_demos\jur\ConsoleApplication1\ConsoleApplication1\Class1.vb:line 7
      InnerException: 
           HResult=-2147467261
           Message=Object reference not set to an instance of an object.
           Source=Microsoft.Azure.Jobs.Host
           StackTrace:
                at Microsoft.Azure.Jobs.BlobClient.ValidateBlobName(String blobName)
                at Microsoft.Azure.Jobs.Host.Blobs.Bindings.BlobAttributeBindingProvider.TryCreate(BindingProviderContext context)
                at Microsoft.Azure.Jobs.Host.Bindings.CompositeBindingProvider.TryCreate(BindingProviderContext context)
                at Microsoft.Azure.Jobs.Host.Indexers.Indexer.CreateFunctionDefinitionInternal(MethodInfo method)
                at Microsoft.Azure.Jobs.Host.Indexers.Indexer.CreateFunctionDefinition(MethodInfo method)
           InnerException: 
    

    Friday, March 11, 2016 3:29 PM
  • Hi together^^,

    I made it. JobHost works perfectly but now I have a problem:

    The application should not have the StorageConnectionString. It should use SAS. Is it possible to create a JobHost via SAS or observe the Queue via SAS?

    Wednesday, March 16, 2016 8:21 AM
  • Hi,

    If you need to delegate access to a resource in just one of the storage services in your case the Queue service you could consider using Service SAS.
    You could refer the following link for details on service SAS:
    https://msdn.microsoft.com/library/dn140255.aspx
    https://msdn.microsoft.com/library/dn140256.aspx

    Regards,
    Malar.

    Wednesday, March 16, 2016 9:05 AM
  • Actually I use polling in the clients. (windows forms app) I need the clients to get notified if a new message is in a queue without using the connectionstring. I know how to use SAS with queues, but that does not solve the problem here. I need a JobHost/WebJob at the Client that gets notified using SAS.


    Wednesday, March 16, 2016 10:15 AM
  • Hi,

         I am also looking for same solution, If a new blob comes to Azure storage or it is updated setup a notification in azure queue- I want to do it with Azure functions but i am new to Functions. So can you help me" how i have to trigger it and add it to queue" ?

    Note : Already some containers are there in blob storage !

    Thanks you.

    Wednesday, September 27, 2017 4:52 PM
  • Check out the recently announced Blob Storage events along with Event Grid:

    https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blob-event-overview

    You can use event grid to connect these events to Azure Functions.  This tutorial provides a detailed example:

    https://docs.microsoft.com/en-us/azure/event-grid/resize-images-on-storage-blob-upload-event

    Tuesday, October 3, 2017 10:19 PM