Problem creating a query using QueryBinder on a standalone server

Question

I am refactoring my application from using an embedded server to using a standalone server. I am doing this since I believe I have thread safety issues involving published streams and I am not currently able to solve them (I think the thread safety problems lie in the StreamInsight Application class, but this post is not about that). I believe (hope) that using a standalone server will solve this problem but I am having some trouble porting the application to a standalone server.

I have got to the point where I can connect to the standalone server (on the same machine as the application) and I can create the application and the adapters. I can create the query as well, but I can't start it. I get the error shown below (the FullyQualified names are object names in my application
'Eventing.EventDefinitions.MyDefinition' is the application that hosts the standing query
'Eventing_EventDefinitions_MyDefinition_Sources_ScheduleSource1 is the name of an InputAdapter'
'Eventing.MiddleTier.Source.Scheduler.SchedulerFactory' is the InputAdapter Factory ):

+  e {"There was an error in the runtime: The query 'Eventing.EventDefinitions.MyDefinition' (query template
'Eventing.EventDefinitions.MyDefinition') could not be started.-->The creation of a new adapter instance 'Eventing_EventDefinitions_MyDefinition_Sources_ScheduleSource1' for query  'cep:/Server/Application/Eventing.EventDefinitions.MyDefinition/Query/Eventing.EventDefinitions.MyDefinition' from factory of type Eventing.MiddleTier.Source.Scheduler.SchedulerFactory' failed.-->The identity must first be initialized"} System.Exception{Microsoft.ComplexEventProcessing.ConnectionException}

The line of code throwing the error is this one. The queryBinder appears to be setup properly with a queryTemplate and input and output bindings.

app.CreateQuery(name, description, queryBinder);

It all used to work in the embedded server case (with the exception of the thread safety issue) and I understand that once the standalone server has been setup properly, the rest of the streaminsight code needs no change, since the APIs are identical. I guess I am misunderstanding something though, since something isn't working.

I can't really post any more code because it is part of production code and is far too big to post here. I hope someone can point me in the direction to solve this though.

Answer 1

First things first ... why do you think that there is a thread safety issue? If you think it's actually in the core StreamInsight Application object, using the remote host won't resolve the issue at all. I highly doubt, however, that it's in the core StreamInsight API. In fact, the only threading issues that I've seen have always pointed to our code.  

Second, do you have the dll's installed and available to the remote service process? Keep in mind that they will need to be either in the GAC or in the folder for the instance. And they should be the same version that you are using for the remote "controller" application.

---

DevBiker (aka J Sawyer)
My Blog
My Bike - Concours 14

---

If I answered your question, please mark as answer.
If my post was helpful, please mark as helpful.

Answer 2

The reason why I suspect thread safety issues is partly because of this statement below, from the documentation on the Application class. Apart from that, I amm seeing some strange results when I am running multiple applications concurrently. The apps operate correctly while running, but when I stop and dispose the applications, they don't always all dispose correctly. Sometimes I have one or two of them saying they can't be deleted because they "are referenced by another object". I have not been able to figure out what the other object is, but am beginning to think it is a problem to do with the multiple apps running concurrently (which gets me to thinking about thread safety). All my dlls are copied to the StreamInsight Host folder (c:\Program Files\Microsoft StreamInsight 1.2\Host) 

Application Class

Thread Safety

---

Any public static (Shared in Visual Basic) members of this type are thread safe. Any instance members are not guaranteed to be thread safe.

See Also

---

Reference

Microsoft.ComplexEventProcessing Namespace

Answer 3

First, you see that on just about every class in the .NET Framework. It's not unique to StreamInsight. You are responsible for ensuring thread safety on any members that can be used in a multi-threaded scenario - such as instance members.

Second, if your apps aren't disposing correctly, using the out of process model won't help you. They still won't dispose properly. If you are getting ObjectDisposedExceptions or your instance is hanging on shutdown, you don't have the lifetime of your adapter right. Check out this blog entry on output adapter lifetime ... much of it also applies to input adapters as well as their lifetimes are very similar.

Finally, where are you getting the "referenced by another object"? I understand that you have some concerns around sharing production/proprietary code but if you could share where you get that error it may help. It may also help to use the Find Usages feature in Visual Studio to see where the objects are referenced and what's holding on to that reference.

There's nothing that you've said that would lead me to believe that using the out of process model would help you at all. You also need to consider that the OOP model does have its own unique set of challenges that you would have to overcome ... for example, if the StreamInsight instance restarts, how do the queries get started back up? And when? Even though the service uses the metadata provider to rehydrate all of your running queries, it doesn't start them back up because there is no good way for StreamInsight to understand if you need the queries started in a particular order and what that order may be.

---

DevBiker (aka J Sawyer)
My Blog
My Bike - Concours 14

---

If I answered your question, please mark as answer.
If my post was helpful, please mark as helpful.

Answer 4

I am asuming, as you say, that my objects aren't disposing correctly but I haven't figured out what else to dispose. The problem really is that my exception only happens when I have about 8 or 9 concurrent apps shutting down, and even then I only get the exception infrequently.

The "referenced by another object" exception is occuring as I attempt to delete the Applicaiton object in the Dispose method of the parent class.

This won't likely make too much sense, but here is the dispose method in question:

                StopQueries();
                if (m_LinqQueryTemplate != null)
                {
                    m_LinqQueryTemplate.Dispose();
                }
                m_Scripter.Dispose();
                m_ConsumingQueries.Clear();
                m_InputQueries.Clear();
                m_EventDefinition.Dispose();
                m_App.Dispose();  <- this is where the exception is throwing              

Answer 5

Why are you keeping references to your queries outside of the Application object? What is the collection class?

I would suspect that your queries aren't fully stopped in all cases when your application is disposed. Inconsistent behavior like this usually points to a thread race condition and, just looking at this quickly, that could well be happening. I would specifically stop all of your queries rather than simply clearing the collection. I typically delete the query templates on after all queries using those templates are already stopped and deleted. I'm pretty sure that Stop() and Delete() on the query are synchronous because I've never run into anything even closely resembling a race condition when shutting down. I also Stop the application ... StreamInsight Application object doesn't implement IDisposable so I'm not sure what the m_App.Dispose() is doing. I'm not sure if you are using the management service but, if you are, that does need to be closed when you shut down.

---

DevBiker (aka J Sawyer)
My Blog
My Bike - Concours 14

---

If I answered your question, please mark as answer.
If my post was helpful, please mark as helpful.

Answer 6

The m_App member variable is an instance of a wrapper class I made around the StreamInsight Application class. It is a fairly thin wrapper that just passes straight through to the StreamInsight class. It also implements the Dispose method to clean up  the Application instance (there may not really be a need for this wrapper but I put it all in place during earlier development while trying to sort out problems I was encountering at the time). Now that you mention it, I think I may need to refactor my class hierarchy a little; I currently have my Event container holding an instance of a Query class which is responsible for building up queries dynamically at runtime. It does this by handling the inputs and outputs that are present at runtime (there may be many) and building up a querytemplate in the querybinder class.

Nonetheless, the "StopQueries()" method in the code block I included previously is the method that stops all the queries. I only clear the colleciton after stopping the queries, as you suggest.

Here is the StopQueries() method. I will look into the possibility that the queries are not stopping when I expect them to have stopped.

        public void StopQueries()
        {
            // Stop input queries.
            if (!m_App.Disposed)
            {
                try
                {
                    s_Log.DebugFormat("Stopping queries for EventDefinition {0}",
                        m_EventDefinition.FullyQualifiedName);
                    foreach (Query query in m_App.Queries.Values)//m_InputQueries)
                    {
                        //s_Log.Debug("Ensuring query " + query.Name + " has stopped");
                        query.Stop();
                    }
                }
                catch (Exception ex)
                {
                    s_Log.ErrorFormat("Error caught stopping queries for EventDefinition {0}: {1}", m_EventDefinition.FullyQualifiedName, ex.Message);
                }
            }
        }

Answer 7

OK ... let's get back to the original question.

From the text of the error, it looks like it's a security identity that is causing the problem. Are you doing anything with the Windows security/principal context? Are you using integrated security and/or anything that may require permission from user account control? Have you tried setting the StreamInsight service to run under the same credentials that you used embedded for it to work?

Honestly, though ... there's no reason why you can't use the embedded model. There is nothing that you mentioned that wouldn't be an issue no matter what model that you used. There's really nothing magical about the StreamInsight service that ships with the product ... it uses the exact same APIs that you would use if you were embedded.

---

DevBiker (aka J Sawyer)

---

If I answered your question, please mark as answer.
If my post was helpful, please mark as helpful.

Answer 8

Based on the discussion here, I will stick with the embedded server I have been using up until now. Need to take a closer look at my shutdown procedure to see who is being left out in the cold. I have in the meantime established that all the queries in the app have stopped before I attempt to delete the app, so they aren't the ones holding the reference, it seems.