Maintain "list" of streams as a relational table in StreamInsight

Question

Hello,

I want to maintain a "list", like a relational DB table, with available streams on a system, for instance input streams of stock quotes that may become unavailable over time, because of the source that emits them. So this isn't a stream, but a relational table, since it allows tuple deletion (when a stream disappears from the system). So i'm thinking of a standing query in the SI server (registered as a Windows Service), which updates a table of streams information (StreamName, SourceName.. etc) from a SQL Server database.

Is there any way to avoid SQL Server, or any other relational DBMS here? Does StreamInsight has some capabilities that allow handling such relational data?

Thanks

Answer 1

Hello,

I'm not completely sure what you're looking for here. If what you want is the actual set of adapters, queries, etc. on the server, then you can just pull these off properties on your Application.

If you want to expose these to other queries, then the simplest way to do this is through a stream. When a new entity you're tracking enters the system, it will be represented by an event starting when it entered, and with an end time at infinity. When an entity leaves the system, you just alter the lifetime of the event so that it ends when it left. Although you're not actually removing events from the stream, you end up with a set at each point in time that represents the objects that were alive then.

Cheers,
-Isaac

---

Isaac Kunen, Microsoft StreamInsight

Answer 2

Hello,

Then

1. does this mean you can represent any static (changing at a lower pace) data repository, like a SQL Server database, like a stream flowing into StreamInsight?

2. if this is the case, do you have full DDL and DML support?

3. if it is not the case, when would you choose to use a SQL Server db and when would you decide to represent such a db by a StreamInsight stream?

Maybe the update operation would be tricky, i.e. to choose an event from the stream (equivalent to a row from the relational table) and perform some modifications on it.. it's not clear for me yet..

Thanks

Answer 3

Yes, you can reference static or slowly changing data in StreamInsight. Like everything else, this is a temporal stream. So ... you read the data in via an input adapter to "convert" it to a temporal stream. You will then use the ToSignal macro to make sure that you have the most recent value. Finally, before joining, you will want to import the CTIs from your fast data stream into your reference stream.

See Mark Simms' blog post for some more details. 

---

DevBiker (aka J Sawyer)
My Blog
My Bike

---

If I answered your question, please mark as answer.
If my post was helpful, please mark as helpful.

Answer 4

Also, see this more recent blog post from Mark Simms.  

---

DevBiker (aka J Sawyer)
My Blog
My Bike

---

If I answered your question, please mark as answer.
If my post was helpful, please mark as helpful.