Loading historical data into StreamInsight

Question

I would like to use StreamInsight for the analysis of some stock price data.

The live data will be read from a socket so I guess I need to create an adapter for that. However, all of the historical data needs to be loaded from some text files.

So do I create an adaper for that too?

Is it possible to load the historic data into StreamInsight from the text files and then have the live data streamed in from the socket and to have queries over the full (both) sets of data?

e.g. some analysis might be yearly averages or 5 year averages. Can a query encompass the historical data that hasn't entered the stream from the adapter that listens on a socket?

Is the data actually stored in StreamInsight or do I need to create an Output adapter and have that handle the storage?

Where does the stream actually get stored in StreamInsight? Is it written to file? Held in memory?

Answer 1

Hi ED Rooney,

To get data in and out, input and output adapters are required. StreamInsight has nothing to do with data storage, it just use data adaptors to received and send streams of data that travel through memory. The actual data is the data in your text files in your scenario.
For more information about reference data files with StreamInsight, please have a look at:
StreamInsight and reference data (lists, databases, etc)
Master Large Data Streams with Microsoft StreamInsight

---

Stephanie Lv

TechNet Community Support