locked
Need to share data via internet for my application (Azure?) RRS feed

  • Question

  • I am developing a C# application that I intend to publish.

    I wish to have the app load data and update data that is shared between all the users currently running my app.

    This is not a great volume of data (much less than 1 GB  max forever).   And it is READ MOSTLY  ... 

    Where would I put this data?  Is that what amazon cloud could be used for? Or a Microsoft product (Azure?) ?

    At the start ...  The data slowly changes (with a few updates per second max, usually 100-200 per hour)

    thanks


    • Edited by ZenMusic Thursday, August 13, 2020 7:52 PM
    Thursday, August 13, 2020 7:51 PM

Answers

  • Got it, sort of like a music/movie library then. In that case you're looking at an Internet-hosted backend talking to a desktop (or mobile or even web app) front end. In that case you'll be using a mix of technologies. On the front end you can use data files or in memory database to store the data your app works with constantly. This is going to be a subset of all the data and most likely limited to what the user wants. How you store this is really dependent upon how you implement your app. For example if you want a user to be able to have multiple movie libraries then a library per file makes the most sense, especially if backup/restore is important. However an in memory database would be faster and you could provide import/export functionality if needed. You can store locally the per-user data.

    But to get the list of available stuff (movies, music, etc) then you'll need something accessible on the Internet. REST APIs (and/or gRPC) have basically won the current generation of IPC so I would recommend you build a REST API that provides the data needed by the front end. This API will eventually need to be hosted either on a web server or perhaps in the cloud depending upon how you want to do it. It doesn't really matter as far as the implementation is concerned. I'd recommend starting with a local web server for development and then host that in a hosting provider until you determine that you need scalability and then move to the cloud. Cloud is more expensive in almost all cases but scales better.

    The app will get any data not available locally via the public API. If none of this data is sensitive then you'll likely not need any authentication but if you want to ensure only your app can communicate then you'll need to take some steps. Building the API is beyond what we can discuss in the forums (although the ASP.NET forums can provide guidance). But I recommend you take a look at some existing APIs that are similar. For example IMDB has an API to search for movies, get data, etc. Music has the same thing. If you need to provide your own data then build a similar API (e.g. a search endpoint to pull back matches, a detail endpoint to get detailed info about an item, etc). If you are really just pulling data from other sources then it might just be easier to have your app talk to the other APIs directly. For example if I were building a movie library I'd not build my own API but rather bounce off IMDB's directly from the client. Of course that requires some licensing but it reduces the complexity.

    Going the REST route means you're bouncing off the same protocol as web browsing so firewalls (other than letting the app out) become less of an issue. Also this allows you to build the app in pretty much any language and consume it as well. Web front ends can call REST APIs as well as desktop and mobile apps. Other technologies could be used but support is more limited right now. gRPC, for example, cannot be used on a web front end currently without using a thunk layer. 


    Michael Taylor http://www.michaeltaylorp3.net

    • Marked as answer by ZenMusic Saturday, August 15, 2020 5:46 PM
    Saturday, August 15, 2020 5:22 PM

All replies

  • Where would I put this data?  Is that what amazon cloud could be used for? Or a Microsoft product (Azure?) ?

    Kind of, but there are plenty of Web hosting providers that support the Windows, IIS, MS SQL Server and .NET framework.

    I would suspect that you would ne an ASP.NET WebAPI service that did CRUD with  a database used by various clients.

    Thursday, August 13, 2020 8:36 PM
  • In the past I have always use a database (SQL Server or Oracle) for data.

    I could do that , but the data use is very light in this application, so I want the lightest , cheapest solution (for now).

    I thought Azure can provide  files read and write easily so might be the answer, but I haven't used Azure at all so don't know about that.


    • Edited by ZenMusic Thursday, August 13, 2020 9:30 PM
    Thursday, August 13, 2020 9:29 PM
  • I thought Azure can provide  files read and write easily so might be the answer, but I haven't used Azure at all so don't know about that.

    I don't see that being viable for some cloud based platform like Azure concerning what I would view as flat files holding data that are not sharable if used concurrently by clients as opposed to a enterprise level database  that provides concurrent client usage.


    • Edited by DA924x Thursday, August 13, 2020 9:41 PM
    Thursday, August 13, 2020 9:40 PM
  • This is a wide open question without a lot more details. You said internet in the post but didn't mention in the description. You also didn't mention the type of "app" you have. Is the app running on different machines across the Internet? On the same network? Is there any sort of user authentication? Who does the updating? What actual type of app is it?

    You can google for IPC and find all the possible ways to communicate between processes (apps) so I would recommend you just start there (REST, WCF, gRPC, database, event hooks, messaging, etc). An open ended question like this doesn't tend to get good responses as there are too many opinions. No matter what option you choose half a dozen people are going to tell you why you're wrong and their solution is better. Pick the one that works best for your needs.

    Whether your communication is hosted in the cloud or not is mostly irrelevant. Your app shouldn't care about how communication is hosted so I'd say forget the cloud stuff for now until you have a working solution. Then if you want to move it to the cloud for performance/price/whatever reasons then do so then.


    Michael Taylor http://www.michaeltaylorp3.net

    Saturday, August 15, 2020 3:57 PM
  • thanks for the thoughtful reply. The application is to be sold as a personal (home) application. There is no direct communication between the individual applications or users. Only a set of data that is shared on startup (load and initialize the app by reading several read only files that will expand and evolve over time but only need be read on the startup of the application).

    Individual users can add data (annotations) in their own personal file (only they can read and write that one).  That data (personal) will be considered for inclusion in the main data files (annotations mostly) if it is useful to all users (they would only see the updates the next time they startup the application). 

    So this is very slowly changing. Read mostly. Low tech. The prototype works now on a local network, each PC represents a users and they used a shared disk drive.


    • Edited by ZenMusic Saturday, August 15, 2020 5:01 PM
    Saturday, August 15, 2020 5:00 PM
  • Got it, sort of like a music/movie library then. In that case you're looking at an Internet-hosted backend talking to a desktop (or mobile or even web app) front end. In that case you'll be using a mix of technologies. On the front end you can use data files or in memory database to store the data your app works with constantly. This is going to be a subset of all the data and most likely limited to what the user wants. How you store this is really dependent upon how you implement your app. For example if you want a user to be able to have multiple movie libraries then a library per file makes the most sense, especially if backup/restore is important. However an in memory database would be faster and you could provide import/export functionality if needed. You can store locally the per-user data.

    But to get the list of available stuff (movies, music, etc) then you'll need something accessible on the Internet. REST APIs (and/or gRPC) have basically won the current generation of IPC so I would recommend you build a REST API that provides the data needed by the front end. This API will eventually need to be hosted either on a web server or perhaps in the cloud depending upon how you want to do it. It doesn't really matter as far as the implementation is concerned. I'd recommend starting with a local web server for development and then host that in a hosting provider until you determine that you need scalability and then move to the cloud. Cloud is more expensive in almost all cases but scales better.

    The app will get any data not available locally via the public API. If none of this data is sensitive then you'll likely not need any authentication but if you want to ensure only your app can communicate then you'll need to take some steps. Building the API is beyond what we can discuss in the forums (although the ASP.NET forums can provide guidance). But I recommend you take a look at some existing APIs that are similar. For example IMDB has an API to search for movies, get data, etc. Music has the same thing. If you need to provide your own data then build a similar API (e.g. a search endpoint to pull back matches, a detail endpoint to get detailed info about an item, etc). If you are really just pulling data from other sources then it might just be easier to have your app talk to the other APIs directly. For example if I were building a movie library I'd not build my own API but rather bounce off IMDB's directly from the client. Of course that requires some licensing but it reduces the complexity.

    Going the REST route means you're bouncing off the same protocol as web browsing so firewalls (other than letting the app out) become less of an issue. Also this allows you to build the app in pretty much any language and consume it as well. Web front ends can call REST APIs as well as desktop and mobile apps. Other technologies could be used but support is more limited right now. gRPC, for example, cannot be used on a web front end currently without using a thunk layer. 


    Michael Taylor http://www.michaeltaylorp3.net

    • Marked as answer by ZenMusic Saturday, August 15, 2020 5:46 PM
    Saturday, August 15, 2020 5:22 PM