locked
xml as database alternative too good to be true? RRS feed

  • Question

  • User2110873642 posted

    hello people of all sorts,

    Im making a website, and currently its so data intensive,  that it takes 20-30 seconds to read and write from database on avarage pageload. Everything needs to be realtime and synchronised, so i truly cant avoid using cashe. using cashe and indexing techniques ive brought it back to milliseconds, but having almost 2/3 of the data in cashe feels scarey to me if the website process might crash or lose data, and im wondering how i can avoid feeling scared. or what kind of alternatives or safety features i could use.

    The use for the database has become useless as everything is now working from cashe, and i decided to go with serialisation instead, and load it on server startup, and save it in background processes way after the page has been returned. as this makes the website startup faster than loading it over TCP trough the databaseconnection.

    since everything is cashed and serialised from xml to classes, programming has become magically powerfull, allowing all kinds of operations that no programmer would advice over SQL connections. The website runs smooth, solid, and more powerfull than anything ive seen before, and i almost feel like im outsmarting the world by doing this, but i expect it to be a really bad design, because it seems too good to be true. what are the opinions of you guys?

    Monday, November 18, 2019 12:04 PM

All replies

  • User753101303 posted

    Hi,

    XML files was never meant a replacement for db. As you told, you'll likkely lose data if the app is recycled. What about consistent backup, db integrity, concurrency etc... 

    You tried to understand first why your database was slow? It's IMO bad to just try something else because something that should work doesn't rather than trying first to understand why something that should work doesn't. 20 to 30 seconds is clearly not the kind of response time you have from a db (you were trying to read thousands of rows each time ???)

    You have a whole lot of database flavor depending on  what you are trying to do such as in memory databases or document oriented databases. Relational engines may have also optimization features such as "in memory  tables" etc...

    Edit: for example maybe transitioning to a https://en.wikipedia.org/wiki/Document-oriented_database. The very first step would be likely to check what you need from https://en.wikipedia.org/wiki/ACID (you likely want Durability ?)

    Monday, November 18, 2019 12:27 PM
  • User475983607 posted

    Im making a website, and currently its so data intensive,  that it takes 20-30 seconds to read and write from database on avarage pageload. Everything needs to be realtime and synchronised, so i truly cant avoid using cashe. using cashe and indexing techniques ive brought it back to milliseconds, but having almost 2/3 of the data in cashe feels scarey to me if the website process might crash or lose data, and im wondering how i can avoid feeling scared. or what kind of alternatives or safety features i could use.

    Most applications take advantage of indexes and caching.   Caching should not cause data loss.  I recommend testing your design to see what happens if there is an application restart or fatal exception.

    The use for the database has become useless as everything is now working from cashe, and i decided to go with serialisation instead, and load it on server startup, and save it in background processes way after the page has been returned. as this makes the website startup faster than loading it over TCP trough the databaseconnection.

    This design sounds like an in-memory data store not cache.  Cache is temporary storage that refreshes during run-time.  Often cache will expire which causes a reload.  Other times cache is refreshed due to a change in the back end data store.

    since everything is cashed and serialised from xml to classes, programming has become magically powerfull, allowing all kinds of operations that no programmer would advice over SQL connections. The website runs smooth, solid, and more powerfull than anything ive seen before, and i almost feel like im outsmarting the world by doing this, but i expect it to be a really bad design, because it seems too good to be true. what are the opinions of you guys?

    In my opinion this design is atypical and not an option for most web applications.  If this design works for your application than fantastic.  

    Monday, November 18, 2019 12:37 PM