locked
How to create a file writing queue

    Question

  • I have this part in my class that saves an XML document to file.

    It can be called from anywhere in the programm, and the caller does not want to wait for the completion, just fire and forget.

    savesPending: 0,
    saveAsync: function() {
      self = this
      logger.debug("Save requested (_cachedFile=" + self._cachedFile + ")");
      if (self.saving) {
        self.savesPending += 1;
        logger.debug("saveAsync: already saving, " + self.savesPending + " pending now");
        return self.saving;
      }
      return self.saving = getCachedFile(self.serialNumber).then(function(file) {
        logger.debug("saveAsync: saving...");
        return self.node.ownerDocument.saveToFileAsync(file);
      }).then(null, (function(error) {
        return logger.error("saveAsync: failed but " + self.savesPending + " saves pending - good luck!", error);
      }).then((function() {
        logger.debug("saveAsync: saved and " + self.savesPending + " pending");
        self.saving = null;
        if (self.savesPending) {
          self.savesPending = 0;
          return self.saveAsync();
        }
      });
    }

    The calls to the method can happen from async completion handlers so it can happen that the function is called roughly at the same time and then the first self.saving check returns false twice.

    This is an example stack from 2 calls that reach the function which leads to 2 parts of the program ending up writing to the file.

    Function: InspectionMixin.saveAsync, Thread: 0x56FC Main Thread, Callstack: 	InspectionMixin.saveAsync [Inspection.js]
    	InspectionMixin.setProperty [Inspection.js]
    	classInfo.instanceMembers[name].set [models.js]
    	Anonymous function [models.js]
    	Anonymous function [models.js]
    	Anonymous function [Inspection.js]
    	Anonymous function [Inspection.js]
    	notifySuccess [base.js]
    	state_success_notify.enter [base.js]
    	_run [base.js]
    	_completed [base.js]
    	notifySuccess [Function code]
    	doNotify [Function code]
    	doComplete [Function code]
    	op.completed [Function code]
    	
    Function: InspectionMixin.saveAsync, Thread: 0x56FC Main Thread, Callstack: 	InspectionMixin.saveAsync [Inspection.js]
    	InspectionMixin.setProperty [Inspection.js]
    	classInfo.instanceMembers[name].set [models.js]
    	InspectionMixin.calculatePerformanceWeightRatio [Inspection.js]
    	classInfo.ctor [Inspection.js]
    	Anonymous function [models.js]
    	Anonymous function [Inspection.js]
    	InspectionMixin.previous.get [Inspection.js]
    	InspectionMixin.mileageType.get [Inspection.js]
    	InspectionMixin.mileageOrHours.get [Inspection.js]
    	InspectionMixin.validateAsync [Inspection.js]
    	InspectionMixin.setProperty [Inspection.js]
    	classInfo.instanceMembers[name].set [models.js]
    	Anonymous function [models.js]
    	Anonymous function [models.js]
    	Anonymous function [Inspection.js]
    	Anonymous function [Inspection.js]
    	notifySuccess [base.js]
    	state_success_notify.enter [base.js]
    	_run [base.js]
    	_completed [base.js]
    	notifySuccess [Function code]
    	doNotify [Function code]
    	doComplete [Function code]
    	op.completed [Function code]

    Wednesday, August 20, 2014 1:27 PM

All replies

  • Is this a question or a "how-to" instructional?

    Matt Small - Microsoft Escalation Engineer - Forum Moderator
    If my reply answers your question, please mark this post as answered.

    NOTE: If I ask for code, please provide something that I can drop directly into a project and run (including XAML), or an actual application project. I'm trying to help a lot of people, so I don't have time to figure out weird snippets with undefined objects and unknown namespaces.

    Thursday, August 21, 2014 7:20 PM
    Moderator
  • A question, because it does not work.
    Thursday, August 21, 2014 7:29 PM
  • I'd put the data into a queue object, and if there isn't a save worker running - start one in the background to write the data.

    In the worker task, set a flag to indicate a save worker is running, and while there are entities to save in the queue, write them to disk, exiting only after the queue is empty, and the save worker flag is reset.

    You could also do some optimizations this way.  If you look in the queue from the worker task and see the same file is being written twice, drop the first entry in favour of the last.  If you're writing to the same file - the worker could simply eat all but the last one.

    It depends highly on your specific circumstance though.

    Is that what you want to do ?


    Darin R.

    Thursday, August 21, 2014 9:53 PM
  • I am always writing to the same file, its not a general function. I want to write the file as fast as possible, but I think a worker is too much hazzle for such simple task. 

    As I wrote, I got it almost working, with one tiny race-condition on the function entry. Sometimes its called from 2 (async) completion handlers at almost the same time slice. An then both think there is no save in progress, when in fact the first one is currently preparing and assigning the "saving" variable the save promise.

    Thursday, August 21, 2014 10:13 PM