none
FileSystemWatcher - Move file with fix count RRS feed

  • Question

  • Hi Experts, 

    Is there a way I can limit the files to moved which uses the File event watcher for creation & changed. Let say if my directory gets loaded with 100 files of 30 -40 mb each. If I need to move these files to another location just move 3 files and add delay of 10 seconds interval. The reason I am doing this because if I do massive transfer my other background services will try to process all the files and that peaks up the CPU.

    Thanks

    Priya

    Thursday, December 12, 2019 5:55 PM

All replies

  • High level thought is to place new files into a list and process them say on a timer set to a suitable interval.

    Here is a singleton for doing new and renamed files.

    public sealed class WatchFileContainer
    {
        private static readonly Lazy<WatchFileContainer> Lazy =
            new Lazy<WatchFileContainer>(() => new WatchFileContainer());
    
        public static WatchFileContainer Instance => Lazy.Value;
    
        public List<FileContainer> NewFileList { get; set; }
        public List<FileContainer> RenamedFileList { get; set; } 
    
        private WatchFileContainer()
        {
            NewFileList = new List<FileContainer>();
            RenamedFileList = new List<FileContainer>();
        }
    }

    Classes for storing file information

    public class FileContainer
    {
        public bool Processed { get; set; }
        public string Action { get; set; }
        public string Message { get; set; }
        public string EventTime { get; set; }
        public override string ToString()
        {
            return $"{Action},{Message},{EventTime}";
        }
    }

    Using in a form rigged up for the FileSystemWatcher

    private void FileCreated(object sender, FileSystemEventArgs e)
    {
    
        WatchFileContainer.Instance.NewFileList.Add(new FileContainer()
        {
            Processed = false,
            Action = "Created",
            Message = $"{e.Name}",
            EventTime = Now.ToString("yyyy/MM/dd HH:mm:ss")
        });
    }

    On a timer interval check to see if there are files needing processing e.g.

    WatchFileContainer.Instance.NewFileList.Any(item => !item.Processed)
    Process x files, mark Processed as true, move on. At some point clear processed files from the list or remove each one as it's processed.


    Please remember to mark the replies as answers if they help and unmarked them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.

    NuGet BaseConnectionLibrary for database connections.

    StackOverFlow
    profile for Karen Payne on Stack Exchange

    Thursday, December 12, 2019 7:56 PM
    Moderator
  • I assume that you're moving these files in code and not via File Explorer. In that case just put an artificial delay in your move logic. However if you're using something like Directory.Copy then it wouldn't work as the OS handles it.

    However I question if you're actually using FSW properly. FSW is triggered by the journaling API and therefore any code you write to handle events from it are going to slow down the actual file operation. Therefore your usage of FSW should involve copying the relevant event data to someplace else and having a background thread do the actual work. If you're going this route then you shouldn't really notice any perf issues whether you're making 10 or 1000 file changes at once. 


    Michael Taylor http://www.michaeltaylorp3.net

    Thursday, December 12, 2019 8:04 PM
    Moderator
  • Noticed that you are talking about other background processes that will peak. You also asked about optimizing FileSystemWatcher in your other question. Maybe you should describe the whole process not just part of it. This might help to resolve the issue.

    Is it something like:

    1. 100 files gets loaded to directory.

    2. FileSystemWatcher notices this and starts to move files to another location.

    3. In another location some process, possibly other FileSystemWatcher?, detect moved file and starts some process for that file that peak CPU?

    Anyway would follow Michael's advice to queue file information at FileSystemWatcher event handler(s) and then process them from queue with another thread. If the whole process is what I described above, then you might need two queues.

    Friday, December 13, 2019 2:19 PM