locked
audit logging RRS feed

  • Question

  • User1080785583 posted

    I have to audit changes in xml and in sql concurrently. What would be best practice for  2 mediums of delivery in this instance? 

    Friday, August 23, 2013 10:45 PM

Answers

  • User-488622176 posted

    Again, I suggest to go one step back on XML auditing. Do you allow end-users to customize the XML? Or are they a representation of an entity/object? In the last case: you could gain in writing an object compare mechanism (old vs new version), and use this for both the xml & sql requirements. In this case, you'll need to deserialize the XML to an object.

    Auditing at entity level implies:

    • Loading previous version from dbase
    • Comparing with what you assume is the previous version (concurrency checks, row versioning)
    • Compare all properties between the new and old version, and track the changes
    • Do this recursively, including for N-M relations. In this case, you'll need to find a way to determine what was new in the collection, what was deleted, and what was modified. Feasible but complex :-)

    Main costs drivers:

    • IO : loading from dbase
    • Complexity : dealing with concurrency, locking (what is you load version N-1, another user updates this version, you do the comparison and log the changes with the depricated version) 
    • CPU load when applied to object hierarchies

    I'd recommend doing this asynchronously in order to avoid UI issues or blocked services...

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Thursday, August 29, 2013 3:08 AM

All replies

  • User-837620913 posted

    Almost all logging frameworks allow you to write logging/auditing in one place and then output to multiple "appenders" or log outputs.

    Use log4net with 2 output appenders, the log file and SQL ones. http://logging.apache.org/log4net/

    NLog can also do this. http://nlog-project.org/

    Saturday, August 24, 2013 6:02 AM
  • User71929859 posted

    Hello,

    Elaborating a bit more about what Darrel said about log4net:

    You can use FileAppender in log4net for log in a file. You can use AdoNetAppender to log in to a database.

    Sunday, August 25, 2013 11:13 AM
  • User1080785583 posted

    more specifically i need to audit data changes in xml and in sql with a loosely coupled audit engine that is not restricted by the web and can be used in any type of project, example wcf, wpf, etc. I want to avoid using my native mvc language for this, because I want to be able to audit an object that can either be database, file, etc.

    I looked into these links and I cant seem to get this in nuget [https://github.com/nlog/NLog/], because when I download and attempt to examine, no luck!

    Monday, August 26, 2013 12:24 PM
  • User71929859 posted

    that is not restricted by the web and can be used in any type of project, example wcf, wpf, etc.

    log4net work for any kind of .NET project.

    Monday, August 26, 2013 12:46 PM
  • User1080785583 posted

    I downloaded log4net from here, http://logging.apache.org/log4net/download_log4net.cgi

    the problem is with the cs-examples I cannot get anything to build and I start out with 110 errors...

    I am reading documentation and I am still confused how this will allow me to audit different objects history. Some objects are xml, some objects are database values.

    The problems that arise are that I need to be able to say FirstName is in 2 different forms, but I want to know the history of Form A, not Form B. Will log4net help with these kinds of transaction history?

    Monday, August 26, 2013 2:31 PM
  • User-488622176 posted

    There ar 2 things you should not mix up:

    • Logging capabilities
    • object change tracking/logging

    Log4net is a .NET component that gives you the first : logging capabilities. It allows you to store info (text, ...) into different log repositories (file, eventlog, queues, dbase). What you need is a system that will compare 2 business objects/entities, extract the differences, and translate this delta set to a set of logging messages. Log4Net does not allow the first part. You'll need to build this yourself.

    However, there are some frameworks such as EntityFramework & NHibernate that offer this kind of functionality (either internally, or through addins). 

    Beware : this kind of nonfunctionals are expensive (in terms of ressource consumption).

    Tuesday, August 27, 2013 2:59 AM
  • User1080785583 posted

    Point taken. I tried to think of how it would really work and not limit the user experience, which made me think of only doing this on the server. For some reason, I am the only one that thinks the worker process is going to freak out with all the audit code and eventually start to drag the UI down. What I need is what you explained, Illeris. 

    What kind of expenses are we speaking of, other than the obvious cpu tick expense, is there something down the road that would give rise to an open source solution? 

    In regards to Entity, I was able to see you can override the submit changes in the entity, but it is still coupled with my code. So I took another route and found this bad boy, http://autoaudit.codeplex.com/.

    Now this works in all cases of my requirements for column level tracking, except I cannot audit the xml changes, which is the other 98%. I have a column of type XML and I must be able to audit this column as well as the other columns but I am sure this engine does not support xml. 

    Performance is always in the front of my head and I do realize the performance impact on this data which would allow me to think of a non-real time audit to kep the website seperate from this audit beast.

    Tuesday, August 27, 2013 10:56 AM
  • User-488622176 posted

    Again, I suggest to go one step back on XML auditing. Do you allow end-users to customize the XML? Or are they a representation of an entity/object? In the last case: you could gain in writing an object compare mechanism (old vs new version), and use this for both the xml & sql requirements. In this case, you'll need to deserialize the XML to an object.

    Auditing at entity level implies:

    • Loading previous version from dbase
    • Comparing with what you assume is the previous version (concurrency checks, row versioning)
    • Compare all properties between the new and old version, and track the changes
    • Do this recursively, including for N-M relations. In this case, you'll need to find a way to determine what was new in the collection, what was deleted, and what was modified. Feasible but complex :-)

    Main costs drivers:

    • IO : loading from dbase
    • Complexity : dealing with concurrency, locking (what is you load version N-1, another user updates this version, you do the comparison and log the changes with the depricated version) 
    • CPU load when applied to object hierarchies

    I'd recommend doing this asynchronously in order to avoid UI issues or blocked services...

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Thursday, August 29, 2013 3:08 AM