ASP.NET Cache Testing Pattern

Question

I'd like to use CHESS to test for thread safety amongst my cached objects within my ASP.NET application.  I've constructed a few tests but am struggling with confidence that CHESS is really providing any help.

I'd like to know if anyone has a good pattern for testing this that I might be able to compare my tests against?

Answer 1

David,

Can you please give a little more detail about the tests you have constructed? Also, please tell me what behavior you are seeing when you run the tests under CHESS with the attribute

[TestProperty("ChessDebug","true")]

Thanks.

-- Tom

Answer 2

Tom,
Certainly.

My Scenario
Today I have existing ASP>NET application, very large, which have frequently use objects that are cached as Static classes/members of my HttpApplication.  These objects are shared accross sessions as intended.  The "bug" I see today is that if I load my app and use it, everything functions.  If I recycle my app pool, clearing out my cached objects, and the proceed to try and request 2 pages (very fast - before one gets a chance to load), I end up receiving an error that "X" does not exist in the collection, where X represents an item in one of the cached collections.

Since I know that:
a) The collection is always READ ONLY in the application, we do not write back any updates.
b) The item IS in the database - if I try to load the page again, it works.

Then, I have concluded that there must be a concurrency issue within the way my cached objects function.  I examined the code and found that, BINGO, the cached objects with this issue were not using thread lock 's, while others (not causing the error) were - when they loaded the data from the database. 

By adding the lock(this) pattern into those classes, the issue has not been able to be reproduced by normal (human) means.
However, I desire to have a unit test that can seek these scenarios out in the future.  I thought CHESS might be able to provide that but when I wired up a similar scenario as to what you see below, everything came back from CHESS A'OK.

I hope I am not asking too much :)

So, here is a demo app I have put together, I am sure you can assemble it pretty easily in VS. 

Solution has 3 projects.  Web UI, DataLayer, UnitTests
Unit Tests and Web UI need a reference to DataLayer

What I would like CHESS to accomplish?
Find a scenario where two (or maybe more) threads try to have the cached collection (whatever it may be) loaded from the database. 

DataLayer \ Categories.cs

<strong>using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;

namespace DataLayer
{
    public static class Categories
    {
        /// <summary>
        /// Our static collection of data; cached in the ASP.NET context.
        /// </summary>
        private static List<String> categories = null;

        /// <summary>
        /// Returns a list of all categories.
        /// </summary>
        /// <returns></returns>
        public static List<string> GetAll()
        {
            if (categories == null)
            {
                // This would typically be a database call.
                categories = new List<string>();
                categories.Add("Books");
                categories.Add("Movies");
                categories.Add("Cars");
                categories.Add("Food");
                Trace.WriteLine("Categories loaded from cache");
                lastLoadFromCache = true;

                // If this were a database call, we would do something like this....
                //lock (this)
                //{
                //    READ DATA FROM DB
                //}
            }
            else
                lastLoadFromCache = false;
            return categories;
        }

        private static bool lastLoadFromCache;
        /// <summary>
        /// Gets or sets a value indicating whether [last load from cache].
        /// </summary>
        /// <value><c>true</c> if [last load from cache]; otherwise, <c>false</c>.</value>
        public static bool LastLoadFromCache
        {
            get { return lastLoadFromCache; }
            set { lastLoadFromCache = value; }
        }

        /// <summary>
        /// Gets the count of the number of categories.
        /// </summary>
        /// <value>The count.</value>
        public static int Count
        {
            get { return GetAll().Count; }
        }

        /// This is a bit overdone but here for demonstrate the point.

        /// <summary>
        /// Adds the specified name.
        /// </summary>
        /// <param name="Name">The name.</param>
        public static void Add(String Name)
        {
            categories.Add(Name);
        }

        /// <summary>
        /// Removes the specified name.
        /// </summary>
        /// <param name="Name">The name.</param>
        public static void Remove(String Name)
        {
            categories.Remove(Name);
        }
            
    }
}<br/><br/><br/></strong>

UnitTests \ CacheTest.cs

using System;
using System.Text;
using System.Collections.Generic;
using System.Linq;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Threading;
using DataLayer;

namespace UnitTests
{
    /// <summary>
    /// Summary description for CacheTest
    /// </summary>
    [TestClass]
    public class CacheTest
    {
        public CacheTest()
        {
            //
            // TODO: Add constructor logic here
            //
        }

        private TestContext testContextInstance;

        /// <summary>
        ///Gets or sets the test context which provides
        ///information about and functionality for the current test run.
        ///</summary>
        public TestContext TestContext
        {
            get
            {
                return testContextInstance;
            }
            set
            {
                testContextInstance = value;
            }
        }

        #region Additional test attributes
        //
        // You can use the following additional attributes as you write your tests:
        //
        // Use ClassInitialize to run code before running the first test in the class
        // [ClassInitialize()]
        // public static void MyClassInitialize(TestContext testContext) { }
        //
        // Use ClassCleanup to run code after all tests in a class have run
        // [ClassCleanup()]
        // public static void MyClassCleanup() { }
        //
        // Use TestInitialize to run code before running each test 
        // [TestInitialize()]
        // public void MyTestInitialize() { }
        //
        // Use TestCleanup to run code after each test has run
        // [TestCleanup()]
        // public void MyTestCleanup() { }
        //
        #endregion

        /// <summary>
        /// Use CHESS to look for issues.
        /// </summary>
        [TestMethod]
        [HostType("CHESS")]
        [TestProperty("ChessDebug", "true")]
        public void CHESSTest()
        {
            Thread child = new Thread(testWork);
            child.Start();

            int count = Categories.Count;
            Categories.Remove("Books");
            Assert.AreEqual(count - 1, Categories.Count);

            Categories.Add("Books");
            Assert.AreEqual(count, Categories.Count);

            child.Join();
        }

        private void testWork()
        {
            int count = Categories.Count;
            Categories.Add("Test");
            Assert.AreEqual(count + 1, Categories.Count);

            Categories.Remove("Test");
            Assert.AreEqual(count, Categories.Count);
        }
    }
}

Web UI \ Default.aspx.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;

public partial class _Default : System.Web.UI.Page 
{
    protected void Page_Load(object sender, EventArgs e)
    {
        // Hacked together as an example.
        foreach (string category in DataLayer.Categories.GetAll())
            Response.Write(String.Format("{0}<br/>", category));

        Response.Write(string.Format("<br/>The categories {0} loaded from cache.", DataLayer.Categories.LastLoadFromCache ? "were" : "were not"));
    }
}

Answer 3

I went back through and tried to clean the grammar and formatting up. ;)

Answer 4

OK. The problem with this example (and maybe the real code you are working with) is that by default, CHESS preempts threads at synchronization operations (i.e., Monitor.Enter/Exit) and volatile accesses. In the class Categories, you have neither, so CHESS doesn't
find the bug.  If you make a simple change to your code, the bug will manifest. Change:

private static bool lastLoadFromCache;

to

private static volatile bool lastLoadFromCache;

Another option to get more bugs from CHESS is to:

  - use /detectraces to tell CHESS to find data races in your code (which it should find on the variable lastLoadFromCache);
  - use /preemptaccesses to tell CHESS place a preemption before every read/write to shared memory;

These options require you to use the mchess tool (rather than the Visual Studio integration, which is lagging behind mchess in support).

-- Tom

Answer 5

One more thing: the fix to the code is to make the property

        public static bool LastLoadFromCache
        {
            get { return lastLoadFromCache; }
            set { lastLoadFromCache = value; } }

synchronized. I don't know if this is the same problem you were having in the production code. You previously said:

> By adding the lock(this) pattern into those classes, the issue has not been able to be reproduced by normal (human) means.

If you make LastLoadFromCache synchronized (I believe you can do this via an attribute for static methods/properties in .NET).

I've not checked how CHESS runs with this change to the code.

-- Tom

Answer 6

Ah, but there is still a problem in:

        public static List<string> GetAll()
        {
            if (categories == null)
            {
                // This would typically be a database call.
                categories = new List<string>();
                categories.Add("Books");
                categories.Add("Movies");
                categories.Add("Cars");
                categories.Add("Food");
                Trace.WriteLine("Categories loaded from cache");
                lastLoadFromCache = true;

If you don't make the initialization/write to the database and the setting of lastLoadFromCache appear to be atomic then two instances of

int count = Categories.Count;

can race each other in bad ways. The safest thing (not necc. most efficient) is to make GetAll synchronized too.

Finally, you need to be careful to make sure to use the same lock on this as you do on LastLoadFromCache, otherwise bad things can occur.

-- Tom

Answer 7

Tom,
Fantastic!  That make sense to me and in the production code, what you suggest is already what I implemented.  The example I provided was more/less just hacked together to simulate the scenario.

Also, it would be great to have the "detectraces" as a property with the VS integration.