Custom field values are null if ProjectCollection was loaded with CSOM RRS feed

  • General discussion

  • One of updates (I believe it was somewhere between June and September 2018, but cannot tell for sure) had introduced us the following problem with Project Server 2016:

    - Suppose that:

    1. We have PWA deployed at url "http://pwa/
    2. An enterprise custom field named "Test field" has been set up for projects (lets assume it is a text field);
    3. We have two projects. The value of "Test field" has been specified for each one; 

    - Then the following test (expressed with NUnit) will fail:

    using System.Linq;
    using Microsoft.ProjectServer.Client;
    using Microsoft.SharePoint.Client;
    using NUnit.Framework;
    namespace CSOMProjectCollectionLoadingBugDemo
        class ProjectLoadingWithExplicitlySpecifiedCustomFields
            public void AllProjectsMustHaveValueOfTestCustomField()
                // Create CSOM context:
                var context = new ProjectContext("http://pwa/");
                // Load custom field internal name to load its values explicitly:
                context.Load(context.CustomFields, cfc => cfc.Include(cf => cf.Name, cf => cf.InternalName));
                var fieldKey = context.CustomFields.AsEnumerable().First(f => f.Name == "Test field").InternalName;
                // Load projects (note how we load an entire collection of projects here):
                context.Load(context.Projects, pc => pc.Include(p => p[fieldKey]));
                // Asserting custom field value of each project to be non null
                Assert.That(context.Projects[0][fieldKey], Is.Not.Null);
                Assert.That(context.Projects[1][fieldKey], Is.Not.Null);

    The first assertion completes without any problem but the second one fails. If we try to look into the project's FieldValues dictionary, we find out that both projects have the record for the custom field in the dictionary, but the value for the first project is set up correctly, while the value for the second one is null.

    It could be worked around if one loads projects one by one instead of loading entire collection.

    I've tried to investigate it further and I'm pretty sure I've found the root of the problem: it lies somewhere around caching mechanism used in server-side code serving requests from CSOM and JSOM (JSOM too is subject to this behaviour) in Microsoft.ProjectServer.dll. I'm not sure if I'm allowed to paste some code fragments from the disassembled assembly, so I just describe it in common terms. Is seems like classical problem of cache misuse to me:

    1. As a part of the result construction (on the server) a project collection is created, then iterated to populate the result with project's data;
    2. When accessing any data of the first project in a collection, the PSI method is called to load the data for one particular project (the guid of the project is explicitly passed into the method);
    3. The dataset is used as the data source for project's fields, tasks etc. Another object is created as part of project construction. It is associated with the dataset (remember, it contains the data for the first project only) and later used to create custom field values (each value is represented as an instance of one specific class);
    4. When a custom field value of the project is accessed, an object of the class, mentioned above, is created. Upon creation an assotiation with the cache of project's custom field values is established. If cache miss happened (and it is, since it is the first project created and we access only one custom field value of each project), custom field values are cached, but, again, for the first project only(they are retrieved from the dataset from p.2). It's importance to note, that cache key depends only on storage (the same key used for all published project and another one for all draft projects);
    5. When the second project instance is created and accessed, another dataset is created (p.2);
    6. When custom field value object for the second project is created, all custom field values are loaded from the cache, but since the key is the same as for the first project and the cache has per-request scope, cache hit happened and the correct dataset not used as the source of the custom field values. The old one is used instead. And since all custom fields are from another project (the first one), the situation is exactly the same as if no custom field values exist at all for all projects but the first one, hence all nulls;

    I can show exact places in code of Microsoft.ProjectServer.dll if it is allowed, but I think the description is detailed enough to figure it out. Again, it's just my understanding of the problem, though I'm pretty sure it is the way I've described. But I could be all wrong about all this and the root of the problem lies somewhere else.

    Also, I must appologize for using this forum to provide more like a blog post than a question or a discussion, but I really don't know where exactly should I leave a report like this. Would be grateful if someone points me in correct direction. And would be double grateful if someone in Microsoft note this to fix.

    The version of SharePoint I've used is 16.0.4783.1000 (december, 2018). The version of Microsoft.ProjectServer.dll assembly is 16.0.4774.1000.

    Monday, December 24, 2018 3:55 PM

All replies