Microsoft does not provide a supported way to read all useful task data from Project Online RRS feed

  • Question

  • I have been using PSI to read task values from MS Project Server into my application.  Now that MS Project Online 2013 no longer supports PSI, I am developing CSOM code to do the same thing.   I may be overlooking something simple, but I cannot see how to gain access to local custom fields such as Text1-30, Flag1-20, Number1-20, etc.  These fields are populated and clearly have values when I open my Project Online 2013 projects in the browser or in MS Project Pro, but I cannot see them in CSOM!   In PSI, for on-premises Project Server 2013 or 2010, these field values for a task are stored in the TaskCustomField table, but I see nothing similar in the CSOM namespace.   Watching CSOM read tasks in the debugger, when I examine each PublishedTask, I see a standard set of fields, but no Flag20 or Text30.   Also, when I look in the Task.CustomFields and in the dictionary PublishedTask.FieldValues, I do not see them.  I've searched the Microsoft documentation, StackOverflow and googled all over the place but have found no mention of this omission.  Furthermore Microsoft does not mention omission of local custom fields in the article 'What the CSOM Does and Does Not Do'.  Am I overlooking something simple in the way I do my Load statements, or did Microsoft omit these important fields from the CSOM interface?

    Here, in VB.Net, is the code I am using to gain access to the PublishedTasks.  Can anyone see something that would bring in Flag20 and Text30?

           Function(c) c.Where(Function(p) p.Id = projGuid))
    If mProjContext.Projects.Count = 1 Then
       Dim proj As PublishedProject = mProjContext.Projects.First()
            Function(c) c.IncludeWithDefaultProperties(
               Function(t) t.CustomFields.Include(
                  Function(f) f.Id,
                  Function(f) f.InternalName,
                  Function(f) f.Name
               Function(t) t.Assignments
       Return proj.Tasks
    End If

    ...Jim Black

    • Edited by Jim Black CG Friday, April 8, 2016 12:49 PM clarification of title
    Monday, December 21, 2015 3:13 AM


  • Hi there,
    currently local custom fields are not supported. They come maybe with Project Server 2016.


    • Marked as answer by Jim Black CG Monday, December 21, 2015 1:38 PM
    Monday, December 21, 2015 8:49 AM

All replies

  • Hi there,
    currently local custom fields are not supported. They come maybe with Project Server 2016.


    • Marked as answer by Jim Black CG Monday, December 21, 2015 1:38 PM
    Monday, December 21, 2015 8:49 AM
  • Hi Sansha,

    Thanks very much for the quick reply.   Has Microsoft said this officially somewhere?    I ask because I cannot understand why Microsoft would restrict Project Online programmatic access to CSOM.  This means that there is no programmatic access to fields that most people currently use in Project Online and Project Pro for Office 365.   I wonder why Microsoft does not mention this in their MSDN article: 'What the CSOM Does and Does Not Do'?  Is there a workaround for Project Online local custom fields, since we are prevented by design from using PSI for Project Online?

    ...Jim Black

    • Edited by Jim Black CG Monday, December 21, 2015 1:46 PM clarification
    Monday, December 21, 2015 1:37 PM
  • Hey,
    no they didn't say it that clearly, but I added a request for it, and they told it might get fixed. So there is nothing in stone yet. But even with PSI the local custom field handling was not really good implemented anyway.

    Sadly there is no workaround at this time, I struggle with the same problem. A hint for a custom solution might be to copy the local custom field values into enterprise custom field values with a macro, but in my opinion this is a very bad approach.

    The best what you can do is to add this as well as feedback to Microsoft. The more people question something the higher the chance is that something might happen.

    Monday, December 21, 2015 3:59 PM
  • Hi,

    Yes, I have already submitted a request to the Project Server feedback site, and I hope to hear back from them today.   I doubt that you and I are the only two people in the world who are struggling with a Project Online that has no programmatic access to local custom fields! 

    The workaround will be a pain for my company's customers, so I'll have our salespeople start advising customers to not upgrade to Project Online if they want to use OnePager (Standalone).  I'll ask our marketing department to add  some caveats around our statement of support for Project Online, since we'll only be able to support it as a COM Add-In, using Microsoft Project Pro to give us access to the fields.  Ugh!

    Thanks again for your response on this.  By the way, one of the Microsoft MVP for Project Server responded to me via private email to confirm that CSOM cannot do this.

    ...Jim Black

    Monday, December 21, 2015 5:59 PM
  • Hi,

    I did find a definitive statement on MSDN buried away in the documentation on enterprise custom fields, where you will find the statement:

    "The CSOM can access only enterprise custom fields, not local custom fields that are defined within a project."

    So there we have it!  

    ...Jim Black

    Tuesday, December 22, 2015 2:19 PM
  • The PSI does not support updating local custom fields either.

    Tuesday, December 22, 2015 5:56 PM
  • Hi Carlos,

    Thanks for the information.  My application does not need to update the local custom fields.  It just needs to read them.   PSI gives good support for reading the local custom fields, but CSOM does not.

    ...Jim Black

    Tuesday, December 22, 2015 9:51 PM
  • Jim,

    Can you show us how you "read" LOCAL custom fields with the PSI. As far as I know, this is not supported either. How do you do it?

    Tuesday, December 22, 2015 10:01 PM
  • Carlos,

    Glad to.  The trick is to use the PSI table called TaskCustomFields.   In reality this is just the local custom fields.   To pull values out of there, I devised some ADO.Net filtering and relations to grab the local custom fields for each task in the project.   Below is my code in VB.Net, and it is easy to convert it to C#, of course.   This function does a little bit more than you asked for because I am using the value of a local custom flag field to select tasks to output from the function.  But if you observe how I handle the TaskCustomField table, relating to the Task table via the Task_UIDColumn in both tables, I think you'll see how to grab the local custom fields for each task you are interested in.

     Public Function GetSelectedTasksFromCustFldTable(projGuid As System.Guid) As TaskRow()
         '   Purpose: gets selected tasks for the specified project, assuming that the selection clause is executed on the TaskCustomFields table
         Dim out As New Generic.List(Of TaskRow)
             Dim pds As ProjectDataSet = mProjectSvc.ReadProject(projGuid, DataStoreEnum.PublishedStore)
             If Not IsNothing(pds) Then
                 Dim taskTable As TaskDataTable = pds.Task
                 Dim custFldTable As TaskCustomFieldsDataTable = pds.TaskCustomFields
                 Dim rel As DataRelation = New DataRelation("Task2TaskCustFld", taskTable.TASK_UIDColumn, custFldTable.TASK_UIDColumn)
                 Dim filtCol As String = CSDS.BuildSimpleRowFilter("MD_PROP_ID", CSoftColumnType.IntegerType, pjFlag20)
                 Dim filtVal As String = CSDS.BuildSimpleRowFilter("FLAG_VALUE", CSoftColumnType.BooleanType, True)
                 Dim filt As String = filtCol & " AND " & filtVal
                 For Each r As TaskCustomFieldsRow In custFldTable.Select(filt)
                     Dim r1 As DataRow = r.GetParentRow(rel)
                     Dim r2 As TaskRow = TryCast(r1, TaskRow)
                     If Not IsNothing(r2) Then out.Add(r2)
             End If
             Return out.ToArray
         Catch ex As Exception
            ... log an error message...
             Return out.ToArray
         End Try

    ...Jim Black

    • Edited by Jim Black CG Tuesday, December 22, 2015 11:15 PM cleanup
    Tuesday, December 22, 2015 11:04 PM
  • Jim,

    I use the TaskCustomFields dataset all of the time and I never noticed that it could hold local custom fields. I see you are querying for pjFlag20, so there are even constants defined, apparently. I will have to look at this next time I am using the PSI. Well, you learn something new every day. Thanks!

    Tuesday, December 22, 2015 11:49 PM
  • Carlos,

    Yes, I stumbled upon the usefulness of TaskCustomFields by watching things go by in the debugger.   It seems to work flawlessly for local custom fields, which is why I have been disappointed to see that it is not part of CSOM at all.  The pjFlag20 is an integer constant that I define outside the function I show, so it is not part of PSI per se.   However, the TaskCustomFields table has a column with integer values that correspond exactly to the pjFields of MS Project, as long as you cast the pjFields to integers.   Unfortunately the CSOM returned objects have completely omitted these handy integers, so you cannot use them there.   That's an easy enhancement to CSOM that I'd like to see Microsoft make! 

    ...Jim Black

    • Edited by Jim Black CG Wednesday, December 23, 2015 12:27 AM added more content
    Wednesday, December 23, 2015 12:24 AM
  • I entered an enhancement request at the Microsoft Project user-voice forum, requesting that CSOM be brought up to par with PSI so that we can read all commonly-used task data fields from Project Online.    Votes in favor of this suggestion might help persuade Microsoft that it is an important thing to do.


    ...Jim Black

    • Edited by Jim Black CG Friday, April 1, 2016 11:00 PM clarification
    Friday, April 1, 2016 10:57 PM
  • Please see the details of this issue below: 

    My company’s OnePager products create graphics to enhance the Microsoft Project family by programmatically reading task data from Microsoft cloud applications.   We have been doing this for Microsoft Project Server on-premises using the Project Server Interface (PSI), where our customers require that we read values of their local custom fields (e.g. Flag1, … Flag 20).  The PSI technology from Microsoft does a good job of meeting our customers’ requirements for these fields.

    With Project Online 2013, however, we appear to have a big problem.  Microsoft documentation states that CSOM is preferred and supported for Project Online but that

    [Project Online integration] can use the PSI, but not supported

    However CSOM technology does not provide any access to the local custom fields that our customers need.  We have learned this not only by running many tests ourselves but also by finding documentation in MSDN Office Dev Center, which states

    The CSOM can access only enterprise custom fields, not local custom fields that are defined within a project.

    ...Jim Black

    Friday, April 8, 2016 12:51 PM
  • That's true, but in Project Online Microsoft removed the ProjectService from PSI which enabled all this for you. The same thing happened with Project Server 2016 on premise.

    And to state it clearly PSI is a legacy interface, so you shouldn't rely on it for Project Server 2016 and Project Online. The whole interface will die at some point and you will have to use CSOM / OData.

    Friday, April 8, 2016 3:10 PM
  • How about we start a movement?


    Friday, April 8, 2016 3:16 PM
  • Hi again Sansha,

    You are absolutely right: Microsoft has declared that the whole interface is deprecated and will die at some point.   Until your post, however, I was NOT aware that they've withdrawn Project Service from PSI for Project Server 2016 on premises, so thanks for pointing that out.   Can you find any online documentation that tells us about the removal of this PSI functionality for Project Server 2016 on premises?  If so, please post a link to it.

    Meanwhile the most important thing is for Microsoft partners and customers to persuade Microsoft to complete CSOM so that we can start using it instead of PSI.  At present, it is impossible to read local custom fields via CSOM, and this is a showstopper.  Have you heard anything more of Microsoft plans about local custom fields in CSOM? At one point, you thought that they might do this for Project 2016, but I have no clue whether this came to pass or not.

    My company has an active Microsoft support ticket (SR # 116032513880858) open on this topic and hope to get some idea of how to work around the deficiency for Project Online 2013 until CSOM is made complete.  I have also posted a request to complete CSOM on the Microsoft Project User Voice site.  If you haven't already, please go to that site and add your comments to the request and also vote for it. And tell other developers to do it, too!

    ...Jim Black

    • Edited by Jim Black CG Friday, April 8, 2016 3:40 PM broken link
    Friday, April 8, 2016 3:31 PM
  • Sign me up!!  And, Carlos, if you have not done so already, please go to the Microsoft User Voice site and add your comments, suggestions, and vote for fixing this mess!

    ...Jim Black

    Friday, April 8, 2016 3:44 PM
  • Carlos,

    I just tweeted the PSI situation with the hashtag that you suggested.   I'm not great at Twitter, so see if you can find it by searching for #BringBackThePSI

    ...Jim Black

    Friday, April 8, 2016 4:05 PM
  • Unfortunately I am not good at it either, but I did find the hashtag and I re-tweeted your tweet. :)


    Friday, April 8, 2016 4:14 PM
  • Hey,
    I didn't see the documentation yet, cause it isn't released yet. We have access to pre-release versions of SharePoint and Project Server 2016 and there is the ProjectService gone. And I don't think that they will bring it back. It wouldn't do any sense if they'll do. We posted all our needs for CSOM to different Microsoft channels and we have to see what get's implemented. In the first iteration what we got, we've seen a lot of changes, but these were mostly bug fixes for various scenarios.


    Monday, April 11, 2016 9:06 AM
  • Sansha,

    I've gone far enough in the partner-support process now to know what you told us four months ago is quite true: the product group is aware of the CSOM deficiency, have given some thought to fixing it, but have no firm plans to fix it. I also think that they have some very good reasons to move from PSI to CSOM and are not going back on that decision.   So it's not a matter of bringing back PSI.  It's only a matter of putting local custom fields into CSOM.  What may stand in the way of this is that some people in the product group do not see value in supporting local custom fields on Project Server because customers can always use enterprise custom fields instead.  This may be the key point in the whole issue.   My experience with customers tells me that local custom fields will remain important on Project Server because of inertia.  Local custom fields are firmly established among the users of Microsoft Project Pro, and these people naturally expect to be able to move their data seamlessly from the desktop to the cloud and back again.   But I am not aware of a seamless way for customers to automatically convert their local custom fields into enterprise custom fields and vice versa.  Are you aware of any such capability or plans for such capability in Microsoft or the partner community?  The most useful thing I've found so far is this discussion answered by Dale Howard, and the procedure seems to require administrative intervention and thus does not look very seamless to me.


    ...Jim Black

    • Edited by Jim Black CG Thursday, April 14, 2016 9:44 PM added link to relevant blog
    Thursday, April 14, 2016 9:36 PM
  • You add the point. That MS doesn't see the need of local custom fields, but we work hard on this fact.

    We have the use-case that we have project dependent lookup table values which can only be placed locally. For example when you think about the work breakdown structure which is different for every project you can have a local custom field with around 30 entries containing only the valid elements for this schedule or you can have an enterprise one with 3000 entries which is not project dependent. And as we know all its pain in the ass for users / admins to manage that huge lookup tables. So we have no possibility to switch over to enterprise custom fields with lookup tables. Not to mention the different default values for different projects.


    Tuesday, April 19, 2016 9:20 AM
  • Carlos,

    Since it now clear that bringing back the PSI is not a practical thing for Microsoft to do, we just need to focus our efforts on getting them to fix CSOM so that we can get access to the local custom fields.  Thus there is a better hashtag for our movement:  #FixProjectServer. 

    Today someone in my company posted a non-technical blog post for our customers, which you may find useful in drumming up support for this issue:  http://bit.ly/1VEHw6t   We are also contacting many of our customers and asking them to support the local-custom-field-for-CSOM petition at Microsoft Project User Voice: http://bit.ly/1StHtHb

    We know for sure that the Microsoft Project product team is reviewing this petition, but they still may decide to reject it.  So the next few days is a critical time to get Microsoft Project Server users to tell Microsoft how important local custom fields are to them.  So it would be great if you and your contacts could spread the word, vote for the petition, and tweet the #FixProjectServer hashtag to @project



    ...Jim Black

    Wednesday, April 20, 2016 4:50 PM