Problem with service pack upgrade. Errors. SharePoint, WSS.

Question

*Update:
As it turns out, our upgrade succeeded (verified by dll, db versions, etc), however in the process CONFIGURATION database became corrupted. Because of the corruption various Timer Jobs were not completing and it was snowballing from there.

Solution: Create a new configuration db. This is a nightmare as it entails redeploying all of the solutions and all of the customizations.
Here is an article that outlines things to consider http://technet.microsoft.com/en-us/library/cc512815.aspx

Good luck.

Eventvwr logs this error:

Unknown SQL Exception 208 occured. Additional error information from SQL Server is included below.
Invalid object name 'fn_GetChildren'.
Transaction count after EXECUTE indicates that a COMMIT or ROLLBACK TRANSACTION statement is missing. Previous count = 0, current count = 1.


After a failed upgrade,
Log files shows:

05/10/2009 16:22:22  8  INF            Entering function TaskBase.OnTaskStop
05/10/2009 16:22:22  8  INF              Creating the OnTaskStop event for task upgrade
05/10/2009 16:22:22  8  ERR              Task upgrade has failed
05/10/2009 16:22:22  8  INF              friendlyMessage for task upgrade is An exception of type System.NullReferenceException was thrown.  Additional exception information: Object reference not set to an instance of an object.
05/10/2009 16:22:22  8  INF              debugMessage for task upgrade is Failed to upgrade SharePoint Products and Technologies.  Further information regarding this failure can be found at D:\LOGS\12\LOGS\Upgrade.log.
An exception of type System.NullReferenceException was thrown.  Additional exception information: Object reference not set to an instance of an object.
System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.SharePoint.Administration.SPConfigurationDatabase.DeleteObject(Guid id)
   at Microsoft.SharePoint.Administration.SPConfigurationDatabase.DeleteObject(SPPersistedObject obj)
   at Microsoft.SharePoint.Administration.SPPersistedObject.Delete()
   at Microsoft.SharePoint.Administration.SPJobDefinition.DeleteInternal(Boolean bTargetInstances, Boolean bLockAndRunningJobs)
   at Microsoft.SharePoint.Administration.SPJobDefinition.Delete()
   at Microsoft.SharePoint.Upgrade.SPManager.RemoveUpgradeTimerJobRegKey(Boolean deleteJob)
   at Microsoft.SharePoint.PostSetupConfiguration.UpgradeTask.Run()
   at Microsoft.SharePoint.PostSetupConfiguration.TaskThread.ExecuteTask()


and


05/10/2009 16:22:23  1  ERR          Failed to upgrade SharePoint Products and Technologies.
Failed to upgrade SharePoint Products and Technologies.  Further information regarding this failure can be found at D:\LOGS\12\LOGS\Upgrade.log.
An exception of type System.NullReferenceException was thrown.  Additional exception information: Object reference not set to an instance of an object.
System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.SharePoint.Administration.SPConfigurationDatabase.DeleteObject(Guid id)
   at Microsoft.SharePoint.Administration.SPConfigurationDatabase.DeleteObject(SPPersistedObject obj)
   at Microsoft.SharePoint.Administration.SPPersistedObject.Delete()
   at Microsoft.SharePoint.Administration.SPJobDefinition.DeleteInternal(Boolean bTargetInstances, Boolean bLockAndRunningJobs)
   at Microsoft.SharePoint.Administration.SPJobDefinition.Delete()
   at Microsoft.SharePoint.Upgrade.SPManager.RemoveUpgradeTimerJobRegKey(Boolean deleteJob)
   at Microsoft.SharePoint.PostSetupConfiguration.UpgradeTask.Run()
   at Microsoft.SharePoint.PostSetupConfiguration.TaskThread.ExecuteTask()

Answer 1

Hi,

Which service pack were you installing 1 or 2?  Historically there has been no way to undo an Office Update.  My understanding is that Service Pack 2 was going to be the first to allow an uninstall, but I have not installed SP2 yet in any of my environments, but that will hopefully be happening later this week.

You should always install the WSS 3.0 Service Pack first and then the MOSS Service Pack, and then run the Configuration Wizard.  You could try installing the WSS 3.0 Service Pack, then installing\re-installing the MOSS2007 Service pack and then running the Configuration Wizard and that may work.

The following article contains great instructions on updating a SharePoint Farm.  http://technet.microsoft.com/en-us/library/cc263467.aspx

Hopefully that helps.

Sincerely,
Karl

Answer 2

Hi

 

From the error messages in upgrade log, it seems that it is not able to upgrade one table which throws this error. Please first try the command below to run Post Configuration Wizard:

1.         Open a Command Prompt window and at the command prompt change to the following directory:

%COMMONPROGRAMFILES%\Microsoft shared\Web server extensions\12\Bin

2.         Type the following command:

psconfig -cmd upgrade -inplace b2b -wait –force

3.         Reset IIS

 

If it cannot resolve the issue, collect the error messages in event viewer and answer the questions below:

1.         What operation system do you use on the server? If you use Win Server 2003, did you install SP2?

2.         Did you do any customization on the site?

3.         Did you extend any application pool?

 

Hope this helps.

 

Lu Zou

                    

Answer 3

Hi Shamanovsky Victor,

I have the exact same problem since I installed the "April Cumulative Update"!  Have you figured out a solution for this one?  Could it be bad updates of stored procedures in the DB?


Hi Par4theChords,

I was updating a SharePoint SP2 (12.0.0.6421) with the "April Cumulative Update" (12.0.0.6504) when the problem started.


Hi Lu Zou,

I have tried your solution but still got the problem.  Anything else I could try?


Thanks all!

Answer 4

Hi all,

I have decided to detach and attach every content DB.  The first detach went well then the first attach gived this:

"The attach operation cannot continue because another object in this farm already contains the same ID. Each object in a farm must have a unique ID. In order to proceed with the attach operation you must assign a new ID to this database. To attach this database with a new ID, use the "stsadm.exe -o addcontentdb" operation with the -assignnewdatabaseid parameter. Note that if this new database and an existing database contain the same site collections, attaching this database will likely result in orphaned site collections due to conflicts between the two databases."

I even tried the "stsadm.exe -o addcontentdb" operation with the -assignnewdatabaseid parameter without any success.

Looks like the detach didn't work properly...

Answer 5

BTW everytime I try to run the configuration wizard it ends with the System.NullReferenceException.

Answer 6

Hi all!

We tried to find the solution to this but to no avail. We also have the same problem but  we've noticed that we have a timer job that won't get deleted. Tried deleting it by code but it just throws an exception.

...Upgrade task SPContentDatabaseSequence

We even tried removing the problematic server from the farm but we always get  "System.NullReferenceException" when doing so.

Answer 7

Hi all!

We finally solved this problem. We took a look at production environnement database (sharepoint config) and found that it was missing the 'fn_GetChildren' function ( in SQL Server management studio:  \\Sharepoint config DB\Programmability\Functions\Table-valued functions \fn_GetChildren ). We had to recreate the function using either our staging evironnement version or the one in the CONFIGDB.SQL script ( C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\SQL ). We were able to complete the installation afterwards with the "psconfig -cmd upgrade -inplace b2b -wait –force" command line as mentionned by Lu Zou.

Cheers!

Ex.:

CREATE FUNCTION dbo.fn_GetChildren
(
    @ParentId uniqueidentifier
)
RETURNS @Children TABLE (ChildId uniqueidentifier)
AS
    BEGIN
    INSERT INTO @Children (ChildId) VALUES (@ParentId)
    WHILE (@@ROWCOUNT <> 0)
    BEGIN
        INSERT
            @Children
        SELECT
            Id
        FROM
            Objects WITH (UPDLOCK)
        INNER JOIN
            @Children
        ON
            Objects.ParentId = ChildId
        WHERE
            Id NOT IN (SELECT ChildId FROM @Children)
    END
    RETURN
    END
GO
IF EXISTS (SELECT * FROM SYSOBJECTS WHERE NAME = 'trigger_OnDeleteObjects'
    AND TYPE = 'TR')
DROP TRIGGER trigger_OnDeleteObjects
GO
CREATE TRIGGER trigger_OnDeleteObjects ON Objects FOR DELETE
AS
    INSERT INTO
        Tombstones (Id)
    SELECT
        Id
    FROM
        Deleted  
GO

Answer 8

Hi hazor,

Your solution worked like a charm!  When we installed the "april cumulative update" we ran out of disk space on the C drive.  Looks like this cause the update to not install the dbo.fn_GetChildren fonction in the config database...

Has any one of you ran out of disk space during the update like us?