locked
The FastCGI process exited unexpectedly RRS feed

  • Question

  • User-827550753 posted

    Yet another in a long line of users who are having issues running FastCGI under IIS6.

     I have been receiving this error intermittently during production use and paired with the fact that PHP is no longer going to support the ISAPI extension really gives me no choices. FastCGI has errors more than ISAPI does. I'd love to stick with the 5.2 branch of PHP, however reporting bugs result in them saying they are not going to fix it and I should use FastCGI.

    It is that same negative number error code that is all over this forum. I've tried both NTS and TS builds of 5.3 and all the 5.2.x versions, different snapshots that I've got multiple directories on my drive of, but I still get this darned error. Permissions are fine, the site works great under low load, but once it gets high, what appears to be 1,700 database queries a second (120 connections/s), it just randomly fails on me. It eventually comes back, but the error happens too often to be useful. I've tried 1,000 instance max requests and fcgi max requests of 1,100. Same problem. Changed the idle, activity and request to values in howto documents, no success. Help. For the love of dog, help.

     My fcgiext.ini

     [Types]
    php=PHP
    MaxInstances=300
    IdleTimeout=300
    ActivityTimeout=300
    RequestTimeout=300
    InstanceMaxRequests=10000
    EnvironmentVars=PHP_FCGI_MAX_REQUESTS:15000
    QueueLength=2000

    [PHP]
    ExePath=c:\php\php-cgi.exe

     

    If this is a FastCGI bug, would someone explain to me how to debug on a x64 2003 Server? There is a link to Rick Jame's blog post or something, but that is for IIS7, which I find to be a mess of icons. If there is a Microsoft dev here who works on this and wants get on the server to debug, I'd be willing to give access to it for testing. This is really annoying me. I am very, very tired.

    Wednesday, October 7, 2009 8:58 PM

All replies

  • User97111691 posted

    FastCGI is the recommended SAPI, by PHP.net and Microsoft.

     About your bug, it is hard to help without knowning what you do. Do you have a script to reproduce it? Or the function where it crashes?

    Thursday, October 8, 2009 4:40 AM
  • User-1405480850 posted

    Hi,

    I am not sure if your FastCGI is configured correctly. Are you getting this error for all the PHP pages. I mean even simple phpinfo() page is not working correctly. Please follow my blog post at http://blogs.iis.net/donraman/archive/2009/10/07/installing-php-on-windows.aspx and configure PHP accordingly. It will work for sure.

    Thanks,

    Don.

    Thursday, October 8, 2009 4:58 AM
  • User-827550753 posted

     I am unable to reproduce it willingly. It just happens. In my error log it complains about __autoload() being nested too deeply and it seems to reload our autoload require over and over again causing that error to be logged. There is no single line of code that I could give you that returns a problem, if I had a line, then I wouldn't be here crying over the problems I've been having. All I have is a server which handles close to 4,000 connections at any given moment. This is running off one machine.

     FastCGI works great, until it just stops and starts exiting unexpectedly. I've accessed every page possible by a user and admin alike and it does not suddenly kick out an error to me. htmlentities complain about nonbyte characters or something and then randomly mysql real escape string errors show up because MySQL stopped responding to queries. I guess an active connection is required to MySQL to use that function.

    It complains autoload.php:2 about line 8 which is the } about it being nested too deep.

    2:  function __autoload($class)
    3:   {
    4:        if(strpos(strtolower($class),'exception') !== false)
    5:           require "exceptions/$class.exception.php";
    6:        else
    7:          require "classes/$class.class.php";
    8:    }

     

    *edit*

    Invalid multibyte sequence in argument for htmlentities

     

    *Edit again*

    http://go.microsoft.com/?linkid=9671294

    I ran and installed that. I set up my configuration manually based on how to information on this site.

    Thursday, October 8, 2009 11:55 AM
  • User-1405480850 posted

    Hi,

    'Nesting level too deep' is an error which you get when a function is called recursively and is not able to exit. The reason you can have this might be because in your application you are having some kind of circular dependency in the way you are auto loading the classes. I mean if you have class 'A' which is trying to auto load class 'B' and vice-versa effectively running this function infinitely.

    I would suggest to have a print/echo statement in both 'if' as well as 'else' clause printing the name of the class getting auto loaded and find the culprit set of classes creating the circular dependency. Once you detect the loop try to break it.

    This is my best guess. Again this seems like an application error which you will need to debug and fix it yourself.

    Thanks,

    Don.

    Thursday, October 8, 2009 6:16 PM
  • User-827550753 posted

    Thanks for your response. My application does not call __autoload more than once and there is no class a calling class b which calls class a scenario. I've accessed every page available to the administrators and users, the problem does not appear. Insert concurrent load to the server and the problem eventually pops up for short periods of time.

     I wish to stress this point: This problem only presents itself under heavy load. Sythetic traffic does not seem to get the problem to appear, possibly because it isn't simulated long enough, but real life does since it never ceases.

    Thursday, October 8, 2009 6:48 PM
  • User-827550753 posted

    Just received the line again.

    [08-Oct-2009 22:03:30] PHP Fatal error:  Cannot redeclare __autoload() (previously declared in D:\site\autoload.php:2) in D:\site\autoload.php on line 8

    If I'm not mistaken, this means that it is loading autoload again on line 8 of autoload.php?

    Thursday, October 8, 2009 7:32 PM
  • User-1405480850 posted

    Yes, that's correct. You can use some kind of condition to check if the file is already included just return. This can be achieved in multiple ways,

    • You can define a constant and each time file is included check if the constant is defined or not using 'defined' keyword. if it is not defined include else skip.
    • Another alternative can be using function_exists to check the existence of function.

    And I am sure there are other ways of doing it.

    Thanks,

    Don.

    Thursday, October 8, 2009 7:58 PM
  • User-827550753 posted

    I am now throwing exceptions when a file isn't correctly loaded. Since adding code to do that, I have not received a single error complaining about a class not there. Not a single exception. Go figure.

    I am still getting the fastcgi process exited unexpectedly when putting the site through a normal http stress test. We load the same script over and over again simulating 500 concurrent users for 40K requests. The requests are successful, however, eventually FastCGI seems to just tell the people who are on the queue or tried to connect to a closing process that it has exited unexpectedly.

     Is there some list of PHP functions that may cause the whole FastCGI  process to be forced to exit?

    Tuesday, October 13, 2009 5:40 PM
  • User-1405480850 posted

    Hi,

    There is no fix set of functions which will make FastCGI quit. And if I am right, same request is working fine in normal load. It's only that on a stress environment FastCGI is exiting. Right? This means application is okay from functionality point of view. Can you tell me what version of FastCGI you are using and on what OS?

    Thanks,

    Don.

    Tuesday, October 13, 2009 10:42 PM
  • User-827550753 posted

    Correct. The FastCGI stuff works fine for a while, but once real load starts pouring in, it eventually starts giving me that error. Not ALL the time, but enough to get emails about it.

     

     7.5.7232.0 (fbl_srv_iis_dev(ksingla).090202-1706)

    Which should be the one linked in this thread: http://forums.iis.net/t/1158720.aspx


    Windows Server 2003 Enterprise X64 Service Pack 2

    Dual Quad cores at 2.33GHz

    20GB RAM

    Tuesday, October 13, 2009 11:09 PM
  • User-1653247517 posted

    Problem is probably in PHP. Looks like php-cgi.exe is crashing under certain conditions. Is it possible to get a crash dump?

    Wednesday, October 14, 2009 12:38 AM
  • User-827550753 posted

    I've never been able to capture one so it isn't currently possible.

    Wednesday, October 14, 2009 1:38 AM
  • User-827550753 posted

    Would you be able to explain, in depth or at least link me to a resouce, that will allow me to gain a dump for your analysis? It needs to work for 64bit PHP.

    The current PHP debug process does not cover this and only the 32 bit program way.

    Sunday, October 25, 2009 1:15 AM
  • User-1653247517 posted

    You can use either adplus or debugdiag to collect a dump easily. This article explains how to use adplus. DebugDiag and DebugDiag whitepaper can be downloaded from here. php.net has this article on how to generate the backtrace for PHP which will be useful to generate a stack trace.

    Tuesday, October 27, 2009 4:16 PM
  • User-827550753 posted
    The process described in the PHP article using the tool they recommend does not work for me  using 64bit PHP. DebugDiag does not work for me. Never tried adplus.
    Tuesday, October 27, 2009 5:43 PM
  • User-1653247517 posted

    Oh. Didn't notice that you are running PHP 64-bit. 64-bit PHP is not supported yet by PHP community and is expected to have some issues.Given that PHP with FastCGI only process one request at a time, you don't get any benefit by running 64-bit PHP. If a process doesn't utilize the larger virtual memory available to 64-bit processes, its 32-bit version performs better even on x64 machines. I would suggest you to use 32-bit PHP and see if problem goes away.

    Thanks,
    Kanwal

    Tuesday, October 27, 2009 6:12 PM
  • User-827550753 posted

    Ok, I've switched between using FastCGI, both 32bit and 64 bit so often in the past 4(?) months that I am now at a complete loss as of what I am currently running.

    7.5.7232.0 (fbl_srv_iis_dev(ksingla).090202-1706)

     PHP 5.2.11.11 (Process tab says *32, so I would guess either FastCGI is running in 32 bit mode using the 64 bit extension, or I am running the 32bit version of fastcgi.)

     I am fairly certain I used the 64bit version of 5.3 from the dev with no success.

    Tuesday, October 27, 2009 10:26 PM
  • User-1653247517 posted

    You can use 32-bit or 64-bit version of FastCGI but you should always use 32-bit PHP. If you are seeing php-cgi*32 in task manager then you are definitely using 32-bit PHP which is correct. You should be able to follow steps on php.net to generate a backtrace using debugdiag. Can you try that?

    Wednesday, October 28, 2009 2:08 PM
  • User-827550753 posted

     First chance exception - 0x000006c5

     Are these bad at all to have?

     

    [11/1/2009 9:40:49 PM] First chance exception - 0x000006c5 caused by thread with system id 18920
    [11/1/2009 9:40:49 PM] Stack Trace
    ChildEBP RetAddr  Args to Child              
    00c0c49c 7da4a631 000006c5 00000001 00000000 kernel32!RaiseException+0x53
    00c0c4b4 7da3d92c 000006c5 76ed44e5 00c0c6c4 RPCRT4!RpcpRaiseException+0x24
    00c0c4dc 7da3d765 00c0c6c4 76ed43ae 0011d1e8 RPCRT4!NdrpUnionMemorySize+0x1f9
    00c0c4f0 7da41ec8 00c0c6c4 76ed4307 0011d19c RPCRT4!NdrNonEncapsulatedUnionMemorySize+0x2b
    00c0c53c 7da41f5e 00c0c6c4 76ed44f0 00c0cabc RPCRT4!NdrComplexStructMemorySize+0x364
    00c0c58c 7da38bfb 00c0c6c4 00c0cabc 76ed44dc RPCRT4!NdrComplexStructUnmarshall+0xd3
    00c0c5b8 7da38c6c 00000000 76ed44d4 00c0cabc RPCRT4!NdrpPointerUnmarshall+0x216
    00c0c5d8 7da38bfb 00c0c6c4 00c0cabc 76ed43a0 RPCRT4!NdrPointerUnmarshall+0x30
    00c0c604 7da38c6c 00000000 76ed43a0 00c0cabc RPCRT4!NdrpPointerUnmarshall+0x216
    00c0c624 7da37d85 00c0c6c4 00c0ca94 76ed439c RPCRT4!NdrPointerUnmarshall+0x30
    00c0c67c 7dac01c9 00c0c6c4 00c0c7c4 00000000 RPCRT4!NdrpClientUnMarshal+0x122
    00c0ca64 76ed5049 76ed42d8 76ed421c 00c0ca84 RPCRT4!NdrClientCall2+0x2dd
    00c0ca7c 76ed4f69 00000000 02491f8c 00000001 DNSAPI!R_ResolverQuery+0x1c
    00c0cad8 76ed506f 02491f8c 00000001 14000040 DNSAPI!Query_PrivateExW+0x187
    00c0cafc 7db442af 02491f8c 00000001 14000040 DNSAPI!DnsQuery_W+0x3a
    00c0cb2c 7db4417b 02491f8c 00000001 14000040 mswsock!SaBlob_Query+0x2d
    00c0cb70 7db4406c 00000000 01c842c0 01c842a8 mswsock!Rnr_DoDnsLookup+0xf0
    00c0ce08 71c06dc0 02491f28 00000000 00c0cebc mswsock!Dns_NSPLookupServiceNext+0x24b
    00c0ce20 71c06da0 01c84370 02491f28 00000000 WS2_32!NSPROVIDER::NSPLookupServiceNext+0x17
    00c0ce3c 71c06d6a 01c84188 00000000 00c0cebc WS2_32!NSPROVIDERSTATE::LookupServiceNext+0x1c
    00c0ce68 71c06d08 01c842c0 00000000 00c0cebc WS2_32!NSQUERY::LookupServiceNext+0xae
    00c0ce88 71c078ee 01c842a8 00000000 00c0cebc WS2_32!WSALookupServiceNextW+0x78
    00c0ceac 71c07848 01c842a8 00000000 0000013c WS2_32!WSALookupServiceNextA+0x63
    00c0ced8 71c07d2e 00c0cf08 0000013c 01d0c690 WS2_32!getxyDataEnt+0xa1
    00c0d114 022bfe86 01d0c690 01d0c690 0229d060 WS2_32!gethostbyname+0xb4
    WARNING: Stack unwind information not available. Following frames may be wrong.
    00c0d120 0229d060 01d0c690 00c0d15c 00c0d230 LIBMYSQL!mysql_sqlstate+0x2b5ce
    00c0d190 7d62305a 00000000 0251c400 0000002c LIBMYSQL!mysql_sqlstate+0x87a8
    003e0600 00000000 7d6a0a80 ffffffff 00000000 ntdll!RtlReAllocateHeap+0xdef

     

    Sunday, November 1, 2009 10:47 PM
  • User-827550753 posted

    Bump for a push for more information.

    Is there anything else required or is this trace supplying incorrect information?

    Friday, November 13, 2009 4:56 PM
  • User-827550753 posted

    Well, we have managed to get rid of this issue by not using the GD dll that PHP supplies and switch over to imagemagick. Now the only errors I get are during file uploads where the script times out.

    Saturday, December 12, 2009 11:52 PM
  • User-1405480850 posted

    Hi,

    You may need to increase max_execution_time in your PHP INI file to ensure that upload functionality gets a good amount of time to do the operation. You should also ensure that FastCGI should not time out by setting a higher ActivityTimeout value.

    Thanks,

    Don.

    Sunday, December 13, 2009 12:30 PM
  • User-827550753 posted

    Obviously. I set timeouts to 20 minutes at all the places I could find them. Still receiving this one problem, but at least it isn't affecting everyone.

    Monday, December 14, 2009 2:29 AM