locked
SSRS 2017 Error 503 RRS feed

  • Question

  • Upon a server reboot on Monday, I have been unable to have a working SSRS instance.   All I get is Error 503 on both Web Service URL and Web Portal URLs.  The server reboot on Monday was due to May patches being installed, however, I am unsure if there is a link to those patches, or if it was just because the server rebooted.   The website was working great before then with zero downtime issues.   Here is a list of things I have tried and a copy of the error logs:

    1. Restart of SSRS Service

    2. Reboot Server

    3. Uninstall .net patch from May and reboot

    4. Download and install .net framework 4.8

    5. Download latest SSRS 2017.  Now version 14.0.7393.5407 Reboot

    6. Remove web portal URL and Web Service URL and add back again.

    7. Change service account from virtual service account to network service and back while testing between each change.

    8. Add service account to local admin group

    9. Verify firewall port rule in effect to allow port 80, 443, 1433, 1434.

    10. Try to setup on alternate VM, however license is SQL Server Standard, which does not allow scale-out. 

    11. Look at error logs for additional clues.  Nothing seems obvious as the log is quite limited. 

    12. Uninstall remaining May 2020 patches, reboot.

    Other information to note about this server is that it is also running IIS and those websites are not affected.  They work perfectly fine.  It is just the SSRS.  Any help on how to trace the issue further is greatly appreciated.  This is installed on windows 2012 R2 OS.

    Here is the log files:

    RSHostingService_2020_06_04_13_44_22.log

    2020-06-04 13:44:22.1220|INFO|1|File Logger created: 'D:\Program Files\Microsoft SQL Server Reporting Services\SSRS\LogFiles\RSHostingService_${DailyDate}.log' - level Info, will roll at 32 Mb, process id 12148
    2020-06-04 13:44:22.1376|INFO|1|CFG : WebApiMaxWaitMS = [1000] (default)
    2020-06-04 13:44:22.1376|INFO|1|CFG : WebApiBackOffExpireSeconds = [60] (default)
    2020-06-04 13:44:22.1376|INFO|1|CFG : WebApiInitialWaitMS = [10] (default)
    2020-06-04 13:44:22.1376|INFO|1|CFG : WebApiBackoffMultiplier = [2] (default)
    2020-06-04 13:44:22.1376|INFO|1|CFG : ServiceWatchdogIntervalInSeconds = [10] (default)
    2020-06-04 13:44:22.1376|INFO|1|Current Directory : D:\Program Files\Microsoft SQL Server Reporting Services\SSRS\RSHostingService
    2020-06-04 13:44:22.3251|INFO|5|Updating application config files
    2020-06-04 13:44:22.3251|INFO|5|Syncing service machine key
    2020-06-04 13:44:22.3720|INFO|5|CFG : MaxWaitMinutes = [60] (default)
    2020-06-04 13:44:22.3720|INFO|5|CFG : BackOffExpireMinutes = [60] (default)
    2020-06-04 13:44:22.3720|INFO|5|CFG : InitialWaitSeconds = [30] (default)
    2020-06-04 13:44:22.3720|INFO|5|CFG : BackOffMultiplier = [2] (default)
    2020-06-04 13:44:22.3876|INFO|5|Launching Process : Management Service
    2020-06-04 13:44:22.3876|INFO|5|Starting Process: D:\Program Files\Microsoft SQL Server Reporting Services\SSRS\Management\RSManagement.exe

    RSManagement_2020_06_04_13_44_22.log

    2020-06-04 13:44:22.6063|INFO|1|File Logger created: 'D:\Program Files\Microsoft SQL Server Reporting Services\SSRS\LogFiles\RSManagement_${DailyDate}.log' - level Info, will roll at 32 Mb, process id 11036
    2020-06-04 13:44:22.7157|INFO|1|CFG : Dumper.path = [..\LogFiles\] (env)
    2020-06-04 13:44:22.7157|INFO|1|CFG : Name = [] (env)
    2020-06-04 13:44:22.7157|INFO|1|CFG : Dumper.flags = [SendToWatson, AllThreads, AllMemory] (env)
    2020-06-04 13:44:22.7157|INFO|1|CFG : Dumper.preventIfContains = [Microsoft.BIServer.HostingEnvironment.Exceptions.TrustedProcessTokenExpiredException] (env)
    2020-06-04 13:44:22.7157|INFO|1|Do not dump on: Microsoft.BIServer.HostingEnvironment.Exceptions.TrustedProcessTokenExpiredException
    2020-06-04 13:44:22.7157|INFO|1|CFG : listenerUrl = [http://+:8082/] (env)
    2020-06-04 13:44:22.7157|INFO|1|CFG : rsConfigFilePath = [..\ReportServer\rsreportserver.config] (env)
    2020-06-04 13:44:22.7313|INFO|1|SKU: SQL Server Standard
    2020-06-04 13:44:22.8563|INFO|1|The web server needs to run elevated
    2020-06-04 13:44:22.8563|INFO|1|Web Server is running.



    • Edited by jarnaud1 Thursday, June 4, 2020 7:26 PM Add OS info.
    Thursday, June 4, 2020 7:16 PM

All replies

  • Hi jarnaud1,

    Have you changed the rsreportserver.config file manually? Incorrect changes to rsreportserver.config may cause HTTP error 503.

    You can try to remove all the urls for web service and web portal in Reporting Services Configuration Manager, and manually edit the rsreportserver.config file and removing any entries between the <url></url> tags.Then restarting the SSRS service, and re-adding the URL’s with a fresh port number.

    In addition, please remember to back up rsreportserver.config before changing it.

    Please refer to the following articles which might help:

    How to fix problem 503 service unavailable in SSRS ?

    SSRS error HTTP Error 503. The service is unavailable

    Best  Regards,

    Amelia


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Friday, June 5, 2020 6:25 AM
  • I have not modified the rsreportserver.config file.  However, I did confirm when removing the URL's via the configuration manager, that it did remove all URLs in the config file.  So, that part of the configuration manager seems to be working. 

    I should add that I did modify the ReportingServicesService.exe.config file by adding the http:4 tag to the RSTrace section as shown below.  However, when trying to connect a browser to SSRS, it does not generate any additional log files. 

    <RStrace>
        <add name="FileName" value="ReportingServicesService_" />
        <add name="FileSizeLimitMb" value="32" />
        <add name="KeepFilesForDays" value="14" />
        <add name="Prefix" value="appdomain, tid, time" />
        <add name="TraceListeners" value="file" />
        <add name="TraceFileMode" value="unique" />
        <add name="Components" value="all:3,http:4" />
      </RStrace>

    As a result, I don't think requests are making it to the reporting service.

    Friday, June 5, 2020 12:19 PM
  • Hi jarnaud1,

    >>However, when trying to connect a browser to SSRS, it does not generate any additional log files.

    Adding http:4 to the RStrace section of the ReportingServicesService.exe.config file can enable the HTTP server. The following example includes all settings so that you can paste the whole section over the RStrace section, and then delete the settings you do not need:

    <RStrace>  
             <add name="FileName" value="ReportServerService_" />  
             <add name="FileSizeLimitMb" value="32" />  
             <add name="KeepFilesForDays" value="14" />  
             <add name="Prefix" value="tid, time" />  
             <add name="TraceListeners" value="debugwindow, file" />  
             <add name="TraceFileMode" value="unique" />  
             <add name="HttpTraceFileName" value="ReportServerService_HTTP_" />  
             <add name="HttpTraceSwitches" value="date,time,clientip,username,serverip,serverport,host,method,uristem,uriquery,protocolstatus,bytesreceived,timetaken,protocolversion,useragent,cookiereceived,cookiesent,referrer" />  
             <add name="Components" value="all:3,http:4" />  
       </RStrace>

    Based on my test, this works fine.

    Please refer to Report Server HTTP Log for more details.

    In addition, could you please share us the <Authentication> tag in rsreportserver.config?

    Best Regards,

    Amelia



    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Monday, June 8, 2020 8:18 AM
  • In rsreportserver.config, my Authentication settings are as follows:

    <Authentication>
            <AuthenticationTypes>
                <RSWindowsNTLM/>
            </AuthenticationTypes>
            <RSWindowsExtendedProtectionLevel>Off</RSWindowsExtendedProtectionLevel>
            <RSWindowsExtendedProtectionScenario>Proxy</RSWindowsExtendedProtectionScenario>
            <EnableAuthPersistence>true</EnableAuthPersistence>
        </Authentication>

    I have added the RSTrace section as you have suggested.   I have restarted the service, but I still do not generate any additional logs.   So, I am thinking the requests are being blocked before even connecting to the SSRS Service.  When a request comes in, what is the path that the request must process through to connect to SSRS?    It looks like the firewall has not been changed, but perhaps I am missing a port rule?  Or is there something else that could block the request?  I cannot bring up the page even on the same VM as what SSRS is running on. 

    Thanks,

    Jeff Arnaud

    Monday, June 8, 2020 8:13 PM
  • Hi Jeff Arnaud,

    Thanks for reply.

    The Report Server HTTP log file keeps a record of every HTTP request and response handled by the report server. Because request overflow and timeout errors do not reach the report server, they are not recorded in the HTTP log file.

    You could enable and review HTTP.SYS log file for information about what is causing the error.

    Please refer to HTTP 503 Service is Unavailable.

    In addition, after a lot of research, it seems uninstalling then reinstalling SSRS can get SSRS work.

    Please refer to HTTP Error 503. The service is unavailable and SSRS HTTP Error 503.

    Best Regards,

    Amelia


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Tuesday, June 9, 2020 6:55 AM
  • Amelia,

    Ok, so I reviewed the links that you had posted.   I added the registry settings and reviewed the log location, but nothing apparent is showing up.  A clip of the http log is as follows:

    2020-06-09 17:43:16 XXX.XXX.235.42 63942 XXX.XXX.13.7 443 - - - - - Timer_ConnectionIdle -
    2020-06-09 17:43:16 XXX.XXX.235.42 63943 XXX.XXX.13.7 443 - - - - - Timer_ConnectionIdle -
    2020-06-09 17:48:17 XXX.XXX.13.7 61189 XXX.XXX.13.7 443 HTTP/1.1 GET /Reports 503 - N/A -
    2020-06-09 17:49:21 XXX.XXX.13.7 61742 XXX.XXX.13.7 80 HTTP/1.1 GET /ReportServer 503 - N/A -
    2020-06-09 17:49:24 XXX.XXX.13.7 61774 XXX.XXX.13.7 443 HTTP/1.1 GET /ReportServer 503 - N/A -
    2020-06-09 17:56:03 XXX.XXX.13.7 64803 XXX.XXX.13.7 443 HTTP/1.1 GET /ReportServer 503 - N/A -
    2020-06-09 17:56:07 XXX.XXX.13.7 64843 XXX.XXX.13.7 80 HTTP/1.1 GET /ReportServer 503 - N/A -
    2020-06-09 17:56:50 XXX.XXX.235.76 55208 XXX.XXX.13.7 443 HTTP/1.1 GET /Reports/browse/ 503 - N/A -

    No other http log files are generated and there appears to be no further information. 

    So, I decided I would try messing with the report database.   First, I restored an earlier ReportServer database from when SSRS was working.   The same error 503 resulted. I then deleted the database (after a backup of course) and used the SSRS configuration manager to recreate the database.   But that now gives me the following error:

    Microsoft.ReportingServices.WmiProvider.WMIProviderException: An error occurred when attempting to connect to the report server remote procedure call (RPC) end point. Verify that the Report Server Windows service is running, and then retry the operation.

     ---> System.Runtime.InteropServices.COMException: The RPC server is not listening. (Exception from HRESULT: 0x800706B3)
       --- End of inner exception stack trace ---
       at Microsoft.ReportingServices.WmiProvider.RSWmiAdmin.ThrowOnError(ManagementBaseObject mo)
       at Microsoft.ReportingServices.WmiProvider.RSWmiAdmin.GenerateDatabaseScript(String databaseName, Int32 lcid, Boolean isSharePointIntegrated, String& script)
       at ReportServicesConfigUI.WMIProvider.RSReportServerAdmin.GenerateDatabaseScript(String databaseName, Int32 lcid, Boolean isSharePointIntegrated, String& script)

    Could it be the RPC server has been the issue all along? 

    One of my earlier troubleshooting steps was to uninstall and reinstall SSRS, but it did not work (See item 5 in OP).    I will try it one more time, because well,..  I don't have any other options at this point.   If I have success with the reinstall, I will post a followup. 

    Thanks,

    Jeff Arnaud

    Tuesday, June 9, 2020 6:26 PM
  • Ok, so I uninstalled and reinstalled SSRS and it made no difference.   I even tried recreating a new ReportServer database without success. 

    I went ahead and installed SSRS on another VM using the evaluation license and that did work.  So, for the time being, I will use the evaluation copy until we get this worked out. 

    Tuesday, June 9, 2020 8:37 PM
  • Does anybody have any suggestions on how to trace where the problem might be?  Certainly there has to be a way to figure out what is going on with this installation.  Are there any group policy settings that may break SSRS?   Any logging I can enable that would show where the problem might lie?   I still find it strange the two log files in SSRS are so short.  Working logs are usually much longer.   Would this be a clue? 

    Any suggestions would be appreciated.  I need to get this working soon.   Thanks!

    Jeff

    Monday, June 15, 2020 9:49 PM