Which version of SSRS are you using? If it is SSRS2005, since the SSRS2005 engine processes the whole report at once, it will run into resource constraints for reports having huge amount of data. In 2008, there have been quite a few scalability enhancements through on-demand procesing and it will be less likely to hit OOM exception.
http://blogs.msdn.com/robertbruckner/archive/2008/07/20/Scale-up-improvements-in-reporting-services-2008.aspx
http://blogs.msdn.com/robertbruckner/archive/2008/08/11/on-demand-report-processing-in-rs-2008.aspx