Expression Encoder 4 : Out Of Memory Exceptions with Expression Encoder when running as a Windows Service

Question

Hello,

I've been experiencing a problem with the Expression Encoder 4 engine from the SDK, as wrapped into in a Windows Service for post processing files on a server. I'm getting errors / crashes with the error/exception messages included bellow.

My Source is based on AVISYNTH files (AVS), which are sourcing 2 WMV files and creating a PIP composite image.

Now the interesting part is that if I run these files within the Expression Encoder 4 GUI, I don't have the problem, and the encoding are capable of processing through. I'm actually using the same Encoding profile in both cases, as to make sure it's not an issue there.

There is definitively a noticeable difference in the amount of processor / memory the application can access when running as a service or as an interactive GUI based application. Any suggestions on how to best address this ?  I'm assuming this has something to do w/ the Treading of the EncoderEngine request, which I simply instantiated from SDK samples. Any suggestion on how to address this would be sincerely appreciated, since my encoder needs to run in the background as a Windows Service and should actually be able to run multiple instances at the same time.

I know the IIS Transform Manager uses the Expression Encoder 4 engine, and have not had a chance to try my AVS files there, but maybe someone from that team can suggest best way to use it as wrapped in a service ? I can't move to IIS Transform Manager at this time since our back end processing does quite a bit more processing types jobs, but maybe there is something in the way the encoder engine / thread / job is created and called which lets it run without these crashs when in a Windows Service.

Included bellow is the sample code requesting the encoding of the file and the EncodeThreadCall

 

EDIT TO INCLUDE CONTEXT from README on posting bugs

 

A.    Version and SKU of Expression Encoder: Expression Encoder 4 Pro

B.    OS used and whether it's 32 or 64bit: 2008 Server R2, 64bit

C.    If using a device, the model and version of the drivers. none / file

D.    If using a source file, file type and video and audio codecs. File AVS (avisyth), sourcing WMV through Directshow

E.     If 3rd party codecs are used to deal with this source, the name and version of each of them.
AVISYNTH, 2.58, using DirectShowSource.dll,  2.5.8.6, Avisynth DirectShow Reader pluggin

F.     If specific encode settings are required, the job or the preset. Standard WMV9, WMA profile, Source Width/Height

 

Public Sub EncodeFile(ByVal pInputFile As String, ByVal pOutputFile As String, Optional ByVal pProfile As String = "DEFAULT")
	Me.InputFileName = pInputFile
	Me.profileUsedToEncode = pProfile
	Me.OutputFileName = pOutputFile
	' Create a thread to do the encode.
	Me.encodeThread = New Threading.Thread(New Threading.ThreadStart(AddressOf EncodeThreadCall))
	Me.encodeThread.Start()
End Sub


Private Sub EncodeThreadCall()

Try
    ' Create the media item.
    Dim mediaItem As MediaItem
    Try
	mediaItem = New MediaItem(Me.InputFileName)
    Catch exp As InvalidMediaFileException
	'IndicateEncodingIsFinished(exp.Message)
	RaiseEvent EncoderError(Me, exp.ToString)
	Return
    End Try

    mediaItem.OutputFormat = New WindowsMediaOutputFormat()

    If profileUsedToEncode = String.Empty Then
	Dim preset As Preset = preset.FromFile("default.PRX.xml")
	mediaItem.ApplyPreset(preset)
    Else
	Dim preset As Preset = preset.FromFile(profileUsedToEncode)
	mediaItem.ApplyPreset(preset)
    End If

    ' Create the job, add the media item and encode.
    Using job As New Job()
	If System.IO.Path.IsPathRooted(OutputFileName) Then
	    mediaItem.OutputFileName = System.IO.Path.GetFileName(OutputFileName)
	    job.OutputDirectory = System.IO.Path.GetDirectoryName(OutputFileName)
	Else
	    mediaItem.OutputFileName = OutputFileName
	End If

	job.MediaItems.Add(mediaItem)
	job.JobId = Guid.NewGuid().ToString()
	job.CreateSubfolder = False

	AddHandler job.EncodeProgress, AddressOf handler_EncodeProgress
	AddHandler job.EncodeCompleted, AddressOf handler_EncodeCompleted
	job.Encode()

    End Using

Catch ex As Exception
    RaiseEvent EncoderError(Me, ex.ToString)
End Try

End Sub
    

The following are the errors I am logging:

 

MyExpressionEncoder_EncoderError : Microsoft.Expression.Encoder.EncodeErrorException: Not enough memory to complete requested operation. ---> Microsoft.Expression.Encoder.EncodeErrorException: Not enough memory to complete requested operation. ---> Microsoft.Expression.Encoder.UnableToEncodeFileException: Insufficient memory to continue the execution of the program. ---> System.OutOfMemoryException: Insufficient memory to continue the execution of the program. at System.Runtime.InteropServices.Marshal.ThrowExceptionForHRInternal(Int32 errorCode, IntPtr errorInfo) at System.Runtime.InteropServices.Marshal.ThrowExceptionForHR(Int32 errorCode) at ThrowHRESULT(Int32 hr) at MS.Internal.Expression.Encoder.Media.DShowEncode.Encode(EncodeSettings settings) --- End of inner exception stack trace --- at MS.Internal.Expression.Encoder.Media.DShowEncode.Encode(EncodeSettings settings) at Microsoft.Expression.Encoder.EncoderBase.EncodeFile(EncodeSettings settings) at Microsoft.Expression.Encoder.MediaItemEncoder.CreateMediaFiles(PublishedItem item, List`1& rgScripts, PartualRebuildRulesItem rebuildItem, TimeSpan& markerOffset, String[]& manifestFiles, SimpleEncodeOptions[]& rgEncodeOptions, OutputFormatType& outputFormat) at Microsoft.Expression.Encoder.MediaItemEncoder.EncodeItem(MediaItem item, PublishedItem& publishedItem) --- End of inner exception stack trace --- --- End of inner exception stack trace --- at Microsoft.Expression.Encoder.MediaEncoder.Encode() at Microsoft.Expression.Encoder.Job.Encode() at COOLService.ExpressionEncoderWrapper.EncodeThreadCall()

MyExpressionEncoder_EncoderError : Microsoft.Expression.Encoder.InvalidMediaFileException: An unknown error has occurred ---> Microsoft.Expression.Encoder.UnableToAnalyzeFileException: Error HRESULT E_FAIL has been returned from a call to a COM component. ---> System.Runtime.InteropServices.COMException: Error HRESULT E_FAIL has been returned from a call to a COM component. at System.Runtime.InteropServices.Marshal.ThrowExceptionForHRInternal(Int32 errorCode, IntPtr errorInfo) at System.Runtime.InteropServices.Marshal.ThrowExceptionForHR(Int32 errorCode) at ThrowHRESULT(Int32 hr) at MS.Internal.Expression.Encoder.Media.MediaFileInfo..ctor(String fileName) --- End of inner exception stack trace --- at MS.Internal.Expression.Encoder.Media.MediaFileInfo..ctor(String fileName) at MS.Internal.Expression.Encoder.MediaImport.MediaInfoCache`1.CreateMediaFileInfoFromFile(String strFilename) at MS.Internal.Expression.Encoder.MediaImport.MediaInfoCache`1.OpenMediaFileInfo(String strFilename) at Microsoft.Expression.Encoder.MediaImportInfo.OpenMediaFileInfo(String strFilename) at Microsoft.Expression.Encoder.MediaImportInfo.OpenAndValidateMediaFile(String strFilename, MediaLoadOptions loadOptions) --- End of inner exception stack trace --- at Microsoft.Expression.Encoder.MediaImportInfo.OpenAndValidateMediaFile(String strFilename, MediaLoadOptions loadOptions) at Microsoft.Expression.Encoder.MediaItem.Analyze(AudioVideoFile audioVideoFile) at Microsoft.Expression.Encoder.MediaItem.Analyze() at Microsoft.Expression.Encoder.MediaItem..ctor(String fileName) at COOLService.ExpressionEncoderWrapper.EncodeThreadCall()

Any suggestion would be greatly appreciated.

 

Answer 1

It could be that your SDK app is just running out of memory. By default an SDK app just has access to 2GB of memory whereas the UI will have access to 3GB as we set the large address aware flag. You could try setting this flag on your SDK app by using EDITBIN and see if that helps. See the following link for more details on the flag.

http://msdn.microsoft.com/en-us/library/203797te.aspx

Regards

---

Dean

Answer 2

FWIW, I wrote a blog entry a while ago about the LARGEADDRESSAWARE flag and it recommended use with Expression Encoder: http://blogs.msdn.com/b/expressionencoder/archive/2010/03/10/9976633.aspx

 

Answer 3

Thanks, that helped me - to a point. I have a similar problem as the original poster. I'm using the Encoder SDK in a C# app to transcode a collection of MPEG-2 TS files to a single H264 .ismv file. Each of my source files is 48 MB in size, representing 58 seconds of video, and I have 175 of them.

Before I found the editbin trick, my app was dying as it had allocated around 1,063,000 K of memory (as seen in the Windows task manager). The exception was "Not enough storage is available to process this command" and came from DShowEncode.Encode(EncodeSettings). After applying the /LARGEADDRESSAWARE flag, my app runs until it's allocated 2,382,796 K of memory, at which point it abruptly dies without even throwing an exception. Only 65% of my physical memory is in use when this happens.

Help? Suggestions?

Peter

Answer 4

There is a limit on the number of clips you can merge together within a media item in Expression Encoder. More information is available on multiple thread on this forum (here is an example). I suggest you merge multiple items, in bunches of 8-10 and merge those outputs via Smart Encode in a secend step, which will minimize re-encoding. Otherwise, you may have to use a 3rd party tool to merge your TS sources together before using them within Encoder.

Answer 5

Nic,

 

It seems that you and I are the only ones having this problem or so it seems.  My app is not running as a service but instead it is a vb.net windows forms SDK app. I am trying to convert DVD VOB files to WMV files. When I run it on my Sony laptop it dies part way thru with an "Out of Memory". However, when I run it on my development machine it never fails. I have activated the large address space but it did not help and both machines ar 8gb 64 bit machines. Like you it will run successfully in the GUI.

If you were able to resolve this would you mind sharing with me your solution?  If you were not able to share this, I would like to know that too since I'm getting ready to open a ticket with MS.

 

Thank you,

 

Vic

 

 

---

Visual Basic; Access; PHP; SQL Server; MySQL Developer

Answer 6

Hi

I am also expeiencing the same issue with the Epression Encoder V4 when using the SDK from a Windows service. It runs for a day or so converting WAV files to WMA format before throwing an out of memory exception. The service process memory usage is stable at about 120MB and the server has over 4GB of RAM available.

If anyone has any suggestions as to how to resolve this they would be greatfully received.

Regards

Steve

Answer 7

Hi

Further analysis seems to point to a bug in the encoder causing it to die after having encoded a large number of files. I ran two tests.

1. Converts the same file repeatedly. This run fine and has so far done over 200000 transcodings.

2. Copies the same file to an incrementing file name then converts it. This application consistently fails with an exception raised from native code on the 16374th conversion.

The test code is below.

using System;
using System.IO;
using System.Threading;
using Microsoft.Expression.Encoder;
using Microsoft.Expression.Encoder.Profiles;

namespace ExpressionTestConsole
{
    class Program
    {
        static void Main(string[] args)
        {
            string folder = Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location);
            string testfile = Path.Combine(folder, @"Test.WAV");
            int id = 1;
            string infile = Path.Combine(folder, id.ToString() + ".WAV");
            string outfile = null;

            try
            {
                for (id=1; id<10000000; id++)
                {
                    infile = Path.Combine(folder, id.ToString() + ".WAV");

                    if (string.IsNullOrEmpty(outfile))
                        File.Copy(testfile, infile);
                    else
                        File.Move(outfile, infile);

                    using (var job = new Job())
                    {
                        var mediaItem = new MediaItem(infile);

                        mediaItem.OutputFormat = new WindowsMediaOutputFormat { AudioProfile = new WmaAudioProfile(WmaAudioProfile) };
                        mediaItem.OutputFileName = string.Format("{0}.wma", id);

                        job.MediaItems.Add(mediaItem);
                        job.OutputDirectory = folder;
                        job.CreateSubfolder = false;

                        job.Encode();

                        Console.WriteLine("Job='{0}'; ID='{1}'; ConvertedFile='{2}'", job.JobId, id, mediaItem.ActualOutputFileName);
                        outfile = mediaItem.ActualOutputFileFullPath;
                    }

                    Settings.DeleteMediaCache();

                    File.Delete(outfile);
                    outfile = infile;
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.ToString());
            }

            Console.ReadLine();
        }

        private static WmaAudioProfile _wmaAudioProfile = null;
        private static WmaAudioProfile WmaAudioProfile
        {
            get
            {
                if (_wmaAudioProfile == null)
                {
                    _wmaAudioProfile = new WmaAudioProfile();
                    _wmaAudioProfile.Codec = AudioCodec.Wma;
                    _wmaAudioProfile.Channels = 1;
                    _wmaAudioProfile.BitsPerSample = 16;
                    _wmaAudioProfile.SamplesPerSecond = 8000;
                    _wmaAudioProfile.Bitrate = new ConstantBitrate(5, false);
                }
                return _wmaAudioProfile;
            }
        }
    }
}

Answer 8

I'm seeing the same thing as steve pitt. I'm getting exceptions after ~16K encodings. Did anyone ever find a solution/workaround for this?