GPU encode problem with GTX 670 and R300 drivers

Question

Hi

I am trying to set up a system for GPU encoding, and got a new NVidia GTX 670 board for the job. It all looks fine in Expression where the GPU is detected and possible to enable, but no GPU encoding is done no matter what file, source or output profile I try. I also tried the EE4GPUPerfTool but it just creashes after CPU encode when it is about to start GPU encoding with 1 stream. Any suggestions?

UPDATE: I searched through the server room and found an old Quadro FX 3800, installed NVidia driver version 296.10 and Cuda encoding works fine. Any one that are able to use the new GPUs with their respective new drivers?

Setup is SuperMicro workstation with 2x Xeon E5500 CPU 12GB RAM, Expression Encoder 4 Pro SP2

Kenneth

Answer 1

I am having the same problem with DVDFab and my new GTX 670

Answer 2

I would suggest contacting Nvidia on this one. Anything that's got a newer chip than G84 is working fine up to and including the 500 series, so I suspect they must have broken backward compatibility with their 600 series drivers.

Answer 3

Hi Just to add to your post I have a GTX 680 and I am experiencing the same issue. Purchased Expression Pro purely for CUDA encodes , not best pleased!

Answer 4

Same problem with both DVDFab and Sony Vegas Pro 11.

GPU acceleration is not recognized by either program with my GTX 670.

Must just be for gaming at the moment as 3DMark11 runs just fine.

Plugged in a Quadro 2000 and everything works.

My rig is ASUS P8-Z77-V Pro, i7 3770K, 16GB Ram, Win7-Pro

Looks like we will have to wait for new drivers from nVidia or software updates.

Answer 5

Thanks for reporting it Dan.

As I suggested above, anybody experiencing this should contact NVidia about this.

It's fine if Nvidia wants to add extra Cuda performance hooks unique to Kepler (6xx series), but it's not ok of them to break backward compatibility with released products based on an earlier version of the Cuda implementation.

Bottom line, they should be 100% backward compatible with Cuda 1.2 based applications in their Kepler drivers and from the reports I see here, they are not.

The more people contacting them on this issue, the faster there will be a resolution at the driver level, which is the right approach to solve this IMO.

Answer 6

Just to let you know - I filed a bug report to nVidia. Seems nobody else did that yet, as their tech support was unaware of the problem. This is the first message I got back:

Thank you for your submission. I've submitted a bug to our quality lab to see if they can replicate the failure with Expression Encoder 4 SP2. We have to treat each application as a separate issue because the CUDA device query is actually done at the application side. Generally the failure is at the detection level for a compatible/supported CUDA device. In many cases the application is unable to properly detect the new GPU and may need to be updated or patched properly to support the new architecture. In the case of Sony Vegas Pro 11, the application doesn't appear to be able to detect the 600 series at all and we've reported this back to Sony. Sony has provided NVIDIA with a new version of Sony Vegas Pro 12 that is able to detect and work with 600 series. We suspect Sony will need to update Vegas Pro 11 in order to be able to support 600 series. I suspect this is true for DVDFab as well since users from forums are reporting that the latest DVDFab version resolved the issue, at least for two users that have confirmed with their GTX 680.

Having said that, the next step is for our lab to try to replicate the failure. Once we are able to replicate the failure then we can escalate to development to investigate and determine where the failure is, whether it is at the application level or graphics/CUDA drivers. I'll keep you posted."

Answer 7

Hi Eric,

Your backward compatibility words are well taken.

I guess that all the dpc_watchdog_timeout error blue screens  many Windows 8 users are experiencing shows that Microsoft is also guilty of the backward compatibilty sin.  

Rovi has updated encoders that support the new cards, as the architecture updates of the hardware have dictated updated code to take advantage of the new functionality.  The 'Driver' code that you point the finger at, is actually the encoder software and not the nvidia driver. If you are suggesting that Nvidia forces the hardware to 'emulate' the prior architecture to support your outdated encoder, this is crazy, and would significatly degrade the performance.

Just as Microsoft updates Windows continuously to kill bugs and take advantage of new hardware, Microsoft is responsible to maintain their code to be able to use curently available hardware. Supporting current Nvidia hardware is the responsibility of Microsoft... I assure you that the other microsoft graphics software in windows can take advantage of newer Nvidia, AMD and Intel GPU hardware.

If you are suggesting that a few users complaining to Nvidia will encorage Nvidia to rewrite drivers, versus direct contact from Microsoft or Rovi on this matter, this is even more bizarre to me.

Beyond all the politics and BS, it would be really great just to get a straight answer on why Microsoft refuses to update one the the best encoding SDK and applications in existance.

Please feel free to contact me directly if I can help make a case for this somewhere in Microsoft....

Roger Berk

  

---

roger berk

Answer 8

This is very dissapointing

The GPU Encoding in Expression Encoder 4 Pro SP2 page does not list the Kepler architecture as a 'known limitation'.

And it recommends "...using only higher-end CUDA GPUs..." with the latest drivers.

However, if you you look at the Tech Specs tab on the Main Concept CUDA H.264/AVC Encoder SDK 2.0 page here:

http://www.mainconcept.com/products/sdks/gpu-acceleration/cuda-h264avc.html

...it states "Boards with Kepler architecture are not supported"

Eric, will you update Expression Encoder as soon as Main Concept supports Kepler?

And can you find out from Main Concept roughly when Kepler support is expeccted to be implemented?

Thanks

Answer 9

I also have an existing case open with Nvidia about this, although I've not had much more than a cursory "we're investigating" response to date. I do however note the following addition in the latest Beta driver (310.64) release notes:

NVIDIA CUDA
Includes support for applications built using CUDA 5 or earlier version of the CUDA Toolkit. More information at http://developer.nvidia.com/cuda-toolkit

Sadly, I've tried the beta drivers on my system with Expression Encoder SP2 and its still not making use of the GPU (although someone else may wish to confirm that as I've only had a quick go), but it does at least suggest that perhaps something is getting done about it....



Clive

Answer 10

I'm unable to get CUDA encoding to work either.  I have a GTX 650 Ti (768 cores, 2 GB GDDR5) with 306.97 drivers.  The GPU is detected by EE and is enabled (I set it to 1 CUDA GPU streams), but encoding shows 0 GPU streams encoding.

It leaves a bad taste in my mouth when we the users have to discover these things.  Did no one from Microsoft or Nvidia test this before rolling out the new hardware and drivers?  It seems like a no-brainer to me.  "Hey, there are new cards coming out!  Let's try one and see how it does!"  How did that not happen?

Answer 11

I haven't tried the beta driver myself, but here's what I get with 306.97:

As some other people have said, aside from this issue, this is my favorite encoder that I've tried, so it would be nice if this were fixed…

Answer 12

Here is my query to Nvidia on 12/11/2012

QUESTION SUBMITTED TO NVIDIA 

Why does Microsoft claim that the GTX680 (and all kepler cards) has broken backward compatability with CUDA 1.2 applications, causing many GPU accelerated applications to fail on all Kepler cards. This includes Mainconcept and Microsoft Expression video encoders, Adobe Mercury playback engine, Sony Vegas Applications and Adobe Encore, to name a few.

Microsoft insists that NVIDIA has let down end users and Cuda software authors by breaking CUDA 1.2 application compatability.

Can you please answer if this is true, and if the architecture of the Kepler chips will never be unable to support these existing applications.

Microsoft suggests users ask Nvidia to modify the drivers for the 6xx cards to repair these compatability problems.

I understand that several accelerated functions in Fermi hardware are not supported in Kepler, and I am wondering if this is the root of the problems.

Response from NVIDIA

Hello Roger,

Thank you for contacting NVIDIA Customer Care

I am sorry; but NVIDIA is aware of this issue and our Engineering Team is attempting to verify the issue in-house to determine where the problem is, once reproduced we may provide the fix.

Again, I sincerely apologize for any inconvenience this may cause and please get in touch with us, if you need further assistance and I would be happy to help you.

Best regards,
Farzana,
NVIDIA Customer Care

---

roger berk

Answer 13

Reply from Nvidia as of 17/01/2013: Hello, Thank you for your submission, sorry for the delay in getting back to you. We've received reports of this problem and have investigated the problem. This is a known bug that will need to be fixed in the Microsoft Expression Encoder Pro side. Our investigation revealed that the Microsoft Expression Encoder Pro application has a pre-defined list of supported GPUs, and anything that is not on that list will not be supported. Unfortunately the new GeForce 600 series are not included in the pre-defined list. This can only work if the application is updated to support Kepler based GPUs (600 series). We've reported problem back to Microsoft and they have acknowledged the problem. We were also informed that the Microsoft Expression Encoder is end-of-life and that no further updates will be available. Unfortunately the problem can only be fixed on the application side. You can contact Microsoft and request that they reconsider and release an update to support 600 series GPUs. Let me know if you have any further questions. Best regards, Ray Eric, are you in a position to make some noises internally about at least a hotfix to rectify this issue?

Answer 14

Hi Clive,

This information is not quite accurate.

I can't discuss the details, but in a nutshell, Microsoft was told by Nvidia that the Kepler hardware series was not backwards compatible with the Main Concept H.264 Cuda encoder SDK that EE4 SP2 shipped with in the fall of 2011 and would require Microsoft to re-release their product with a newer version of the Main Concept H.264 Cuda encoder SDK which was released recently. Unfortunately, this would require a multi-week effort for a development/testing perspective to resolve all the issues introduced by taking the new codec drop, as well as a new service and licensing contract with Main Concept and a vehicle to ship the product (marketing, documentation, site hosting, etc). At this point in time, the Expression Encoder product is in Sustained Engineering mode and doesn't have the means to re-release the product that way.

I personally still believe that this issue should have been resolved at the Nvidia driver level to support backward compatibility to all applications that were using the Cuda 1.2 SDK either directly or indirectly. Unfortunately, it doesn't look like Nvidia can, and/or has the will to, resolve this backward compatibility issue.

If Cuda H.264 encoding is really important for your EE usage, I would suggest purchasing a 4XX or 5XX Nvidia hardware which are both supported by EE4 SP2. Alternatively, I would highly recommend looking into Intel SandyBridge or IvyBridge GPU encoding in EE4 SP2 instead of using Cuda.

As a Microsoft employee, I apologize for any inconvenience you may have experienced for the lack of Kepler hardware support in EE4 SP2.

Answer 15

Hi Eric and All,

Thank you for the answer without the obfuscation. It was like a breath of fresh air.

It's a shame that one of the best encoders available ( if not the best) has been relegated to the junk heap. Your SDK is unparalleled for building quick, high quality GUIs for an encoder, and the results are world class. I am very sorry that Microsoft did not seem to understand the Jewel they had and how to market it.

If Microsoft intends to be here well into the future, it should look at why it succeeded in the past and why IBM is still here after 100 years. It is too focused on why Apple and Google found (current and possibly fleeting) success, instead of what people really liked about Microsoft.

Regardless of the politics and BS, I want to applaud you and the team for Expression Encoder. It is truely a masterwork !

PS... Rovi (MainConcept) offers paid upgrades for other software that uses their codecs. For instance they (Rovi) sell upgrades for Adobe Premiere. Is there any chance Microsoft could approach them about offering an upgrade path for the existing codecs used by EE4pro ?  

---

roger berk

Answer 16

This is from Nvidia:

We have investigated this issue and the problem is in the Microsoft Expression Encoder. The encoder doesn't support Kepler based GPUs (GeForce 600 series). Microsoft will have to update the encoder to support GeForce 600 series. We reached out to Microsoft and they acknowledged the issue, but they have no plans to update Expression Encoder unfortunately.

NVIDIA Techical Support

Original link:  https://forums.geforce.com/default/topic/526882/?comment=3728371

Answer 17

Eric,

If there is a list of compatibale hardware can you do me a favor and add the 660 card to it ? (so that we can test it. )
I Don't ask to make a permenant build but just as a test to check if it helps.

Is this possible ?

Thanks

Marcel Balk

Media Hosting
ITDev

Answer 18

EE4 SP2 doesn't have a hardcoded list of hardware. It uses the Main Concept Cuda H.264 Encoder SDK v2.0. That version of the SDK doesn't support any Kepler-grade GPU (6XX GeForce and K-based Quadro and Tesla). Any application based on this version of the MC SDK cannot support Kepler hardware, not just Expression Encoder.

In a nutshell, Expression Encoder would have to re-release with a new version of the MC Cuda Encoder SDK, something that unfortunately cannot be achieved without a full development product team on the Microsoft side.

As I suggested above, I believe you will be better served by either using Intel QSV encoding or an older Nvidia hardware (5XX or older) if you want to use Expression Encoder with GPU encoding. Both of those options still work fine, even on Windows 8. Otherwise, if exercising your Kepler hardware is paramount, I'm sure there are other encoder products available that support it. From what I read on the Kepler technology, it looks like there is a dedicated hardware H.264 encoder on the GPU itself, which might explain why backward compatibility was lost between the SDKs since the technology is so different.