Stuttering video on playback when using HDR2SDR

[ukmark62]

Trying out the HDR2SDR function. I'm using the latest QSVEnc v5.03 and I have Ice Lake GPU (i7-1065G7). When playing back video, it is stuttering, jittery - the colours look great, not washed out at all. If I remove the "hdr2sdr=reinhard" colorspace option, the video plays back correctly retaining the HDR metadata. I have attached the QSV log below and attach a link to the output sample and a sample of the HDR input file from my MEGA account.

I also get the warning "avqsv: Failed to get header for hardware decoder, switching to software decoder..." - this only happens when the input file is HDR. I had previously opened an issue about this (#54) and I closed it.

Thanks for a great bit of software! The only alternative to this is HDR to SDR with FFMPEG and that is a lot slower.

Sample output: (15.2Mb)
https://mega.nz/file/qgoEzDpL#wgDeu5tpkU7aJyDx_V4fEkUD0ske0ojm9jPyIKcE2ak
Also, a sample of the input HDR file if you want to try it out yourself: (46.5Mb)
https://mega.nz/file/OggUADKZ#6l5VMnDlIJ5X_7GvFrVcF7deyUI6VUKpUIxrJa5czmo
(I had to copy and paste the text of these links to a new browser tab to get there)

I notice that the metadata on the output file still shows the HDR metadata instead of SDR metadata? I'll look into this - first time trying HDR2SDR. (EDIT: I figured that bit out in my next comment below).

--------------------------- Video encoding ---------------------------

QSVEnc 5.03

D:\StaxRip\Apps\Encoders\QSVEnc\QSVEncC64.exe --avhw --codec hevc --quality best --profile main10 --bframes 16 --b-pyramid --async-depth 4 --vpp-colorspace "hdr2sdr=reinhard" --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)" --colormatrix bt2020nc --colorprim bt2020 --transfer smpte2084 --max-cll "2473,596" --chromaloc 2 --aud --cqp 32:32:32 --qp-offset 2:6:8 --trim 0:2488 -i "D:\Downloads (D)\HDR\Army of the Dead (2021) 1080p HEVC AAC.mkv" -o "D:\Downloads (D)\HDR\Army of the Dead (2021) 1080p HEVC AAC_temp\Army of the Dead (2021) 1080p HEVC AAC_out.hevc"

D:\Downloads (D)\HDR\Army of the Dead (2021) 1080p HEVC AAC_temp\Army of the Dead (2021) 1080p HEVC AAC_out.hevc

avqsv: Failed to get header for hardware decoder, switching to software decoder...
cop3.DirectBiasAdjustment value changed off -> auto by driver
cop3.GlobalMotionBiasAdjustment value changed off -> auto by driver
QSVEncC (x64) 5.03 (r2317) by rigaya, May 23 2021 13:54:18 (VC 1928/Win/avx2)
OS             Windows 10 x64 (19042) [UTF-8]
CPU Info       Intel Core i7-1065G7 @ 1.30GHz [TB: 3.48GHz] (4C/8T) <Icelake>
GPU Info       Intel Iris(R) Plus Graphics (64EU) 300-1100MHz [15W] (27.20.100.9466)
Media SDK      QuickSyncVideo (hardware encoder) PG, 1st GPU, API v1.34
Async Depth    4 frames
Buffer Memory  d3d11, 34 work buffer
Input Info     avsw: hevc(yv12(10bit))->p010 [AVX2], 1920x1080, 24/1 fps
VPP            cspconv(p010 -> yv12(16bit))
               colorspace: cspconv(yv12(16bit) -> yuv444(16bit))
                                          matrix:bt2020nc->GBR
                                          transfer:smpte2084->linear
                                          hdr2sdr(reinhard): source_peak=1000.00 ldr_nits=100.00
                                            contrast 0.50, peak 1.00
                                            desat base 0.18, strength 0.75, exp 1.50
                                          prim:bt2020->bt709
                                          transfer:linear->bt709
                                          matrix:GBR->bt709
               cspconv(yuv444(16bit) -> p010)
Trim           0-2488 [offset: 0]
AVSync         cfr
Output         HEVC(yuv420 10bit) main10 @ Level 5 (high tier)
               1920x1080p 1:1 24.000fps (24/1fps)
Target usage   1 - best
Encode Mode    Constant QP (CQP)
CQP Value      I:32  P:32  B:32
QP Limit       min: 1, max: 63
Trellis        Auto
Ref frames     6 frames
Bframes        16 frames, B-pyramid: on
Max GOP Length 240 frames
VUI            matrix:bt2020nc,colorprim:bt2020,transfer:smpte2084,chromaloc:topleft
MasteringDisp  G(0.265000 0.690000) B(0.150000 0.060000) R(0.680000 0.320000)
               WP(0.312700 0.329000) L(1000.000000 0.000100)
MaxCLL/MaxFALL 2473/596
Ext. Features  WeightP WeightB QPOffset tskip ctu:64 sao:all 
encoded 2489 frames, 32.25 fps, 1225.05 kbps, 15.15 MB
encode time 0:01:17, CPU: 19.0, GPU: 96.9, VD: 10.7
frame type IDR   11
frame type I     22,  total size   0.85 MB
frame type P    156,  total size   4.58 MB
frame type B   2322,  total size  10.14 MB

Start:    3:27:16 PM
End:      3:28:35 PM
Duration: 00:01:19

[ukmark62]

I managed to correct the metadata by overwriting these values in the StaxRip gui after I had imported the HDR input file. I had to change the values "colormatrix", "colorprimary" and "transfer" to BT.709 from their imported HDR values and also blank out the "master display", "max cll", "max fall", "chromaloc" and "aud" values from the imported HDR values. Now the output file metadata looks like a standard SDR file when viewed with MediaInfo. However, the playback is still stuttering.
MediaInfo output below (I did not encode the audio, just video):-

General
Unique ID                                : 285770780933396012374891356074203996722 (0xD6FD734EECE398644BEEF78B0AE8E232)
Complete name                            : D:\Video Output\To Do\Army of the Dead (2021) 1080p HEVC AAC.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 15.1 MiB
Duration                                 : 1 min 43 s
Overall bit rate                         : 1 224 kb/s
Encoded date                             : UTC 2021-05-23 15:17:45
Writing application                      : mkvmerge v56.1.0 ('My Friend') 64-bit
Writing library                          : libebml v1.4.2 + libmatroska v1.6.4

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5@High
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 1 min 43 s
Bit rate                                 : 1 222 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 24.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.025
Stream size                              : 15.1 MiB (100%)
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

[parasupaman]

I can also confirm that video playback stutters in version 5.03, when using HDR2SDR and software decoding (--avsw). Hardware decoding (--avhw), does not appear to have stutter issues. However, hardware decoding introduces another issue (#57 (comment))

[rigaya]

Thank you for letting me know the issue.

Video stuttering was caused when using readers except --avhw and with OpenCL based filters (such as HDR2SDR), QSVEnc 5.04 shall fix this problem.

[ukmark62]

Thanks for fixing this so quickly. Tested v5.04 and file now plays back smoothly!

I very rarely use the HDR2SDR function, but do you have any idea why I receive the warning "avqsv: Failed to get header for hardware decoder, switching to software decoder..." ref (#54)? As far as I can tell this only happens when the input file is encoded as HDR. Maybe I need to download/update some drivers on my PC??

I think it only happens on Ice Lake, as the log from #57 does not show this warning when using --avhw decoding with Tiger Lake GPU. Do you have a PC with Ice Lake GPU, as the sample HDR file I made a link to at the top of this thread
(https://mega.nz/file/OggUADKZ#6l5VMnDlIJ5X_7GvFrVcF7deyUI6VUKpUIxrJa5czmo) could be used to try to re-create it?

Thanks again for your efforts - they are very much appreciated!

[ukmark62]

I'm thinking it's a driver issue/bug with Intel. I found some other posts on the web about people using Plex to transcode 10-bit HEVC HDR and saying that hardware decoding is not working, but hardware encoding does work. I'll try to post it on the intel site and see if they know anything about it.

[ukmark62]

Just checked intel media site and these 2 open issues do seem related and suggest it is still work in progress:-
Intel-Media-SDK/MediaSDK#2597 and
Intel-Media-SDK/MediaSDK#1949

I'm not 100% sure it's the same thing, but it does look like there could still be issues with hardware decoding of 10-bit HEVC HDR.

[parasupaman]

Good question. However, I can confirm that I do not see this error on Tiger Lake. So even if it is a driver issue, it does not appear to affect Tiger Lake CPUs.

[ukmark62]

Good question. However, I can confirm that I do not see this error on Tiger Lake. So even if it is a driver issue, it does not appear to affect Tiger Lake CPUs.

I have a ? I compared fixed function CQP encoding quality with hybrid quality at about the same bitrate. Obviously FF encoding is much faster. With Ice Lake, if I compare snapshots of the same frame with FF and hybrid encodes, the only (small) differences I see in quality is with facial detail - skin texture, wrinkles etc. The FF encodes tend to lose some of those facial textures, whereas hybrid encodes are very good with minimal loss when compared with the source. I probably would not notice a difference during playback.

Is that the same with TGL, that FF encode compared to hybrid can fall a little short when retaining facial textures? I must admit, that everywhere else (apart from the fine facial details), the quality between the 2 is very close. I was wondering if this had been improved with TGL FF encoding. Thx.

If it was very close, then I'd be tempted to update to TGL and just use FF (esp as you can use bframes and bpyramid with TGL FF). If possible I would love to see a single frame comparison showing a facial closeup.

[parasupaman]

I have a ? I compared fixed function CQP encoding quality with hybrid quality at about the same bitrate. Obviously FF encoding is much faster. With Ice Lake, if I compare snapshots of the same frame with FF and hybrid encodes, the only (small) differences I see in quality is with facial detail - skin texture, wrinkles etc. The FF encodes tend to lose some of those facial textures, whereas hybrid encodes are very good with minimal loss when compared with the source. I probably would not notice a difference during playback.

Is that the same with TGL, that FF encode compared to hybrid can fall a little short when retaining facial textures? I must admit, that everywhere else (apart from the fine facial details), the quality between the 2 is very close. I was wondering if this had been improved with TGL FF encoding. Thx.

If it was very close, then I'd be tempted to update to TGL and just use FF (esp as you can use bframes and bpyramid with TGL FF). If possible I would love to see a single frame comparison showing a facial closeup.

I have never used Fixed Function, so I just did a quick test...

Using the same CQP value (all other encoding options the same), Fixed Function results in a very slight increase in bitrate compared to Hybrid (11.5 Mbit FF vs 11.3 MBit Hybrid). Yet Fixed Function has a very slight softness to the image (probably less detail) compared to Hybrid. Probably similar to what you are experiencing with Ice Lake. I'm guessing that the motion and image analysis is better in hybrid. And yes, I noticed that B Frames and B Pyramid seemed to be working with Fixed Function.

I'm very happy with the speed of hybrid though. The faster GPU of the i7-1165G7 Tiger Lake (96 EUs) approaches almost a 50% encoding speed increase at times, compared to my old i5 Kaby Lake. Not sure how the speed compares to Ice Lake though.

[ukmark62]

Thanks for that. I suppose if fixed function were as good as hybrid, then what is the point of hybrid. I'll stick with ICL for now. To get the same (or similar) bitrate with FF CQP on ICL I have to increase the CQP settings by 2 (because of no bframe or bpyramid support on FF with ICL). So, if CQP hybrid is 32 with 16 bframes etc, I will set FF CQP to 34.
As you say, hybrid CQP is extremely good quality and still a good encoding speed.
What I also find on ICL is that with async-depth set to 4, my CPU only runs at 7% when encoding in hybrid mode.