Re: [PATCH RFC 0/7] media: qcom: iris: add support for decoding 10bit formats

From: Nicolas Dufresne

Date: Thu Apr 09 2026 - 09:30:45 EST

Le jeudi 09 avril 2026 à 09:36 +0200, Neil Armstrong a écrit :
> Hi,
>
> On 4/9/26 03:04, Nicolas Dufresne wrote:
> > Hi,
> >
> > Le jeudi 09 avril 2026 à 03:02 +0300, Dmitry Baryshkov a écrit :
> > > On Wed, Apr 08, 2026 at 06:43:53PM +0200, Neil Armstrong wrote:
> > > > This adds the plumbing to support decoding HEVC and AV1
> > > > streams into 10bit pixel formats, linear and compressed.
> > > >
> > > > This has only been tested on SM8650 with HEVC, and was inspired by
> > > > Venus and the downstream vidc driver for the buffer
> > > > calculations and HFI messages.
> > > >
> > > > I was unable to get 10bit decoding working with Gstreamer
> > > > and ffmpeg, but v4l2-ctl works with:
> > >
> > > Any particular errors? I assume Gstreamer needs to be taught about
> > > Q10C. But P010 should (hopefully) work.
> >
> > P010 should work for both Gst and FFMPEG, its probably a user error, or there is
> > a hidden bug in the driver that make it fail, v4l2-ctl is very permissive as it
> > simply dump to disk. You should provide an updated fluster score, so you have to
> > use one of these.
>
> I did run fluster and all main10 fails with Gstreamer and FFmpeg, I tried to manually
> run the gst and ffmpeg commands with v4l2-tracer and logs but I can't explain the reason,
> all returns from the driver seems valid but somehow they just error out with:
>
> FFmpeg:
> $ ffmpeg -c:v hevc_v4l2m2m -i Big_Buck_Bunny_1080_10s_30MB_main10.h265 -y -f null -
> ...
> [hevc_v4l2m2m @ 0x55c0328aa0] Using device /dev/video-dec0
> [hevc_v4l2m2m @ 0x55c0328aa0] driver 'iris_driver' on card 'iris_decoder' in mplane mode
> [hevc_v4l2m2m @ 0x55c0328aa0] requesting formats: output=HEVC/none capture=NV12/yuv420p10le
> ...
> [hevc_v4l2m2m @ 0x55c0328aa0] An invalid frame was output by a decoder. This is a bug, please report it.
> [vist#0:0/hevc @ 0x55c02dc9b0] [dec:hevc_v4l2m2m @ 0x55c029d510] Decoding error: Internal bug, should not have happened
>

This one needs further investigation for sure. This error can be various things,
and it requires going up to the v4l2 code to figure-out:

Case 1:
if (!frame->buf[0] || frame->format < 0)
goto fail;

Case 2
if (frame->width <= 0 || frame->height <= 0)
goto fail;

But a quick look lead me to think it case 1 (frame->format < 0) since I don't
see P010 in the format map in ./libavcodec/v4l2_fmt.c (at least in mainline).
Its also missing support for any opaque format, in fact I believe the DMABuf/DRM
context is only in LibreELEC fork. But overall, it points toward ffmpeg for this
error so far.

> The v4l2 trace shows a normal sequence with the driver returning P010 as G_FMT after the source change event,
> and the capture planes dequeued but for an unknown reason the buffer is rejected by ffmpeg.
>
> Gst:
> $ gst-launch-1.0 -v -m filesrc location=Big_Buck_Bunny_1080_10s_30MB_main10.h265 ! h265parse ! v4l2h265dec ! tee ! fakevideosink
> Setting pipeline to PAUSED ...
> Pipeline is PREROLLING ...
> ...
> Got message #37 from element "h265parse0" (latency): no message details
> ERROR: from element /GstPipeline:pipeline0/GstH265Parse:h265parse0: Internal data stream error.
> Redistribute latency...
> Additional debug info:
> ../gstreamer/subprojects/gstreamer/libs/gst/base/gstbaseparse.c(3702): gst_base_parse_loop (): /GstPipeline:pipeline0/GstH265Parse:h265parse0:
> streaming stopped, reason not-negotiated (-4)
> Got message #39 from pad "h265parse0:src" (property-notify): ERROR: pipeline doesn't want to preroll.
> GstMessagePropertyNotify, property-name=(string)caps, property-value=(GstCaps)"video/x-h265\,\ width\=$int$1920\,\ height\=$int$1080\,\ framerate\=$fraction$30/1\,\ chroma-format\=$string$4:2:0\,\ bit-depth-luma\=$uint$10\,\ bit-depth-chroma\=$uint$10\,\ parsed\=$boolean$true\,\ stream-format\=$string$byte-stream\,\ alignment\=$string$au\,\ pixel-aspect-ratio\=$fraction$1/1\,\ profile\=$string$main-10\,\ tier\=$string$main\,\ level\=$string$4";
> /GstPipeline:pipeline0/GstH265Parse:h265parse0.GstPad:src: caps = video/x-h265, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, chroma-format=(string)4:2:0, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, parsed=(boolean)true, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)1/1, profile=(string)main-10, tier=(string)main, level=(string)4
> ...

There is not a lot of details here, but I would start looking into
V4L2_CID_MPEG_VIDEO_HEVC_PROFILE and V4L2_CID_MPEG_VIDEO_HEVC_LEVEL
implementation. GStreamer (and Chromium too) will refuse to use a decoder that
does not advertise the supported profile (though I see there is code for that,
maybe its just some bug).

>
> In this case OUTPUT is not STREAMON and no OUTPUT buffers are queued, so I wonder why this one fails....
>
> My gstreamer and ffmpeg foo is bad and I probably missed something obvious...
>
>
You may get a different hint with more traces, just enabling general warnings:

export GST_DEBUG=2

Or the full V4L2 traces too:

export GST_DEBUG="v4l2*:7,2"

I'm sure its just a bug (or two). Happy to help to find it.

>
> >
> > For Q10C on GStreamer, it needs mapping [0] and you need some bugfix [1] and
> > another that I will be sending tomorrow. We had never tested video compression
> > with this module before.
> >
> > [0] https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8195
> > [1] https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/11222
> >
> > The last issue has to do with:
> > https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/38965e2d9c1119674a65dc437ee7e8ec95339f31/subprojects/gst-plugins-good/sys/v4l2/gstv4l2object.c#L4378
> >
> > V4L2 format gives us the number of allocation, but not really the number of
> > planes, and we forgot to initialize that number for the "opaque" format case. A
> > tempory fix might be to add this after S_FMT:
> >
> > if (GST_VIDEO_INFO_FORMAT (&info.vinfo) == GST_VIDEO_FORMAT_DMA_DRM)
> >    n_v4l_planes = format.fmt.pix_mp.num_planes;
> >
> > Works for AFBC and QC at leat, since both are unambiguously single plane, and so
> > cannot have mplane variants.
>
> I'll definitely try that ! thanks for the pointers !
>
> >
> > Let me know how far you get! Please be aware that Robert and I are making these
> > patches based on feedback, we don't have access to any boards capable of
> > decoding to QC compressed formats.
> >
> > >
> > > > v4l2-ctl --verbose --set-fmt-video-out=pixelformat=HEVC --set-fmt-video=pixelformat=P010 --stream-mmap --stream-out-mmap --stream-from-hdr Big_Buck_Bunny_1080_10s_30MB_main10.h265.hdr --stream-to out.P010
> > > > v4l2-ctl --verbose --set-fmt-video-out=pixelformat=HEVC --set-fmt-video=pixelformat=Q10C --stream-mmap --stream-out-mmap --stream-from-hdr Big_Buck_Bunny_1080_10s_30MB_main10.h265.hdr --stream-to out.QC10
> > > >
> > > > The non-10bit decoding still works as before.
> > > >
> > > > With Big_Buck_Bunny_1080_10s_30MB reencoded in 10-bit profile
> > > > and tranformed in v4l2 header format with [1]:
> > > > ffmpeg -i Big_Buck_Bunny_1080_10s_30MB.h264 -pix_fmt yuv420p10le -c:v libx265 -crf 28 -x265-params profile=main10 Big_Buck_Bunny_1080_10s_30MB_main10.h265
> > > > /path/to/mkhdr.sh Big_Buck_Bunny_1080_10s_30MB_main10.h265 raw Big_Buck_Bunny_1080_10s_30MB_main10.h265.hdr
> > > >
> > > > The frames correctness has been verified buy displaying them
> > > > via Vulkan DMA_BUF import, including QC10C and QC08C.
> >
> > In GStreamer, once the video4linux plugin issues are fixed, you should be able
> > to display the frames using glimagesink. GL only allow for RGB render, which
> > damages the data, so its not good enough for conformance testing with
> > compression enabled, but usually just doing visual inspection is acceptable.
> >
> > > >
> > > > The support is probably incomplete for other platforms and
> > > > I'm unsure what's required to conform to the V4L2 M2M stateless
> > >
> > > stateful
> > >
> > > > spec, especially since AFAIK the decoder doesn't support
> > > > decoding 10bit streams in 8bit pixel format, thus the RFC state.
> > > > Review is welcome !
> >
> > Why would your decoder need to support decoding 10bit into 8bit ? This is quite
> > rare and its only possible with post-processed capture buffer.
>
> Yeah it's just a note, the Amlogic one supported outputing main10 in NV12 but
> only supported 10bit in their compressed format.
>

Ack. There is a lot of variation with various hardware for sure. The Allwinner
stateless one swallow the extra 2bit plane (since they do some funky NV12 + two
more planes for the missing 2bits, and that's complicate as a format hehe).

cheers,
Nicolas

>
> >
> > > >
> > > > [1] https://github.com/superna9999/pyv4l2compliance
> > > >
> > > > Signed-off-by: Neil Armstrong <neil.armstrong@xxxxxxxxxx>
> > > > ---
> > > > Neil Armstrong (7):
> > > >        media: qcom: iris: add QC10C & P010 buffer size calculations
> > > >        media: qcom: iris: gen2: add support for 10bit decoding
> > > >        media: qcom: iris: add helpers for 8bit and 10bit formats
> > > >        media: qcom: iris: vdec: update size and stride calculations for 10bit formats
> > > >        media: qcom: iris: vdec: forbid g_fmt while waiting for first source change
> >
> > The is suspicious, it should just send the previous state until you get the
> > event. Typical use case is for application to set a format based on bitstream
> > parsing, and use that as a guess to try and pre-allocate the buffers. I don't
> > usually review iris code, but I'll give that series some more eyes in the coming
> > days.
>
> Yeah I wasn't sure, venus does that, same as the vendor driver so I thought it would
> be good but I'm probably mistaken.
>
> >
> > cheers,
> > Nicolas
>
> Thanks!
> Neil
>
> >
> > > >        media: qcom: iris: vdec: update find_format to handle 8bit and 10bit formats
> > > >        media: qcom: iris: vdec: allow decoding into 10bit format
> > > >
> > > >   drivers/media/platform/qcom/iris/iris_buffer.c     | 81 +++++++++++++++++++++-
> > > >   .../platform/qcom/iris/iris_hfi_gen2_command.c     | 71 ++++++++++++++++++-
> > > >   .../platform/qcom/iris/iris_hfi_gen2_defines.h     | 1 +
> > > >   .../platform/qcom/iris/iris_hfi_gen2_response.c    | 35 ++++++++--
> > > >   drivers/media/platform/qcom/iris/iris_instance.h   | 2 +
> > > >   .../platform/qcom/iris/iris_platform_common.h      | 1 +
> > > >   .../media/platform/qcom/iris/iris_platform_gen2.c | 3 +-
> > > >   drivers/media/platform/qcom/iris/iris_state.c      | 6 ++
> > > >   drivers/media/platform/qcom/iris/iris_state.h      | 1 +
> > > >   drivers/media/platform/qcom/iris/iris_utils.c      | 16 ++++-
> > > >   drivers/media/platform/qcom/iris/iris_utils.h      | 2 +
> > > >   drivers/media/platform/qcom/iris/iris_vdec.c       | 70 +++++++++++++++++--
> > > >   drivers/media/platform/qcom/iris/iris_vidc.c       | 14 +++-
> > > >   13 files changed, 285 insertions(+), 18 deletions(-)
> > > > ---
> > > > base-commit: f3e6330d7fe42b204af05a2dbc68b379e0ad179e
> > > > change-id: 20260408-topic-sm8x50-iris-10bit-decoding-074c3ac7975c
> > > >
> > > > Best regards,
> > > > --
> > > > Neil Armstrong <neil.armstrong@xxxxxxxxxx>
> > > >

Attachment: signature.asc
Description: This is a digitally signed message part