RE: [PATCH] drm/amd/display: Fix swap in median3() for DSC rate control

[Li, Roman]

Public

Reviewed-by: Roman Li <roman.li@xxxxxxx>

> -----Original Message-----
> From: amd-gfx <amd-gfx-bounces@xxxxxxxxxxxxxxxxxxxxx> On Behalf Of Keita
> Morisaki
> Sent: Tuesday, September 22, 2026 10:25 PM
> To: Zheng, Austin <Austin.Zheng@xxxxxxx>; Lei, Jun <Jun.Lei@xxxxxxx>;
> Wentland, Harry <Harry.Wentland@xxxxxxx>; Li, Sun peng (Leo)
> <Sunpeng.Li@xxxxxxx>; Deucher, Alexander <Alexander.Deucher@xxxxxxx>
> Cc: Rodrigo Siqueira <siqueira@xxxxxxxxxx>; Koenig, Christian
> <Christian.Koenig@xxxxxxx>; David Airlie <airlied@xxxxxxxxx>; Simona Vetter
> <simona@xxxxxxxx>; amd-gfx@xxxxxxxxxxxxxxxxxxxxx; dri-
> devel@xxxxxxxxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; Keita Morisaki
> <kmta1236@xxxxxxxxx>
> Subject: [PATCH] drm/amd/display: Fix swap in median3() for DSC rate control
>
> [Some people who received this message don't often get email from
> kmta1236@xxxxxxxxx. Learn why this is important at
> https://aka.ms/LearnAboutSenderIdentification ]
>
> median3() is meant to return the median of three values, but its last compare-and-
> swap step swaps b and c instead of a and b. When c is smaller than both a and b, it
> returns max(a, b) instead of min(a, b).
> For example, median3(2, 3, 1) returns 3 instead of 2.
>
> _do_calc_rc_params() always passes 0 as the first argument, and the second
> argument is never below 12. So whenever the third argument is negative,
> first_line_bpg_offset (and second_line_bpg_offset for 4:2:0) is set to the second
> argument instead of 0.
>
> Fix this by swapping a and b in the last step, as intended.
>
> Note that this change does not affect valid DSC configurations. The third argument
> is negative only when bits per pixel exceeds the uncompressed rate of the colour
> format:
>
>  - 3 * bpc for 4:4:4, 3 * bpc + 2 for RGB
>
>  - 2 * bpc for 4:2:2
>
>  - 1.5 * bpc for 4:2:0
>
> Fixes: 97bda0322b8a ("drm/amd/display: Add DSC support for Navi (v2)")
> Signed-off-by: Keita Morisaki <kmta1236@xxxxxxxxx>
> ---
>  drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c
> b/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c
> index b8fc27a5361e..657173d8081c 100644
> --- a/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c
> +++ b/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c
> @@ -46,7 +46,7 @@ static int median3(int a, int b, int c)
>         if (b > c)
>                 swap(b, c);
>         if (a > b)
> -               swap(b, c);
> +               swap(a, b);
>
>         return b;
>  }
>
> base-commit: 93f51579e7df248780214094418f205253383cc5
> --
> 2.43.0