[PATCH v1 0/4] media: atomisp: prevent integer overflow in

[Siho Lee]

CVE-2022-50399 (commit 51b8dc5163d2) fixed an integer overflow in
sh_css_set_black_frame() by using array3_size(). However, the same
overflow pattern exists in 9 other locations that were not updated.

The most critical is in alloc_dvs_6axis_table() (sh_css_param_dvs.c)
where width_y * height_y multiplication overflows on 32-bit arithmetic
when the user sets a large resolution via VIDIOC_S_FMT. ATOM_ISP_MAX_WIDTH
is set to UINT_MAX which imposes no limit.

For example, with frame_res width=16777216, width_y becomes 262145:
    262145 * 262145 = 524289 (32-bit, overflowed)
    262145 * 262145 = 68720001025 (64-bit, actual)

This causes kvmalloc() to allocate only ~2 MB instead of the required
~64 GB, leading to an out-of-bounds write in
init_dvs_6axis_table_from_default() that triggers a kernel panic.

Patches 1-3 use array3_size() to prevent overflow at the remaining
locations. Patch 4 reduces ATOM_ISP_MAX_WIDTH/HEIGHT from UINT_MAX
to 8192 as a hard limit (mathematically: DVS_BLOCKDIM=64, max blocks
129*129, 129*129*sizeof(u32)=66564 bytes which cannot overflow u32).

Tested on: 5.15.0 (hardware), 6.8.0 (hardware), v7.0.10 stable (QEMU),
v7.1-rc5 mainline (static analysis). No Intel Atom ISP hardware
available for V4L2 ioctl path verification.

Siho Lee (4):
  staging: atomisp: prevent integer overflow in DVS 6-axis allocation
  staging: atomisp: prevent integer overflow in sh_css_params DVS
    allocation
  staging: atomisp: prevent integer overflow in shading table allocation
  staging: atomisp: add resolution limits to prevent DVS overflow

 drivers/staging/media/atomisp/pci/atomisp_internal.h     | 4 ++--
 drivers/staging/media/atomisp/pci/sh_css_param_dvs.c     | 8 ++++----
 drivers/staging/media/atomisp/pci/sh_css_param_shading.c | 2 +-
 drivers/staging/media/atomisp/pci/sh_css_params.c        | 8 ++++----
 4 files changed, 11 insertions(+), 11 deletions(-)

-- 
2.43.0