Re: [PATCH 4/4] iio: light: tsl2591: improve lux calculation precision

[Andy Shevchenko]

On Tue, Oct 13, 2026 at 04:39:43PM +0200, Jan Remes wrote:

> The lux calculation computes the ratio CH1 / CH0 as a truncated integer
> (CH1 * 1000 / CH0) before using it. When CH1 is close to CH0, the
> truncation makes the result off by up to ~35%, e.g. CH0 = 56097,
> CH1 = 55973 at LOW gain and 100ms gives 1.517 lux instead of 1.118 lux.
> 
> The formula (CH0 - CH1) * (1 - CH1 / CH0) / CPL is equal to
> 
>    (CH0 - CH1)^2 * DF / (CH0 * ATIME_ms * AGAIN)
> 
> which needs no intermediate ratio. Compute it like that, with a single
> division at the end.
> 
> Also report the result in micro-lux instead of milli-lux. At MAX gain
> and 600ms one count is well below 1 mlx, so the milli-lux resolution
> throws away information. The numerator still fits into u64.

...

> static int tsl2591_read_channel_data(struct iio_dev *indio_dev,

>  	int int_time_fval, gain_multi;
>  	u16 als_ch0, als_ch1, als_max;
>  	u64 lux;
> -	u32 rem;
> +	u32 diff, rem;

Keep that to be in reversed xmas tree order.

>  	int ret;

-- 
With Best Regards,
Andy Shevchenko