Skip to content

Commit

Permalink
iio: afe: rescale: fix accuracy for small fractional scales
Browse files Browse the repository at this point in the history
The approximation caused by integer divisions can be costly on smaller
scale values since the decimal part is significant compared to the
integer part. Switch to an IIO_VAL_INT_PLUS_NANO scale type in such
cases to maintain accuracy.

Signed-off-by: Liam Beguin <liambeguin@gmail.com>
Reviewed-by: Peter Rosin <peda@axentia.se>
Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>
Link: https://lore.kernel.org/r/20220213025739.2561834-5-liambeguin@gmail.com
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
  • Loading branch information
Liam Beguin authored and Jonathan Cameron committed Feb 27, 2022
1 parent a29c328 commit f5fc003
Showing 1 changed file with 17 additions and 3 deletions.
20 changes: 17 additions & 3 deletions drivers/iio/afe/iio-rescale.c
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ int rescale_process_scale(struct rescale *rescale, int scale_type,
int *val, int *val2)
{
s64 tmp;
s32 rem;
s32 rem, rem2;
u32 mult;
u32 neg;

Expand All @@ -43,9 +43,23 @@ int rescale_process_scale(struct rescale *rescale, int scale_type,
tmp = (s64)*val * 1000000000LL;
tmp = div_s64(tmp, rescale->denominator);
tmp *= rescale->numerator;
tmp = div_s64(tmp, 1000000000LL);

tmp = div_s64_rem(tmp, 1000000000LL, &rem);
*val = tmp;
return scale_type;

if (!rem)
return scale_type;

tmp = 1 << *val2;

rem2 = *val % (int)tmp;
*val = *val / (int)tmp;

*val2 = rem / (int)tmp;
if (rem2)
*val2 += div_s64((s64)rem2 * 1000000000LL, tmp);

return IIO_VAL_INT_PLUS_NANO;
case IIO_VAL_INT_PLUS_NANO:
case IIO_VAL_INT_PLUS_MICRO:
mult = scale_type == IIO_VAL_INT_PLUS_NANO ? 1000000000L : 1000000L;
Expand Down

0 comments on commit f5fc003

Please sign in to comment.