drm/amd/display: Fix the delta clamping for shaper LUT
The shaper LUT requires a 10-bit value of the delta between segments. We were using dc_fixpt_clamp_u0d10() to do that but it doesn't do what we want it to do. It will preserve 10-bit precision after the decimal point, but that's not quite what we want. We want 14-bit precision and discard the 4 most-significant bytes. To do that we'll do dc_fixpt_clamp_u0d14() & 0x3ff instead. Tested-by:Daniel Wheeler <daniel.wheeler@amd.com> Reviewed-by:
Krunoslav Kovac <krunoslav.kovac@amd.com> Acked-by:
Rodrigo Siqueira <rodrigo.siqueira@amd.com> Signed-off-by:
Harry Wentland <harry.wentland@amd.com> Signed-off-by:
Alex Deucher <alexander.deucher@amd.com>
Loading
Please sign in to comment