drm/amd/display: Make assert in DCE's program_bit_depth_reduction more lenient.

This is needed to avoid warnings with linebuffer depth 36 bpp.
Testing on a Polaris11, DCE-11.2 on a 10 bit HDR-10 monitor
showed no obvious problems, and this 12 bpc limit is consistent
with what other function in the DCE bit depth reduction path use.

Signed-off-by: Mario Kleiner <mario.kleiner.de@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>