serial: imx: Drop a few unneeded casts

There is no point in casting a struct uart_port to a struct imx_port
just to access the first member of the latter (a struct uart_port).

This introduces no code changes.

Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Link: https://lore.kernel.org/r/20230302115417.1860210-1-u.kleine-koenig@pengutronix.de


Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>