USB: serial: f81232: drop time-based drain delay

The f81232 driver now waits for the transmit FIFO to drain during close
so there is no need to keep the time-based drain delay, which would add
up to two seconds on every close for low line speeds.

Fixes: 98405f81 ("USB: serial: f81232: add tx_empty function")
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Johan Hovold <johan@kernel.org>