video: smscufx: don't restore fb_mmap after deferred IO cleanup

Deferred IO now preserves the fb_ops.

Cc: Steve Glendinning <steve.glendinning@shawell.net>
Cc: linux-fbdev@vger.kernel.org
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/480dcc682481e6972e5648181d7e92120929ec6b.1575022735.git.jani.nikula@intel.com