soundwire: cadence: use directly bus sdw_defer structure

Copying the bus sdw_defer structure into the Cadence internals leads
to using stale pointers and kernel oopses on errors. It's just simpler
and safer to use the bus sdw_defer structure directly.

Link: https://github.com/thesofproject/linux/issues/4056


Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Reviewed-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
Signed-off-by: Bard Liao <yung-chuan.liao@linux.intel.com>
Link: https://lore.kernel.org/r/20230119073211.85979-4-yung-chuan.liao@linux.intel.com


Signed-off-by: Vinod Koul <vkoul@kernel.org>