soundwire: bus: clock_stop: don't deal with UNATTACHED Slave devices

We don't need to do anything for the slave if it is unattached during
clock stop prepare and exit sequences.

Signed-off-by: Bard Liao <yung-chuan.liao@linux.intel.com>
Link: https://lore.kernel.org/r/20200531151806.25951-1-yung-chuan.liao@linux.intel.com


Signed-off-by: Vinod Koul <vkoul@kernel.org>