greybus: gb-audio: fix build breakage on 4.1-rc1

In commit 1efb53a2 ("ASoC: simple-card: Remove support for setting
differing DAI formats"), the .fmt field was removed from struct
asoc_simple_dai.  Fix this build breakage by not trying to set it.

Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Acked-by: John Stultz <john.stultz@linaro.org>