greybus: audio: fix a bunch of endian issues

sparse is rightfully complaining about a lack of converting when
accessing or assigning to little endian fields.  Fix them all up to work
properly.

Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Reviewed-by: Alex Elder <elder@linaro.org>