media: mediatek: vcodec: Coverity issues in encoder driver

CERT-C Characters and Strings:
check core id is in valid range:
dev->reg_base[dev->venc_pdata->core_id] evaluates to an address
that could be at negative offset of an array.

CERT-C Expression:
check buf is not NULL before used:
Dereferencing buf, which is known to be NULL.

Signed-off-by: Irui Wang <irui.wang@mediatek.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>