drm/amd/amdgpu: Bail out of BO node creation if not enough VRAM (v3)

(v2): Return 0 and set mem->mm_node to NULL.
(v3): Use atomic64_add_return instead.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>