Cannot run rendering_example.py

[qaz8788817]

Hi, this is such a great project.
I've met some problem after I set up my cuda and tensorflow.
I followed the steps of "Compile the rendering layer - CUDA code" in README.md, and I entered the command:
python rendering_example.py
the window showed like this:

`/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:524: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:532: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
2024-01-11 18:19:28.412525: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA
2024-01-11 18:19:28.475230: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:897] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-01-11 18:19:28.475339: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1392] Found device 0 with properties:
name: NVIDIA RTX A4000 major: 8 minor: 6 memoryClockRate(GHz): 1.56
pciBusID: 0000:0b:00.0
totalMemory: 15.73GiB freeMemory: 15.56GiB
2024-01-11 18:19:28.475354: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1471] Adding visible gpu devices: 0
2024-01-11 18:19:28.944101: I tensorflow/core/common_runtime/gpu/gpu_device.cc:952] Device interconnect StreamExecutor with strength 1 edge matrix:
2024-01-11 18:19:28.944139: I tensorflow/core/common_runtime/gpu/gpu_device.cc:958] 0
2024-01-11 18:19:28.944145: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0: N
2024-01-11 18:19:28.944234: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1084] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 15071 MB memory) -> physical GPU (device: 0, name: NVIDIA RTX A4000, pci bus id: 0000:0b:00.0, compute capability: 8.6)
Loading 3DMM tri ...
DONE
Loading 3DMM vertex tri ...
DONE
WARNING:tensorflow:From /home/csie/NonLinear_env/Nonlinear_Face_3DMM/rendering_ops.py:245: calling l2_normalize (from tensorflow.python.ops.nn_impl) with dim is deprecated and will be removed in a future version.
Instructions for updating:
dim is deprecated, use axis instead
2024-01-11 18:19:35.677227: E tensorflow/stream_executor/cuda/cuda_blas.cc:647] failed to run cuBLAS routine cublasSgemmBatched: CUBLAS_STATUS_EXECUTION_FAILED
2024-01-11 18:19:35.677264: E tensorflow/stream_executor/cuda/cuda_blas.cc:2505] Internal: failed BLAS call, see log for details
Traceback (most recent call last):
File "/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1322, in _do_call
return fn(*args)
File "/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1307, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1409, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InternalError: Blas xGEMMBatched launch failed : a.shape=[16,4,3], b.shape=[16,53215,4], m=3, n=53215, k=4, batch_size=16
[[Node: MatMul = BatchMatMul[T=DT_FLOAT, adj_x=true, adj_y=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](concat, concat_1)]]
[[Node: Greater_2/_29 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_360_Greater_2", tensor_type=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "rendering_example.py", line 39, in
tf.app.run()
File "/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "rendering_example.py", line 30, in main
s_img = sess.run( images, feed_dict={ texture_ph: data['sample_texture'], shape_ph:data['sample_shape'], m_ph:data['sample_m']})
File "/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 900, in run
run_metadata_ptr)
File "/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1135, in _run
feed_dict_tensor, options, run_metadata)
File "/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1316, in _do_run
run_metadata)
File "/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1335, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InternalError: Blas xGEMMBatched launch failed : a.shape=[16,4,3], b.shape=[16,53215,4], m=3, n=53215, k=4, batch_size=16
[[Node: MatMul = BatchMatMul[T=DT_FLOAT, adj_x=true, adj_y=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](concat, concat_1)]]
[[Node: Greater_2/_29 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_360_Greater_2", tensor_type=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

Caused by op 'MatMul', defined at:
File "rendering_example.py", line 39, in
tf.app.run()
File "/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "rendering_example.py", line 28, in main
images, foreground_mask = warp_texture(texture_ph, m_ph, shape_ph, output_size = output_size)
File "/home/csie/NonLinear_env/Nonlinear_Face_3DMM/rendering_ops.py", line 74, in warp_texture
vertex2d = tf.matmul(m, vertex4d, True, True) # batch_size x 3 x vertex_num
File "/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/ops/math_ops.py", line 1976, in matmul
a, b, adj_x=adjoint_a, adj_y=adjoint_b, name=name)
File "/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/ops/gen_math_ops.py", line 1236, in batch_mat_mul
"BatchMatMul", x=x, y=y, adj_x=adj_x, adj_y=adj_y, name=name)
File "/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 3414, in create_op
op_def=op_def)
File "/home/csie/NonLinear_env/NonEnv/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1740, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InternalError (see above for traceback): Blas xGEMMBatched launch failed : a.shape=[16,4,3], b.shape=[16,53215,4], m=3, n=53215, k=4, batch_size=16
[[Node: MatMul = BatchMatMul[T=DT_FLOAT, adj_x=true, adj_y=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](concat, concat_1)]]
[[Node: Greater_2/_29 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_360_Greater_2", tensor_type=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]`

And here is the status of my gpu:

I've search Internet to find solution, all of them said that need to set
allow_growth = True
And I checked the rendering_example.py, it had been set.

And my environment is:
Ubuntu 20.04 Tensorflow-gpu 1.8.0 CUDA 9.0
I also check I've installed all CUDA9.0 patches.
Can someone help me? Thank you.