Some problem of 0.3.16 in Ubuntu

143 · 2019 年12 月 30 日 21:36

I just updated taichi to 0.3.16 on my Ubuntu. I used the CUDA 10-1 version. My torch is 1.3.1. I get the errors:

python(_PyFunction_FastCallDict+0x1d5) [0x5608c4e565d5]
python(_PyObject_Call_Prepend+0x63) [0x5608c4e6dc43]
python(PyObject_Call+0x6e) [0x5608c4e6295e]
python(_PyEval_EvalFrameDefault+0x1e20) [0x5608c4f11250]
python(_PyEval_EvalCodeWithName+0x2f9) [0x5608c4e554f9]
python(_PyFunction_FastCallDict+0x1d5) [0x5608c4e565d5]
python(_PyObject_Call_Prepend+0x63) [0x5608c4e6dc43]
python(+0x17116a) [0x5608c4eb016a]
python(_PyObject_FastCallKeywords+0x49b) [0x5608c4eb8d2b]
python(_PyEval_EvalFrameDefault+0x537e) [0x5608c4f147ae]
python(_PyFunction_FastCallKeywords+0xfb) [0x5608c4eb779b]
python(_PyEval_EvalFrameDefault+0x6a0) [0x5608c4f0fad0]
python(_PyEval_EvalCodeWithName+0x2f9) [0x5608c4e554f9]
python(PyEval_EvalCodeEx+0x44) [0x5608c4e563c4]
python(PyEval_EvalCode+0x1c) [0x5608c4e563ec]
python(+0x22f874) [0x5608c4f6e874]
python(PyRun_FileExFlags+0xa1) [0x5608c4f78b81]
python(PyRun_SimpleFileExFlags+0x1c3) [0x5608c4f78d73]
python(+0x23ae5f) [0x5608c4f79e5f]
python(_Py_UnixMain+0x3c) [0x5608c4f79f7c]
/lib/x86_64-linux-gnu/libc.so.6: __libc_start_main
python(+0x1e0122) [0x5608c4f1f122]

yuanming · 2019 年12 月 30 日 23:08

Could you share with me the code to reproduce this?

143 · 2019 年12 月 31 日 00:13

Well, I will try to smaller the code and send it to you. I just got the full error list here:

[E 12/30/19 16:11:40.611] [llvm_jit_ptx.cpp:compile@180] Cuda Error CUDA_ERROR_ILLEGAL_ADDRESS: an illegal memory access was encountered
[E 12/30/19 16:11:40.611] Received signal 6 (Aborted)
***********************************
* Taichi Compiler Stack Traceback *
***********************************
/home/user/anaconda3/lib/python3.7/site-packages/taichi/core/../lib/taichi_core.so: taichi::signal_handler(int)
/lib/x86_64-linux-gnu/libc.so.6(+0x3ef20) [0x7f7de2b7ff20]
/lib/x86_64-linux-gnu/libpthread.so.0: raise
/home/user/anaconda3/lib/python3.7/site-packages/taichi/core/../lib/taichi_core.so: taichi::Tlang::CUDAContext::compile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
/home/user/anaconda3/lib/python3.7/site-packages/taichi/core/../lib/taichi_core.so: taichi::Tlang::CodeGenLLVMGPU::compile_module_to_executable()
/home/user/anaconda3/lib/python3.7/site-packages/taichi/core/../lib/taichi_core.so: taichi::Tlang::GPUCodeGen::codegen_llvm()
/home/user/anaconda3/lib/python3.7/site-packages/taichi/core/../lib/taichi_core.so: taichi::Tlang::KernelCodeGen::compile(taichi::Tlang::Program&, taichi::Tlang::Kernel&)
/home/user/anaconda3/lib/python3.7/site-packages/taichi/core/../lib/taichi_core.so: taichi::Tlang::Program::compile(taichi::Tlang::Kernel&)
/home/user/anaconda3/lib/python3.7/site-packages/taichi/core/../lib/taichi_core.so: taichi::Tlang::Kernel::compile()
/home/user/anaconda3/lib/python3.7/site-packages/taichi/core/../lib/taichi_core.so: taichi::Tlang::Kernel::operator()()
/home/user/anaconda3/lib/python3.7/site-packages/taichi/core/../lib/taichi_core.so(+0xdd03a6) [0x7f7d7c4853a6]
/home/user/anaconda3/lib/python3.7/site-packages/taichi/core/../lib/taichi_core.so(+0xb34054) [0x7f7d7c1e9054]
python(_PyMethodDef_RawFastCallDict+0x267) [0x55587e671987]
python(_PyCFunction_FastCallDict+0x21) [0x55587e671ae1]
python(_PyObject_Call_Prepend+0x63) [0x55587e663c43]
python(PyObject_Call+0x6e) [0x55587e65895e]
python(+0x17119a) [0x55587e6a619a]
python(_PyObject_FastCallKeywords+0x49b) [0x55587e6aed2b]
python(_PyEval_EvalFrameDefault+0x4bf6) [0x55587e70a026]
python(_PyEval_EvalCodeWithName+0xbb9) [0x55587e64bdb9]
python(_PyFunction_FastCallDict+0x1d5) [0x55587e64c5d5]
python(_PyEval_EvalFrameDefault+0x1e20) [0x55587e707250]
python(_PyEval_EvalCodeWithName+0x2f9) [0x55587e64b4f9]
python(_PyFunction_FastCallDict+0x1d5) [0x55587e64c5d5]
python(_PyObject_Call_Prepend+0x63) [0x55587e663c43]
python(+0x17116a) [0x55587e6a616a]
python(PyObject_Call+0x6e) [0x55587e65895e]
python(_PyEval_EvalFrameDefault+0x1e20) [0x55587e707250]
python(_PyEval_EvalCodeWithName+0xbb9) [0x55587e64bdb9]
python(_PyFunction_FastCallDict+0x1d5) [0x55587e64c5d5]
python(_PyObject_Call_Prepend+0x63) [0x55587e663c43]
python(PyObject_Call+0x6e) [0x55587e65895e]
python(_PyEval_EvalFrameDefault+0x1e20) [0x55587e707250]
python(_PyEval_EvalCodeWithName+0x2f9) [0x55587e64b4f9]
python(_PyFunction_FastCallDict+0x1d5) [0x55587e64c5d5]
python(_PyObject_Call_Prepend+0x63) [0x55587e663c43]
python(+0x17116a) [0x55587e6a616a]
python(_PyObject_FastCallKeywords+0x49b) [0x55587e6aed2b]
python(_PyEval_EvalFrameDefault+0x4bf6) [0x55587e70a026]
python(_PyFunction_FastCallDict+0x10b) [0x55587e64c50b]
/home/user/anaconda3/lib/python3.7/site-packages/torch/lib/libtorch_python.so: THPFunction_apply(_object*, _object*)
python(_PyMethodDef_RawFastCallKeywords+0x1e0) [0x55587e6ae660]
python(_PyCFunction_FastCallKeywords+0x21) [0x55587e6ae801]
python(_PyEval_EvalFrameDefault+0x4e8c) [0x55587e70a2bc]
python(_PyFunction_FastCallDict+0x10b) [0x55587e64c50b]
python(_PyObject_Call_Prepend+0x63) [0x55587e663c43]
python(PyObject_Call+0x6e) [0x55587e65895e]
python(_PyEval_EvalFrameDefault+0x1e20) [0x55587e707250]
python(_PyEval_EvalCodeWithName+0x2f9) [0x55587e64b4f9]
python(_PyFunction_FastCallDict+0x1d5) [0x55587e64c5d5]
python(_PyObject_Call_Prepend+0x63) [0x55587e663c43]
python(+0x17116a) [0x55587e6a616a]
python(_PyObject_FastCallKeywords+0x49b) [0x55587e6aed2b]
python(_PyEval_EvalFrameDefault+0x537e) [0x55587e70a7ae]
python(_PyFunction_FastCallDict+0x10b) [0x55587e64c50b]
python(_PyObject_Call_Prepend+0x63) [0x55587e663c43]
python(PyObject_Call+0x6e) [0x55587e65895e]
python(_PyEval_EvalFrameDefault+0x1e20) [0x55587e707250]
python(_PyEval_EvalCodeWithName+0x2f9) [0x55587e64b4f9]
python(_PyFunction_FastCallDict+0x1d5) [0x55587e64c5d5]
python(_PyObject_Call_Prepend+0x63) [0x55587e663c43]
python(+0x17116a) [0x55587e6a616a]
python(_PyObject_FastCallKeywords+0x49b) [0x55587e6aed2b]
python(_PyEval_EvalFrameDefault+0x4bf6) [0x55587e70a026]
python(_PyEval_EvalCodeWithName+0xbb9) [0x55587e64bdb9]
python(_PyFunction_FastCallDict+0x1d5) [0x55587e64c5d5]
python(_PyObject_Call_Prepend+0x63) [0x55587e663c43]
python(PyObject_Call+0x6e) [0x55587e65895e]
python(_PyEval_EvalFrameDefault+0x1e20) [0x55587e707250]
python(_PyEval_EvalCodeWithName+0x2f9) [0x55587e64b4f9]
python(_PyFunction_FastCallDict+0x1d5) [0x55587e64c5d5]
python(_PyObject_Call_Prepend+0x63) [0x55587e663c43]
python(+0x17116a) [0x55587e6a616a]
python(_PyObject_FastCallKeywords+0x49b) [0x55587e6aed2b]
python(_PyEval_EvalFrameDefault+0x537e) [0x55587e70a7ae]
python(_PyFunction_FastCallKeywords+0xfb) [0x55587e6ad79b]
python(_PyEval_EvalFrameDefault+0x6a0) [0x55587e705ad0]
python(_PyEval_EvalCodeWithName+0x2f9) [0x55587e64b4f9]
python(PyEval_EvalCodeEx+0x44) [0x55587e64c3c4]
python(PyEval_EvalCode+0x1c) [0x55587e64c3ec]
python(+0x22f874) [0x55587e764874]
python(PyRun_FileExFlags+0xa1) [0x55587e76eb81]
python(PyRun_SimpleFileExFlags+0x1c3) [0x55587e76ed73]
python(+0x23ae5f) [0x55587e76fe5f]
python(_Py_UnixMain+0x3c) [0x55587e76ff7c]
/lib/x86_64-linux-gnu/libc.so.6: __libc_start_main
python(+0x1e0122) [0x55587e715122]

It seems that there is some illegal access, but I don’t get this error on CPU.

archibate · 2020 年6 月 4 日 01:33

Try export TI_USE_UNIFIED_MEMORY=0?