Undefined symbol but symbol is there when importing _pywrap_tensorflow_internal.so

[Mo Zhou]

Hi mensors,

Tensorflow's python package is almost in shape[1] but when imporing
tensorflow from python, python complains that the symbol is undefined

_ZN10tensorflow25CreateRemoteMemoryManagerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_

however with readelf -sW xxx | ripgrep, the symbol is indeed in the
shared object. I also checked the build and the CXX file which provides
this symbol is indeed included in the object list when linking the
shared object in question.

Does anyone have idea on this problem?

```
(sid-lumin)***@wrath:/dev/shm/tensorflow.pkg/tensorflow$ PYTHONPATH=. python3 tensorflow/python/tools/api/generator/create_python_api.py --root_init_template=tensorflow/api_template.__init__.py --apidir=tensorflow --package=tensorflow.python --apiname=tensorflow api_init_files_list.txt
Traceback (most recent call last):
File "/dev/shm/tensorflow.pkg/tensorflow/tensorflow/python/pywrap_tensorflow_internal.py", line 14, in swig_import_helper
return importlib.import_module(mname)
File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 658, in _load_unlocked
File "<frozen importlib._bootstrap>", line 571, in module_from_spec
File "<frozen importlib._bootstrap_external>", line 922, in create_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: /dev/shm/tensorflow.pkg/tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so: undefined symbol: _ZN10tensorflow25CreateRemoteMemoryManagerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/dev/shm/tensorflow.pkg/tensorflow/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "/dev/shm/tensorflow.pkg/tensorflow/tensorflow/python/pywrap_tensorflow_internal.py", line 17, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "/dev/shm/tensorflow.pkg/tensorflow/tensorflow/python/pywrap_tensorflow_internal.py", line 16, in swig_import_helper
return importlib.import_module('_pywrap_tensorflow_internal')
File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: /dev/shm/tensorflow.pkg/tensorflow/_pywrap_tensorflow_internal.so: undefined symbol: _ZN10tensorflow25CreateRemoteMemoryManagerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "tensorflow/python/tools/api/generator/create_python_api.py", line 27, in <module>
from tensorflow.python.tools.api.generator import doc_srcs
File "/dev/shm/tensorflow.pkg/tensorflow/tensorflow/__init__.py", line 24, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "/dev/shm/tensorflow.pkg/tensorflow/tensorflow/python/__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
File "/dev/shm/tensorflow.pkg/tensorflow/tensorflow/python/pywrap_tensorflow.py", line 74, in <module>
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "/dev/shm/tensorflow.pkg/tensorflow/tensorflow/python/pywrap_tensorflow_internal.py", line 14, in swig_import_helper
return importlib.import_module(mname)
File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 658, in _load_unlocked
File "<frozen importlib._bootstrap>", line 571, in module_from_spec
File "<frozen importlib._bootstrap_external>", line 922, in create_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: /dev/shm/tensorflow.pkg/tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so: undefined symbol: _ZN10tensorflow25CreateRemoteMemoryManagerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/dev/shm/tensorflow.pkg/tensorflow/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "/dev/shm/tensorflow.pkg/tensorflow/tensorflow/python/pywrap_tensorflow_internal.py", line 17, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "/dev/shm/tensorflow.pkg/tensorflow/tensorflow/python/pywrap_tensorflow_internal.py", line 16, in swig_import_helper
return importlib.import_module('_pywrap_tensorflow_internal')
File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: /dev/shm/tensorflow.pkg/tensorflow/_pywrap_tensorflow_internal.so: undefined symbol: _ZN10tensorflow25CreateRemoteMemoryManagerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_


Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/install_sources#common_installation_problems

for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.
(sid-lumin)***@wrath:/dev/shm/tensorflow.pkg/tensorflow$ readelf -sW _pywrap_tensorflow_internal.so | rg _ZN10tensorflow25CreateRemoteMemoryManagerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_
1146: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND _ZN10tensorflow25CreateRemoteMemoryManagerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_
197099: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND _ZN10tensorflow25CreateRemoteMemoryManagerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_
(sid-lumin)***@wrath:/dev/shm/tensorflow.pkg/tensorflow$
```


[1] https://salsa.debian.org/science-team/tensorflow/tree/lumin
rules is broken due to significant change to the build system.

[Andrey Rahmatullin]

Post by Mo Zhou
python complains that the symbol is undefined
_ZN10tensorflow25CreateRemoteMemoryManagerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_
however with readelf -sW xxx | ripgrep, the symbol is indeed in the
shared object.

[...]

Post by Mo Zhou
1146: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND _ZN10tensorflow25CreateRemoteMemoryManagerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_
197099: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND _ZN10tensorflow25CreateRemoteMemoryManagerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_

I don't know about this output format but if UND means undefined it's not
really "the symbol is indeed in the shared object".

Post by Mo Zhou
I also checked the build and the CXX file which provides
this symbol is indeed included in the object list when linking the
shared object in question.

Are you sure that file really provides this symbol? Are you sure it's the
shared object in question? Why is it in /dev/shm?

[Mo Zhou]

Post by Andrey Rahmatullin

Post by Mo Zhou
python complains that the symbol is undefined
_ZN10tensorflow25CreateRemoteMemoryManagerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_
however with readelf -sW xxx | ripgrep, the symbol is indeed in the
shared object.

[...]

Post by Mo Zhou
1146: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND _ZN10tensorflow25CreateRemoteMemoryManagerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_
197099: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND _ZN10tensorflow25CreateRemoteMemoryManagerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_

I don't know about this output format but if UND means undefined it's not
really "the symbol is indeed in the shared object".

I found this:
https://stackoverflow.com/questions/3065535/what-are-the-meanings-of-the-columns-of-the-symbol-table-displayed-by-readelf

Further investigation into the CXX source suggests that I didn't
give the preprocessor a proper flag which guards the implementation,
such that actually the function is referenced by some other components
in the same shared object, but the function isn't defined.

I have fixed this "undefined symbol" error, by excluding related objects
since they are not important for the final shared object.

Post by Andrey Rahmatullin

Post by Mo Zhou
I also checked the build and the CXX file which provides
this symbol is indeed included in the object list when linking the
shared object in question.

Are you sure that file really provides this symbol? Are you sure it's the
shared object in question? Why is it in /dev/shm?

No, it doesn't provide such symbol. What's in question is indeed the
shared object itself. And /dev/shm is simply the build directory
because the machine has 512GB of RAM.

Post by Andrey Rahmatullin
--
WBR, wRAR

[Andrey Rahmatullin]

Post by Mo Zhou
And /dev/shm is simply the build directory
because the machine has 512GB of RAM.

I don't think it's a good idea to abuse /dev/shm as a normal tmpfs, you
should mount your own instead.