In my C++ application I am using embedded python interpreter using the stable C API to execute python scripts. It works well, but I want to extend this functionality to be able to configure a virtualenv path to use instead of the system's python packages.
What I gathered is that I have to set some variables before calling Py_Initialize to make it use the virtualenv. I tried a lot of combination of Py_SetPythonHome, Py_SetPath, Py_SetProgramName and setting environment variables PATH, VIRTUAL_ENV, PYTHONPATH, but none of the combinations seem so to work.
In my last appended my PATH environment variable with the virtualenv's bin directory, added the VIRTUAL_ENV environment variable with the virtualenv path, set the PYTHONPATH environent variable to the list of subdirectories of the virtualenv and called Py_SetPythonHome with the virtualenv path.
std::string old_path = utils::Environment::getEnvironmentVariable("PATH").value();utils::Environment::setEnvironmentVariable("PATH", ((virtualenv_path_ / "bin").string() +":" + old_path).c_str());utils::Environment::setEnvironmentVariable("VIRTUAL_ENV", virtualenv_path_.string().c_str());auto lib_path = virtualenv_path_ / "lib" ;std::string python_dir_name;for (auto const& dir_entry : std::filesystem::directory_iterator{lib_path}) { if (minifi::utils::string::startsWith(dir_entry.path().filename().string(), "python")) { python_dir_name = dir_entry.path().filename().string(); break; }}static const auto package_path = lib_path / python_dir_name / "site-packages";static const auto python_dir_path = lib_path / python_dir_name;static size_t virtualenv_path_size = virtualenv_path_.string().size();Py_SetPythonHome(Py_DecodeLocale(virtualenv_path_.c_str(), &virtualenv_path_size));utils::Environment::setEnvironmentVariable("PYTHONPATH", (package_path.string() +"/:" + lib_path.string() +"/").c_str());After starting up the application it seems that I have the right directories set, but I still get an error (the virtualenv path is /home/user/pythonrunnerapp/pyenv):
Python path configuration: PYTHONHOME = '/home/user/pythonrunnerapp/pyenv' PYTHONPATH = '/home/user/pythonrunnerapp/pyenv/lib/python3.10/site-packages/:/home/user/pythonrunnerapp/pyenv/lib/' program name = 'python3' isolated = 0 environment = 1 user site = 1 import site = 1 sys._base_executable = '/home/user/pythonrunnerapp/pyenv/bin/python3' sys.base_prefix = '/home/user/pythonrunnerapp/pyenv' sys.base_exec_prefix = '/home/user/pythonrunnerapp/pyenv' sys.platlibdir = 'lib' sys.executable = '/home/user/pythonrunnerapp/pyenv/bin/python3' sys.prefix = '/home/user/pythonrunnerapp/pyenv' sys.exec_prefix = '/home/user/pythonrunnerapp/pyenv' sys.path = ['/home/user/pythonrunnerapp/pyenv/lib/python3.10/site-packages/','/home/user/pythonrunnerapp/pyenv/lib/','/home/user/pythonrunnerapp/pyenv/lib/python310.zip','/home/user/pythonrunnerapp/pyenv/lib/python3.10','/home/user/pythonrunnerapp/pyenv/lib/python3.10/lib-dynload', ]Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encodingPython runtime state: core initializedModuleNotFoundError: No module named 'encodings'Current thread 0x00007f33a4b50780 (most recent call first):<no Python frame>If I don't do any of this, but I activate the environment before running the application and the executing it from that shell it works.What is the proper way to use an embedded python in a virtualenv here without the need to activate it beforehand?