Python-Llama-CPP (Nvidia Inference)

Below are the files used in the tutorial at:

set CMAKE_ARGS="-DLLAMA_CUBLAS=on"
set CUDACXX="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\nvcc.exe"
pip install llama-cpp-python[server] --upgrade --force-reinstall --no-cache-dir

Code 1

Code 2