Installing Llama.cpp with Python (Install & Coding)
Added 2024-06-15 23:00:59 +0000 UTCThis is such an exciting tutorial! I walk you through every step necessary to bring Llama.cpp into the world of Python programming, NVIDIA Inference, and onto your browser! This tutorial includes the actual Python code to get you up and running. This means you can start building your own apps today!
We go over every step necessary to get your system read for this NVIDIA build, set up critical system variables, and walk you through the code that will get your chatbot running in your browser! Whew! I'm tired just thinking about it! What took me hours to sort out, will take you 30 minutes.. and most of that simply waiting for installs.
Keep in mind this is a difficult build and has capacity for build errors. However, if you follow my tutorial in a procedural manner, you should be able to breeze through this in no time at all!
What an exciting milestone for you all!
The code used in this tutorial can be found at:
https://www.cognibuild.ai/python-llama-cpp
The following video will help you set up miniconda to use NVIDIA packages:
https://youtu.be/r6fUUg0aZ7g?si=Gdeu1gZA45WoI7Aw
You will need:
VIsiual Studio Build Tools (I suggest 2019)
Miniconda
Cuda
(possibly) cuDNN
