Jarvis Tutorial and code walk through Part 1

I had the same problem. You have to google and install Git. After installing you have to reboot CMD, then the "git" and "clone" commands will work.

gerrit

How did you fix it

Logan Ridley

fixed it!

Jordan Stroud

when I type git clone part in command prompt I get this 'git' is not recognized as internal or external command, operable program or batch file. What am I doing wrong?

Jordan Stroud

In the minute 15 I put run and it says this: PS C:\Users\ivanc\Desktop\Projects\belfast> & C:/Users/ivanc/AppData/Local/Programs/Python/Python310/python.exe c:/Users/ivanc/Desktop/Projects/belfast/assist.py File "c:\Users\ivanc\Desktop\Projects\belfast\assist.py", line 25 thread.id ^^^^^^^^^ SyntaxError: invalid syntax. Perhaps you forgot a comma? PS C:\Users\ivanc\Desktop\Projects\belfast> what should I do?

Iván Cámpora

I got everything running! I just can't update/change the voice

Agentloc

I can’t get it to respond to the tell me a joke response. What do I do?

Logan

Which API key do you generate for OpenAI. They have project keys, which start with sk-proj - Which don't work with your assist.py code. I always get invalid API key. The real question I have here is how do you assign the assistant to a project, so the project api key will work?

Jedd

After 2 weeks I couldn't figure out how to get this to speak through my speakers and had no help whatsoever, so unfortunately I'm giving up and no longer going to pay for the patreon, good luck to everyone else

Weecho Brownhawk

I actually had the same issue and i solved it by installing VS build tools.

Liam Williamsson

I have Anaconda could you send me the environment?

Sade K

You need to install cmake and go to the pytorch website to see compatible versions and once you have that you need to download microsoft visuals c/c++ on the microsoft website

Dev Narang

I've been working at getting this to speak through the speakers for 3 days now and I cannot get it at all, I have followed this to a T but I can't seem to hear jarvis speak. Anyone have any tips??

Weecho Brownhawk

This is probably a dumb question but I got everything else right expect I don’t know how to use the api key I generated 1 but I don’t know how to put it on python as a environment variable I’m very new to coding and it won’t speak back to me until I do

nick sala

Yea I did I started over using python version 11

nick sala

Did you ever figure it out because I am having a very similar problem.

John Fitzgerald

Okay, I got up to def TTS but when I run it to tell me a joke, it won't play through the speaker, it'll answer it in text but not out loud, what do I do?

Weecho Brownhawk

I am Assuming I will Need to figure this out before moving to that tutorial as he is still using the first tutorial to run everything and without my api key issues resolved i cant even run the program correct?

Cody

same but on the new tutorial

Alexander Jones

Struggling with getting openai api key to work can anyone help?

Cody

Ok downloading 3.11 rn and anaconda thank you will follow up shortly

nick sala

Hey, so I'm not sure your exact problem, but I have a few recommendations if you are looking to try some stuff. The first would be try with python 3.11. that's the version I am running. The next would be to start using an environment manager like anaconda. That will save you a bunch of headache for this project and many others. If you use anaconda I could always just send you my environment which should make it run without issues. I am also working on an application version of this so that you could just download the software and not worry about setup.

Kevin Coda

I’ve been stuck for 2 days on this and I’m honestly so close to breathing out fire from my nostrils 😭 I can’t get pygame installed it keeps saying failed building wheel for pygame and then says error failed build installable wheels for some pyproject.toml based projects (pregame) and is likely not a problem with pip I don’t know how to fix it I’ve watch 100s of YouTube vidoes and found so many Reddit chats and I can’t find anything someone please help I’m running python version 3.13 on a Mac OS 13.6.6..

nick sala

I'm getting stuck at "main loop starting" now. And when it did work i only got the listening for hotwords loop to work three times and then it errored. Any thoughts?

Jonathan Roberts

I get the same thing. It takes about a minute and then i get a whole bunch of errors: Traceback (most recent call last): File "C:\Users\Mainpc\Documents\jarvis\jarvis.py", line 109, in main() File "C:\Users\Mainpc\Documents\jarvis\jarvis.py", line 80, in main result = audio_model.transcribe(temp_file, fp16=torch.cuda.is_available()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Mainpc\AppData\Local\Programs\Python\Python312\Lib\site-packages\whisper\transcribe.py", line 133, in transcribe mel = log_mel_spectrogram(audio, model.dims.n_mels, padding=N_SAMPLES) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Mainpc\AppData\Local\Programs\Python\Python312\Lib\site-packages\whisper\audio.py", line 140, in log_mel_spectrogram audio = load_audio(audio) ^^^^^^^^^^^^^^^^^ File "C:\Users\Mainpc\AppData\Local\Programs\Python\Python312\Lib\site-packages\whisper\audio.py", line 58, in load_audio out = run(cmd, capture_output=True, check=True).stdout ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Mainpc\AppData\Local\Programs\Python\Python312\Lib\subprocess.py", line 548, in run with Popen(*popenargs, **kwargs) as process: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Mainpc\AppData\Local\Programs\Python\Python312\Lib\subprocess.py", line 1026, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "C:\Users\Mainpc\AppData\Local\Programs\Python\Python312\Lib\subprocess.py", line 1538, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [WinError 2] The system cannot find the file specified

Jonathan Roberts

Fixed it! Had date_queue not data!

Jonathan Roberts

Hi all. Newbie here. I've managed to get the assist file up and running, but i am struggling with the jarvis file. The data_queue is coming up as undefined. I am guessing it has something to do with the import queue from Queue line. Any thoughts would be appreciated. Cheers.

Jonathan Roberts

I have been doing this for the past week and i cant do it, i dont understand most of it every time i try to do something its always says error or no module named OpenAI can someone reach out to me on discord it is military_kid08 (dont judge the name)

Unknown Person

Hi, i followed every step and installed everything from the requirements.txt file but it keeps giving me this error: "ModuleNotFoundError: No module named 'torch'". i searched for any solution related to vs code online but it didn't help. could anyone please help me? thanks in advance

Federico Parisi

For ease use state it like openai_api_key="yourkey" client = OpenAI(api_key=openai_api_key,default_headers={"OpenAI-Beta": "assistants=v2"}) That is in the assist.py script

Sheetyman Sheety

i tried doing this but everytime io try to run it, it says "The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable" even after i set my envirment

Tazer Grazer

Unfortunately no.

Mack Marchant

How to include the shader to JARVIS

Sam R

Did you ever figure this out?

Marquis

I am running on a Raspberry PI 4 btw.

Robert E

Also do you respond to direct messages? I sent you a few questions over a week ago and have not seen a response.

Robert E

I have Python 3.10.0 & PIP 24.1.1. I have reloaded Python & PIP several times trying to find a version that will install Torch.

Robert E

What version of Python and PIP are you running. I am having difficulty loading Torch.

Robert E

but yes you can open your terminal and type the command choco install ffmpeg it will help you...

WhiteBoss Gamer

same error message

WhiteBoss Gamer

Fixed! run in the terminal choco install ffmpeg

Juan Carlos P. Melgar

Juan Carlos P. Melgar

could you be more specific?

Ricardo Lesmes

Hey can any one tell me there was an error FileNotFoundError: [WinError 2] The system cannot find the file specified but there was an file too

WhiteBoss Gamer

Actually, you can ask it to change the language, the model understood me when I spoke to it in Spanish, give it a try

Juan Carlos P. Melgar

pip install speechrecognition

Matt Goeckel

its telling me there's no module named 'speech_recognition' from the Jarvis code import list

Alexander Lack

for some reason once i get to jarvis responding nothing happens. i dont get an error or anything.

Sebastian Hidalgo

Thank you so much it worked

SnappyApple7343

The way you set your API key is within the brackets type api_key = 'actual key'

Sartaj Gill

Has anybody gotten this error when making Jarvis: INTEL MKL ERROR: The specified module could not be found. mkl_vml_def.1.dll. Intel MKL FATAL ERROR: cannot load mkl_vml_def.1.dll?

TheK-B1000

Can anyone help me with this. On line 7 it says this client = OpenAI("MY API KEY") I get an error that says TypeError: OpenAI.__init__() takes 1 positional argument but 2 were given. (Where I put MY API KEY, that's my api key I don't want to leak it)

SnappyApple7343

Did you find a fix for this?

Jayden Raby

Hey, I have a thinkpad laptop with a ryzen 7. Its just a windows machine. Nothing special. Not sure why your command prompt wouldn't be working. That seems like a windows problem. Is it launching? can you type anything?

Kevin Coda

What computer and software do you use, my command prompt isn’t working

Gavin Pena

Can anyone help with the following? AttributeError: ‘NoneType’ object has no attribute ‘close’

Spyder Flag Check

Anyone know about this? Says it’s an issue with the file speech_recognition line 189 self.stream.close()

Spyder Flag Check

I seem to finally have made moves but I’m stuck here now AttributeError: ‘nonetype’ object has no attribute ‘close’

Spyder Flag Check

Hi All, Im getting error in this line of code can you please help me out Model loaded. main loop starting e:\FutureAI\jarvis\jarvis.py:63: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC). now = datetime.utcnow() Traceback (most recent call last): File "e:\FutureAI\jarvis\jarvis.py", line 127, in main() File "e:\FutureAI\jarvis\jarvis.py", line 89, in main result = audio_model.transcribe(temp_file, fp16=torch.cuda.is_available())

Sam R

Mine doesnt go past main loop starting any help?

Jack Ross

You can create your own python script to detect the microphones, something like this: # List audio input devices p = pyaudio.PyAudio() for i in range(p.get_device_count()): info = p.get_device_info_by_index(i) print(f"Device {i}: {info['name']}") p.terminate() this should return the index of all connected devices. Hope this helps :)

Joshua Leyva

Great project! I finally got it working, and if I could do it believe me there is hope for everyone. I have a couple of questions though: 1-How do I know if the microphone is connected? From the way it is explained I have to mod the index, I have tried but with not great results, I have tried 0, 1 and 2. however it seems that Jarvis always has a little trouble hearing me, even if I use the headset with microphone which I am sure works # source = sr.Microphone(sample_rate=16000, device_index=0)# What happens if I change rate=16000? 2- Maybe it is related to point 1, often Jarvis to a question of mine answers before I am finished. In this case are there any values to adjust? 3- when will jarvis part 2 come out? i am very excited Overall great job, you explain super well even for a non-mother language

Giovanni Bavai

yes

Giovanni Bavai

Do we have to pay for OpenAI to use their Assistant API, or is it free

Sam R

I haven't done coding since 2005 so I'm not sure if any of this coding is system specific or if it even cares.

Christopher Smith

I am having trouble with the Torch library, I keep getting an error that the specified module could not be found.

MGScion

Nvm I had while mixer.music.get.busy() when it should be get_busy() lol

MGScion

There might be an issue where Visual Studio Code doesn’t have permissions to access or make mp3 files. Otherwise, I suggest using ChatGPT’s Python code helper. It was able to fix a lot of issues I had and change things to be more how I want them.

Owen Ehrlich

I am also having trouble converting TTS, it seems like the problem could be the python bug with the stream_to_file? My output does not convert nor stream it back to me

MGScion

I'm having an issue wherein the assistant (AKA Jarvis) responds back with my prompt for the question. For example, with the first example of Tell me a joke, I have my code setup like this: response = ask_question_memory("Tell me a joke") The result of the code is that Jarvis responds with "Tell me a joke". I can view his actual response to the joke on the OpenAI assistant browser, however this result does not get populated back to my code in Visual Studio. Same thing happens with the TTS, he responds with "Tell me a joke". I've doubled back in the video several times and I can't seem to figure out where the issue is coming from.

Mack Marchant

Hey! I followed along till 22:33 of the video and everything worked fine before that. Jarvis prints the joke but does not convert it to speech. This is the error I am getting: openai.PermissionDeniedError: Error code: 403 - {'error': {'message': 'Project `proj_TAmxxxxxxxx` does not have access to model `tts-1`', 'type': 'invalid_request_error', 'param': None, 'code': 'model_not_found'}} Can anyone help me with this?

Nihaara

Don't know if anyone has asked this yet or not but does this coding work on Pi? I know Pi is Linux based so does the coding change at all from windows to Linux to pi?

Christopher Smith

It's working in open AI fine but idk why its not working in visual studios

Austin

That error is actually from open AI. They return "the run has failed" if your open AI API token isn't working properly. Maybe check billing and your account to see if there are any errors

Kevin Coda

im having a problem where i tried to print a response and it tells me "the run has failed" that on line 44 'return' outside function

Austin

Thank you

Christian Wadkins-white

This is the correct answer and worked to solve this. Thank you

Seth Johnson

Hey, can you do a video explaining how to have it play music, and put the components together into a box?

Oliver

Fixed it, forgot to put my assistant id🤷

stefan doudin

the openai whisper modules also requires the FFMpeg module which you can download using Chocolatey, here are 2 links to help u install both:https://chocolatey.org/install?ref=assemblyai.com https://www.assemblyai.com/blog/how-to-run-openais-whisper-speech-recognition-model/

stefan doudin

for some reason when I launch jarvis.py it without anything else, first tells me a joke(which was one of the prompts i asked it, if not the last while testing the assist.py) then it loads the model says the date time thing is depracted, says main loop is starting and just doesnt do anything, doesnt stop, doesnt write listening just doesnt do anything? Any fix, maybe i am doing something wrong.

stefan doudin

try pip install setuptools, worked for me

stefan doudin

in the assist.py file, the line: client = OpenAI(), do this client = OpenAI(api_key="api key"). You can create an api key when u have resgistered an account with openai and it will show it to you only once so save it somewhere.

stefan doudin

Finally got it to work and it’s awesome. Is there a way I can keep a continuous conversation with him, without having to say “Jarvis” every time?

Owen Ehrlich

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "c:\Users\chris\OneDrive\Desktop\Jarvis project\jarvis\whisper\transcribe.py", line 133, in transcribe mel = log_mel_spectrogram(audio, model.dims.n_mels, padding=N_SAMPLES) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "c:\Users\chris\OneDrive\Desktop\Jarvis project\jarvis\whisper\audio.py", line 140, in log_mel_spectrogram audio = load_audio(audio) ^^^^^^^^^^^^^^^^^ File "c:\Users\chris\OneDrive\Desktop\Jarvis project\jarvis\whisper\audio.py", line 58, in load_audio out = run(cmd, capture_output=True, check=True).stdout ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\chris\AppData\Local\Programs\Python\Python312\Lib\subprocess.py", line 548, in run with Popen(*popenargs, **kwargs) as process: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\chris\AppData\Local\Programs\Python\Python312\Lib\subprocess.py", line 1026, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "C:\Users\chris\AppData\Local\Programs\Python\Python312\Lib\subprocess.py", line 1538, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [WinError 2] The system cannot find the file specified

Christian Wadkins-white

When I try running it, it says the file specified could not be found even though it is all in the right location.

evan '

ModuleNotFoundError: No module named 'distutils'

Christian Wadkins-white

I'm stoked to see how the new chatgpt model will enhance the jarvis experience. These builds your sharing are phenomenal! Thanks!

Sebek's Tech Trek

I am also having this same error. It won’t say it’s listening but soon as I start speaking that error pops up

Sebek's Tech Trek

GitHub Copilot if I’m not mistaken. It’s an extension in VS code. You also have to have a GitHub account

Sebek's Tech Trek

https://platform.openai.com/settings/organization/billing/overview

Vincent

does it cost something ?

Cosmo

Change the Mic Input, but it is slow Im using a mac m1 with no gpu, so i change the hotword detection to picovoice.ai (for free) and the text to speach using azure cognitive service. Got a better performance

Christiano Faig

hey did you find any solution stuck in same part

Kau Mu

make sure to install ffmpeg

Koby Deuel

When I try to run the code it keeps saying that the data queue is empty

Anthony

I’m having trouble getting Jarvis to listen and right at the end, I have him talking up a storm but can’t get the code to start listening

Mathew Russell

how do i put in this api key it just doesn't want to work

Luke Davies

hi im new to coding, does anyone know what shortcut he uses or how he generates the grey sentences cuz i rlly dont know

Edwin Hsieh

hang in : pygame 2.5.2 (SDL 2.28.3, Python 3.11.7) Hello from the pygame community. https://www.pygame.org/contribute.html Model loaded. main loop starting

Christiano Faig

So it works but when I say "hey jarvis" it gives me the same error

Kaydin Smith

I've found the solution, I've executed VS Studio as an admin and that worked.

Mael

Can you consider providing instructions to run Jarvis on a local LLM not dependent on OpenAI API? Maybe utilizing Ollama and Anything2LLM

Jonathan Luna

i'm getting that also.

Luke

I'm on a Mac too, I'm in the same boat as you, I know if my microphone is poorly detected or the model doesn't understand me, then you can test the tiny model which is multilingual.

Valkim

That makes sense....

Juan Carlos P. Melgar

Can’t wait for part 2!

Scarlett

No, I have not.

Kaydin Smith

I had the same problem, you have to pay for the api, openai no longer gives out free credits.

Mael

I have the same problem. Have you found a solution?

Mael

I keep getting this when i try to run "python .\jarvis.py PS C:\Users\Kaydin Smith\Desktop\Projects\jarvis> python .\jarvis.py pygame 2.5.2 (SDL 2.28.3, Python 3.12.3) Hello from the pygame community. https://www.pygame.org/contribute.html main loop starting C:\Users\Kaydin Smith\Desktop\Projects\jarvis\jarvis.py:61: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC). now = datetime.utcnow() Traceback (most recent call last): File "C:\Users\Kaydin Smith\Desktop\Projects\jarvis\jarvis.py", line 121, in main() ^^^^^^ File "C:\Users\Kaydin Smith\Desktop\Projects\jarvis\jarvis.py", line 87, in main result = audio_model.transcribe(temp_file, fp16=torch.cuda.is_available()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Kaydin Smith\Desktop\Projects\jarvis\whisper\transcribe.py", line 133, in transcribe mel = log_mel_spectrogram(audio, model.dims.n_mels, padding=N_SAMPLES) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Kaydin Smith\Desktop\Projects\jarvis\whisper\audio.py", line 140, in log_mel_spectrogram audio = load_audio(audio) ^^^^^^^^^^^^^^^^^ File "C:\Users\Kaydin Smith\Desktop\Projects\jarvis\whisper\audio.py", line 58, in load_audio out = run(cmd, capture_output=True, check=True).stdout ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.1008.0_x64__qbz5n2kfra8p0\Lib\subprocess.py", line 548, in run with Popen(*popenargs, **kwargs) as process: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.1008.0_x64__qbz5n2kfra8p0\Lib\subprocess.py", line 1026, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.1008.0_x64__qbz5n2kfra8p0\Lib\subprocess.py", line 1538, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [WinError 2] The system cannot find the file specified

Kaydin Smith

The error message says that content has not one entry. The data comes form the OpenAI Libray so I think your configuration of GitHub has a mistake. Maybe you have the wrong API key or you don't have any credits for calls anymore.

Florian Middendorf

For my Mac guys. I have to install via brew this two things on the side of the documentation. brew install ffmpeg brew install portaudio then it works. But the model understand me really bad. Maybe because my German accent. But I will try some things.

Florian Middendorf

you have to install apenai 0.28 i had the same issue on my last programm pip install openai==0.28

Jakob Zwingmann

Kaydin Smith

Cheers that worked!

Nico Dwyer

Uninstall then reinstall. Pip uninstall openai Pip install openai

Theo

I have the same problem dose anyone know the solution to this? Thanks

Nico Dwyer

ImportError: cannot import name 'OpenAI' from 'openai' (C:\Users\Theo\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\openai\__init__.py) even though open ai is succesfully pip installed

Theo

Yes, definitely, I'm also really happy about the community you're creating here. I'm personally from another country and speak a completely different language, and it makes me very happy to see people from all over the world gather together to hear you and learn more about this amazing world.

royyalbert

i have a new error. with the api. You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.

Theo

Yeah absolutely. However its super simple and I encourage you to try it. In the same way I look for #s I also look for ?'s. If Jarvis asks a question then the conversation is still going so I toggle a boolean to make the loop not look for hotwords the next time around and just send the next text to the ai if its not ''. I think a few people asked about this so I will include it for next time!

Kevin Coda

Can you show us in the next episode how you continue to talk to him for a whole conversation without saying HOT words?

royyalbert

https://github.com/git-guides/install-git

Kevin Coda

git is not recognised as an internal or external command - command promt

Theo

Yaaaa

Infinite Tech