SamSuka
Concept Bytes
Concept Bytes

patreon


Program your own Jarvis In just 20 lines of code!

Jarvis is easier to build than ever!

Hey everyone. If you are looking to make your own Jarvis this is the place to start. I have spent some time simplifying the process. This is a much cleaner and straightforward approach to programming your own Jarvis. So fire up your terminals, download VSCode, install python and lets get to it!

How Jarvis Comes to Life

1. Speech Recognition with Whisper V3: Our journey begins with converting your spoken words into text. Whisper V3, a robust model developed by OpenAI, excels in understanding and transcribing speech in real-time. This ensures that no matter how fast or slow you speak, Jarvis hears you loud and clear. https://github.com/openai/whisper

2. Intelligent Interactions with GPT-4o Turbo: Once your speech is transcribed, the next step is understanding and generating a fitting response. This is where GPT-4o Turbo comes in. It's not just any chatbot; it's designed to understand context and subtleties in language, making interactions seamless and engaging—almost like you’re talking to a human! https://platform.openai.com/docs/assistants/overview?context=with-streaming

3. Speaking Back with OpenAI’s TTS: After crafting the perfect response, our system uses OpenAI’s Text-to-Speech technology to verbalize the answer. This isn’t a robotic voice; it’s a natural, smooth, and dynamic speech that converses with you just like Jarvis would. https://platform.openai.com/docs/guides/text-to-speech

What you will need:
1)
 Install Python: https://www.python.org/downloads/
2) Download  FFMPEG: https://ffmpeg.org/download.html
3) I use VSCode: https://code.visualstudio.com/download (optional)

4) Get started with the open ai API here: https://platform.openai.com/docs/quickstart?context=python

5) Create your Jarvis assistant here: https://platform.openai.com/assistants

Start Coding: https://github.com/Concept-Bytes/Jarvis

-Note that this is Just version 1 and that all the features in my video are not available just yet.

-Yes they will release as I am getting them ready for you guys asap.

-This is still a full Assistant pipeline that you can talk to and trigger actions.

Features at a Glance:

Join the Journey!

This is just the beginning of what we can achieve with AI in personal assistants. Whether you’re a tech enthusiast, a developer, or someone interested in the future of AI interactions, your support can make a significant difference.

Comments

hello my ai isnt transcribing what i say back at all and i have the code exactly the same it hears me but doesnt say anything back what do i do

Vain Bold

Install scipy 1.14.1 using pip install scipy=1.14.1

Mona Shehata

have the same problem but if i can fix it i tell you

Tom Höner

My assistence doesnt work can anybody help?

Tom Höner

can I have a female sound like Friday, not Jarvis?

Zaid Hamdan

ive been give this error when i install RealtimeSST, ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. realtimestt 0.3.93 requires scipy==1.14.1, but you have scipy 1.13.1 which is incompatible.

jake

i was hoping someone could help me

jake

I already installed RealtimeSTT, but still the error exist.

Taroooq M

But it's not being used by another app I'm sure of it

Ammar

pip install RealtimeSTT, check your python version! RealtimeSTT does not work on current version, as they have not updated it.

Jaron Wilson

You need a headset plugged in or an audio device! Microphone, if its being used by another app that could collide.

Jaron Wilson

pip install RealtimeSTT

Jaron Wilson

How to fix this issue?? PS C:\Users\tariq\OneDrive\Desktop\Jarvis-main> & C:/Users/tariq/OneDrive/Desktop/Jarvis-main/venv/Scripts/python.exe c:/Users/tariq/OneDrive/Desktop/Jarvis-main/jarvis.py Traceback (most recent call last): File "c:\Users\tariq\OneDrive\Desktop\Jarvis-main\jarvis.py", line 1, in from RealtimeSTT import AudioToTextRecorder ImportError: cannot import name 'AudioToTextRecorder' from 'RealtimeSTT' (unknown location) PS C:\Users\tariq\OneDrive\Desktop\Jarvis-main>

Taroooq M

you have any steps for doing this on a mac?

John Spinella

C:\Users\amsal\env\lib\site-packages\torchaudio\backend\utils.py:62: UserWarning: No audio backend is available. warnings.warn("No audio backend is available.") ERROR:root:Microphone connection failed: Selected device validation failed. Retrying... [2024-12-26 15:59:22.106] [ctranslate2] [thread 17396] [warning] The compute type inferred from the saved model is float16, but the target device or backend do not support efficient float16 computation. The model weights have been automatically converted to use the float32 compute type instead. Say something ERROR:root:Microphone connection failed: Selected device validation failed. Retrying... anyone know how to fix?

Ammar

hey I'm trying to figure out how to download the python version you have in the video but when i try it on the website link it says that the version is unavailable, do you know any other versions i can download to get it to work the same?

Ethan

HI SIR the discord link shows it can't be use

謹宇 蔡

There is indeed a workaround! I made a Localized Jarvis further up in the video posts here on the Patreon. This edition of Jarvis runs locally on your machine, is wayy faster (with the proper hardware), and you can use it indefinitely. However, this does mean the performance relies on hardware, and if you're running a GTX card or older AMD card, the performance will not be as optimal as regular Jarvis.

Berle

Yo! Please try seeking the Discord server for support questions like this. There's a lot of people who'd love to help you out! :D

Berle

still error line 1 how can ai fix it?

Ryan Casalme

i am getting the same thing, have you been able to fix it?

Travis Callaway

Open AI uses tokens which charges you money. Any work arounds maybe with huggingface?

6ino

Hi, am getting this error C:\Users\adria\Jarvis Assistant> & C:/Users/adria/AppData/Local/Programs/Python/Python313/python.exe "c:/Users/adria/Jarvis Assistant/Jarvis/Jarvis.py" Traceback (most recent call last): File "c:\Users\adria\Jarvis Assistant\Jarvis\Jarvis.py", line 1, in from RealtimeSTT import AudioToTextRecorder ModuleNotFoundError: No module named 'RealtimeSTT' I cannot figure out how to get RealtimeSTT to be seen as a module, are there any fixes? I am also very very new to coding and don't know if this is a very simple fix and I just don't know about it!

Adrian Anstead

New member. Very excited to start the journey. But, just wanted to acknowledge you and say you are "Awsome man" thanks for the shared knowledge 👍🏾

Warhead

Try this Jarvis2.py code: import whisper import pyaudio import wave import assist import time import tools # Initialize Whisper model model = whisper.load_model("tiny") # Audio recording configuration FORMAT = pyaudio.paInt16 CHANNELS = 1 RATE = 16000 CHUNK = 1024 RECORD_SECONDS = 10 # Adjust the duration as needed TEMP_AUDIO_FILE = "temp_audio.wav" def record_audio(): audio = pyaudio.PyAudio() stream = audio.open(format=FORMAT, channels=CHANNELS, rate=RATE, input=True, frames_per_buffer=CHUNK) print("Recording audio...") frames = [] for _ in range(0, int(RATE / CHUNK * RECORD_SECONDS)): data = stream.read(CHUNK) frames.append(data) stream.stop_stream() stream.close() audio.terminate() with wave.open(TEMP_AUDIO_FILE, "wb") as wf: wf.setnchannels(CHANNELS) wf.setsampwidth(audio.get_sample_size(FORMAT)) wf.setframerate(RATE) wf.writeframes(b"".join(frames)) print("Recording complete.") return TEMP_AUDIO_FILE def transcribe_audio(file_path): print("Transcribing audio...") result = model.transcribe(file_path) text = result["text"] print("Transcription complete:", text) return text if __name__ == '__main__': hot_words = ["Jarvis"] skip_hot_word_check = False print("Say something...") while True: audio_file = record_audio() current_text = transcribe_audio(audio_file) if any(hot_word in current_text.lower() for hot_word in hot_words) or skip_hot_word_check: if current_text: print("User: " + current_text) # Add timestamp to text current_text = current_text + " " + time.strftime("%Y-%m-%d %H-%M-%S") # Process the text through your assistant function response = assist.ask_question_memory(current_text) print(response) # Generate speech response speech = response.split('#')[0] done = assist.TTS(speech) # Skip hot word check if response is a question skip_hot_word_check = True if "?" in response else False # Parse command if available if len(response.split('#')) > 1: command = response.split('#')[1] tools.parse_command(command)

Javier Garza

I had to abandon RealtimeSTT and replace it with whisper for it to work. Couldn't get RealtimeSTT to work on my mac.

Javier Garza

I got this error: PS C:\Users\ivanc\Desktop\Projects\belfast> & c:/Users/ivanc/Desktop/Projects/belfast/.venv/Scripts/python.exe c:/Users/ivanc/Desktop/Projects/belfast/belfast.py File "c:\Users\ivanc\Desktop\Projects\belfast\belfast.py", line 1 from RealtimeSTT AudioToTextRecorder ^^^^^^^^^^^^^^^^^^^ SyntaxError: invalid syntax PS C:\Users\ivanc\Desktop\Projects\belfast> what should I do :c

Iván Cámpora

it's giving me a error, and the error message says: ModuleNotFoundError: No module named 'audio-recorder'

Tobey Martin

I had this myself, the new python 3.13 is not compatible with a lot of those dependencies. uninstall python and install a 3.12 or even the 3.08 and you should be fine

Charlie Pinedo

this is what it show me know " Fatal error in launcher: Unable to create process using '"C:\Users\danie\AppData\Local\Programs\Python\Python311\python.exe" "C:\Users\danie\AppData\Local\Programs\Python\Python311\Scripts\pip.exe" install RealtimeSTT': The system cannot find the file specified. "

Daniel

What is your python version ? It is not working in 3.13, I get similar message for this version when I download previous version 3.12, it is working

alperen öz

use 'pip3'

Agentloc

if pip is not installed python isn't installed try removing python are reinstalling it

Diego Gutierrez-Raghunath

man i give up i don't know how to fix this i can't do it

Filip_esty

help when i type pip install RealtimeSTT it says:pip : The term 'pip' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:1 char:1 + pip install RealtimeSTT + ~~~ + CategoryInfo : ObjectNotFound: (pip:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException

Filip_esty

when i install the "pip install RealtimeSTT" it shows me an error saying " note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for webrtcvad Failed to build webrtcvad ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (webrtcvad)"

Daniel

Hi. I am having the same issue. Did you fix it?

kshitiz

I am getting the same mesage. Did you get to solve this?

kshitiz

https://github.com/KoljaB/RealtimeSTT

Kevin Coda

i cant find the link to the Github from KoljaB

Lou Aschen

How did you connect your 3D printer to Jarvis? Also how did it access STL files?

Lux Tenebris

So my question is it able to add another languages to jarvis and interact with em for example Turkish.

Omar Ahmed Ghoneim

I had this issue, you can fix that with this: os.environ["OMP_NUM_THREADS"] = "1" it will force to the script to avoid conflict in your computer

Martin Lengua Olivares

I had to recover my Macbook Pro A1502... I mean, a computer from 11 years ago and it worked perfectly, there were some issues related to the interfaces of my computer is using but with some tiny changes it could work as expected. The performance is good but it might be better, thank you very much for your knowledge, it's amazing!!!

Martin Lengua Olivares

hey buddy, is there some computer requirements for your apps? I have a Macbook M1 and it appears that I'm having a lot of incompatibilities, so maybe I'll try to use a normal computer, but let me know if this is compatible with M1 processor please

Martin Lengua Olivares

use this instead it worked for me -> pip install RealtimeSTT --no-deps

Mohammed Fuzail

for the assistant playground do i have to buy api keys for that to work and where is the video explaining how to do it?

Oscar

spelling mistakes i can see one you said spiner instead of spinner

Oscar

I am unable to connect to the discord

Samuel Adranyi

Hey i cant get the chat gpt Api key

Patric Amariei

when i try to press play it says PS C:\Users\Ethan> & C:/Users/Ethan/AppData/Local/Microsoft/WindowsApps/python3.12.exe "c:/Users/Ethan/from realtimeSTT import audiototextrecor.py" File "c:\Users\Ethan\from realtimeSTT import audiototextrecor.py", line 4 recorder = audiototextrecorder(spiner=false, model="tyny.en, language="en", post_speech_silence_duration = 0.1) anybody know how to fix it

Ethan Nolasco

change the stilero sesitivity to a higher number like 10

Alexander Jones

mines isn working but everytime i talk into my mic it doesnt work.

Alyx Lane

My Jarvis works absolutely amazing however I want it to be able to play specific songs what do I need to do to make it do that?

Oliver

client = OpenAI(api_key="Your API Key here", default_headers={"OpenAI-Beta": "assistants=v2"}) This is the way I got it to work for myself

Sterling S

Please help I don’t know what I’m doing wrong it won’t let me speak to it, it keeps saying Say something... OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized. OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/. Traceback (most recent call last): File "/Users/27alone/Desktop/Jarvis-main/jarvis.py", line 9, in current_text = recorder.text() File "/Users/27alone/Desktop/Jarvis-main/.venv/lib/python3.10/site-packages/RealtimeSTT/audio_recorder.py", line 893, in text return self.transcribe() File "/Users/27alone/Desktop/Jarvis-main/.venv/lib/python3.10/site-packages/RealtimeSTT/audio_recorder.py", line 844, in transcribe status, result = self.parent_transcription_pipe.recv() File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/connection.py", line 255, in recv buf = self._recv_bytes() File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/connection.py", line 419, in _recv_bytes buf = self._recv(4) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/connection.py", line 388, in _recv raise EOFError EOFError

nick sala

You may need to additionally run pip install RealtimeSTT

Perciville

same problem here

Alexander Jones

bro how fix this problem File "c:\Users\Lenovo\Desktop\projects\Jarvis_4\Jarvis.py", line 4, in import assist File "c:\Users\Lenovo\Desktop\projects\Jarvis_4\assist.py", line 7, in client = OpenAI(default_headers={"OpenAI-Beta": "assistants=v2"}) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Lenovo\AppData\Local\Programs\Python\Python312\Lib\site-packages\openai\_client.py", line 105, in __init__ raise OpenAIError( openai.OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

Sabarikrishnan Chinnamuthu

I don't see an option for this other than VC 2022 community

Santosh Vijayakumar

PLEASE HELP 😭i cant get openai to work, i keep getting this error message; openai.OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

Alexander Jones

i didnt get an error but it is definitely safe

Alexander Jones

i fixed it by changing the interpreter so microsoft python

Alexander Jones

for the second one i couldnt install either, fixed by installing a more recent version of C++ build tools from microsoft store

Alexander Jones

Running the same pip as the video says resulted in the "Failed to build webrctvad " message others have got and directly pip-ing that also failed

Santosh Vijayakumar

Hi, I tried running the jarvis file as it is from the github repo on my VSC app but even after installing python for windows and importing the pip, I get this error: ModuleNotFoundError: No module named 'RealtimeSTT'

Santosh Vijayakumar

didnt work for me

Flashoutt

use this 'pip install webrtcvad-wheels'

Kraqtpott

Thank you for the videos, is there a way to implement a local LLM like Ollama and not utilize OpenAI?

Dale Allan

Hey Robert Collins, I had this error code, ERROR: Failed building wheel for webrtcvad Failed to build webrtcvad ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (webrtcvad), and I was able to fix it by downloading the c++ build tools on vs studio, not sure if that will help, but give it a shot.

Frodo Bacon

Hey anyway you can do a complete stand a lone walk through on the install process? With nothing installed. I'm pretty sure you are missing steps that you assume you already taken care of. I follow your video and install process but always have issues, not sure what the issue is. I could possibly throw more patronage your way for the help.

Chris McCormick

If you are using windows, you can try and install the visual studio, after that tick on everything that has C++ in it to install the dependencies. After that, you can try to reinstall again the RealtimeSTT

Trần Minh

Hi all, can i put into my alexa echo, someone know how to replace the custom assistant and put this one?

Alessandro Acciai

Yup likewise, wish he would explain more in detail and help out the ones in comments

Chris McCormick

Now I’m not getting the error about installing webrtcvab failing, but it keeps saying that there is no module named RealtimeSTT even though I did the pip install RealtimeSTT

Doug Reed

I'm having the same issue. It is displaying an error now saying "ERROR: Failed building wheel for webrtcvad" and also that it cannot find the RealtimeSTT

Doug Reed

I've just found a solution for the same issue. If you find client.py then set line 72 to api_key: str = "your api key",

jabob33

Hey there! Great video! I followed all the way through and everything seemed to look good. However when I ran the code, after importing assist and assigning the assistant and thread ID's, I was getting an error for the OpenAI API Key. Im not sure where i'm supposed to place my API key within the lines of code. I tried different placements and nothing seemed to connect it to OpenAI when called. Also, how would I get this to work with GPT-4o? I could really use some help! Thanks!!

Cameron Bahri

Hey, I have a question. Is fmpegg safe? i go to download it and i always get a warning that i can harm my pc. Idk what to do?

Jacob Mrgroth

On it! Thks!

מֶֹּשֱּׁהּ Moussan

Just did!

Kevin Coda

Hi did you pip install Realtimestt? In the terminal?

Kevin Coda

Hey, great comment! I just posted a tutorial that works towards getting images for the HoloMat using jarvis

Kevin Coda

Hey yes, download pi OS and it should come with python installed.

Kevin Coda

Hey Guys, just recieved my raspberry pi, do i need to Install raspberry os before downloading python? Or can I just download python on the sd card and start programming the code?

David Wendin

Great content! What I really want to know is how to integrate the UI element that are displayed on the videos, I mean the widgets, the camera etc

מֶֹּשֱּׁהּ Moussan

Great, I’m just connecting it with a Heygen Streaming Avatar!

מֶֹּשֱּׁהּ Moussan

I'm trying to get this to work. I think I've done everything right. I still get errors saying that _webrtvcad doesn't exist and also having issues with AudioToTextRecorder no seeming to exist. Can anyone help?

Robert Collins

+1

anas haboub

I get this error, ERROR: Failed building wheel for PyAudio when trying to install RealtimeSTT

Chris McCormick

I need a better setup/install guide because you really don't list everything to do exactly. I love your project, I just hope you can make this easier with steps. I can't get nothing to work.

Chris McCormick

I built Jarvis with the realtimeSTT and speech recognition fails sometimes. It works very well for the most part but there are times it just stops performing like it should. It will not be able to detect what I am saying. I need it to be good no matter what microphone I am using. Can anyone help guide or give tips on how to improve speech recognition?

TheK-B1000

Ho provato con un altra libreria. Devo dire che la situazione é migliorata. Capisce quello che dico ma chatgpt mi risponde con un italiano “inglesizzato”.

Matteo

mhh manco io ci riesco se trovo una soluzione te la passo

Stefano Manca

Im also having the same problem here

Rhvn1

When i try to install RealTimeStt it says it cant find it did i miss a step on installing it?

Whyits_me

Thank you, so much! There were some obstacles to using GitHub sources, but I finally got my own Jarvis! I want to quickly learn how to use shaders and Holomat. But, I will patiently follow your rhythm.

Enchic Gong

I just saw your smart mirror post and about got on one knee to propose lmao! I hope to god you do that tutorial! Love it!

Brian Gouthro

When will the new videos about tools and holomat integration drop?

Hi

Thank you for jumping in and offering help. I really appreciate it. I found out the problem. There was a dependency conflict between the mkl package and the torch package. I had to downgrade mk1 package to work with the compatible torch version.

TheK-B1000

Not sure what you mean? Maybe add some print debugs. are you getting a response from the LLM? Is it just not doing the TTS?

Kevin Coda

Yes, I'm interested in it too

Kevin Müller

I got the code to work for the most part but when I ask Jarvis anything it never responds unless I hard code it in.

TheK-B1000

Yes, I would also be interested in integrating Jarvis into my home assistant.

Sodi

Awesome! Thanks for making this, first off, but thanks for making it so plug and play!

Jon Paul

Great! When will it be released

Hi

Hello. I just bought the upgrade to your site. Its pretty sweet. Am i able to implement your jarvis code to home assistant some how?

Dutchfather

Hey, awesome video i was just wondering its a bit unrealed but when you do your scans of the holomat not the depth mat ones but the ones where it just draws a 2d shape of the object how do you do that i assue its just using a gray image and contours but im really intrested. Thank you

James Mercuri

Hey I've got more videos coming out soon to integrate them. Jarvis has a tools.py file on my GitHub if you make custom Python tools you can have Jarvis call them to interact with the mat

Kevin Coda

I've got a video on that coming out soon

Kevin Coda

Maybe, try typing "pip install realtimestt" in your terminal

Kevin Coda

I tried, but Jarvis can't understand a word.

Matteo

Hello! This is a great video. When I run the code to check at 15:54 in the video I get an error for the first line from RealtimeSTT import AudioToTextRecorder it says Module Not Found. Did I miss the step on how to install this module? Any help is appreciated.

Eric Bates

I appreciate the list I am excited to try this out

Michael Russell

Thank you

Michael Russell

Just a usb camera, any hdmi projector and a computer (can use a pi 5, I used a pi 5 and it works)

Hi

I love this. I have been waiting for something like this to come out. Is there any way to get a parts list for the holomat and jarvis ?

Michael Russell

How do I combine this with the holomat code? So jarvis works with the holomat, also how do I implement the apps that you have on tiktok, controlling 3d printers, lights etc with my holomat?

Hi

Yes, I am working on that right now actually. I am hoping to get that code out soon. It will probably be mixed into the next holomat tutorial when I release the sportify app

Kevin Coda

Hey so the LLM should know Italian, the only thing you might have to change is the parameter for language for STT. where I have "en" you should put "it"

Kevin Coda

How can I change the voice so no it’s matches Jarvis?

Yousef Khavari

how can i change the language to Italian?

Matteo

Hey man, I love all the amazing things you have created so far. Truly genius. I was wondering if you have the QR code record player tutorial coming out soo? I cant wait to learn how to make my own.

Aiden Drybrae

https://platform.openai.com/docs/quickstart?context=python If you follow this guide it should walk you through it

Kevin Coda

Hey! Just go to the command line or terminal and type pip install openai

Kevin Coda

how do i install the openai library?

Matteo

Thanks for following along! I'm working really hard to make this easy and useful for everyone to use!

Kevin Coda

Thanks! Yes! You can replace the earlier Jarvis script with this with no impact to the HoloMat code!

Kevin Coda

Great update! Is this something I can replace the earlier Jarvis build with and will that impact Hand Tracking and Holomat scripts?

Jon Paul

Awesome! Been following along and have loved it. Can’t wait to see how you refined it

Brian Gouthro


More Creators