Vibe Coding Robot Hands w/ Cursor (Inspire RH56DFQ-2L/R)

What is going on everybody? Welcome to a very exciting, at least to me, video on uh we're going to be programming these Inspire robot hands. Uh specifically, these are the RH 56 DFQ-2 LNR for left and right. So, I do have two of them. One of them I've already kind of hooked up. This one I just wanted to use to show. Um and uh I'm going to be honest, I don't know how to program these. And in the world of humanoid and just kind of more advanced robotics, there's really like there's such little information beyond hype demos. And I'm super interested, very curious, but it's very difficult because all these things are ve very expensive. And uh you know, like you might have seen like the unit G1s, they're $16,000. They are not $16,000. I would show the picture. It used to be on their website. It was like it would show the G1. and it'll say $16,000 and then it would say like contact for the real price. And that just made me laugh so hard. Um I bet I could find it on a wayback machine. I'll put it up if I can find it cuz that was just pinnacle funny translation. Probably wasn't intended to sound that way, but that's the reality. You might be able to get it for $16,000, but you've probably seen videos of them doing like back flips, for example. Um can it reliably do a backflip? Potentially. Can it do that once a day every day and not break something? Probably not. And there's just so many questions. Same thing with like uh even like Boston Dynamics, right? Like you see all these awesome demos, but every time I've seen these things like even like that spot robot dog in the wild really just asking it to do a really slight slope for example, it just fall over, right? And be on its back and it's like, okay, well so where are robots, right? So I've always wanted to get into this. I almost got into it with um with Boston Dynamics and then they just like ghosted me. Don't know what happened there. Whatever. Same thing with uh Unitry and then another company called Deep Robotics. Um somehow I've just missed the boat every single time. Finally, I have these hands and hopefully I have a G1 on the way. U so some super exciting things to come potentially. But first, we have to figure out how the heck do we program this stuff? Like I don't actually know. So uh that's what we're going to be doing today. We're going to be using cursor the uh 3 claude 3. 7 sonnet the agent all that. We're going to see can we operate one of these hands because to be honest I don't know what I'm doing. These hands are to me from a uh startup called Lucky Robots. They are involved in uh simulation and training of robots. So one of the ones that they're going to be trying to work with is the uh Unitry G1. So uh let's get started. So the first thing is the uh manual. So these do come with a manual believe it or not. And um this is what you get. And uh it has all like the specification but the conversion of the specification to like Python code. No, you know, like I can't look at this and be like, okay, cool. And like type up some Python code that I can't do that. So there's definitely some instruction here. it has what I would describe as documentation, but I can't do anything with that, right? Um, and this is the power of these language models. Um, I know what's possible and I can sort of help guide it, but I can't I wouldn't I could probably figure this out eventually, but it would probably take me more than a week of like kind of trying at it a few hours a day or something like that uh to figure it out. But with agents, I think we can solve this very fast. Um and and like uh I think it was a few months back I had like a break in my underground fence for my dogs and I didn't know where the break was but I had an underground fence and I had heard that people would do things like you could connect like a lawn mower to make a little signal frequency and there was other tools that you could buy that you could hook up but I was like surely can you just use like a Raspberry Pi right and I used Claude at the time 3 point uh 3. 5 uh sonnet and converted the Raspberry Pi to hook up to that underground wire and it would send these like series of tones like essentially some music and then I could walk around. It just converted the underground dog fence into a giant radio antenna. And then I just walked along until I until essentially the music stopped. And now I could I found the br the break in the fence. And that was like mind-blowing to me because again I know there's a lot of hype and a lot of excitement around uh language models and using them to like help you code and all that. But at the same time uh so in some ways it's overhyped and some in a lot of ways I think it's very underhyped. So anyways that's my piece. So let's go ahead and uh get started with these inspire hands. So um so this is what we have. Uh unfortunately I don't think cursor can actually like read PDFs. But I'm just going to literally control arr c. So copy and we're going

to come over here and I'm just going to tell cursor the agent. Um I have a two uh inspire robot hands the RH56 series. I have one already uh connected that I would like to begin interacting with. Um here's the copy and paste of the entire PDF manual. Paste. Can you help me to write some Python code to interact with these robot hands? Thanks. Okay. And we're off. Uh we are using uh 37 sonnet. Uh auto select is off. I find for some reason um is it really going to make a directory in where we are? Okay, that's fine. Um, Python module that will handle communication with the robot hand. Yeah. Okay, cool. So, uh, in the past what I have found is if auto select is on, this could be wrong. Somebody from cursor maybe if you're watching or somebody who's like a power user of cursor, um, I I've been like using so many different uh, agents basically. So, I did like open hands and then now I've been using cursor. Sometimes I use claude code. Um, I' I've been using like everything. So maybe I'm probably missing something, but I swear if this is on auto select, it doesn't like run commands in this chat. So like the information of the output and all that doesn't happen in the chat. It happens like over here in a different window. And that's not very useful uh because it doesn't communicate back and forth. So uh if you're having that problem, if you're happening to use cursor, I know no there's probably like nobody probably following along with these inspire hands, so it's probably not an issue right now. But um that's something I've noticed. I don't really know uh if that's like actually what the problem is, but I do notice that if I turn off auto select uh it will like run the commands in this window. Okay, so we have um Whoops. A whole bunch of stuff here. Again, I I don't know. Clenching, grasping step is good. Uh okay. I don't know. Accept all. I love it. I think you've done a great job here. Um, cool cool cool. Advanced example action sequences feature. I don't know what it's even talking about. Like I haven't seen action. Oh, okay. I don't know what action sequence is. Okay. Can store 40 sets of actions. I have no idea uh what that is, what that means. We we should ask about that uh later. Um okay. RH56 hand. Uh okay. Make a read me. That's fine. Pi serial. Yep. Great. Great. Yep. Looks awesome. Love it. Run it. Get it done. Okay. It's going to try to find the port. Um I'm going to hit cancel here. I should already have all this stuff, but it should be on Python 310. Uh, will you please use Python 310 instead of Python 3? I need to fix this on my system, but um I have Python 3 corresponds to Python 3. 8. Python 310 is obviously 310, but then I think I've got some like it's so goofy because I think like pip for example, pip is uh yeah, 310, but Python is going to be 3. 8 8 and then I think Python 3 even is probably 3. 8. Yeah, I need to fix that. So instead, if I want to use Python 310, well, obviously it's gonna be 310. Okay, so uh get out of my face. Why is it still I don't even know what are you still generating? I just literally want to do this but with Python. I have no idea. What is it? Why was that? Why is that a long generation? What is it stuck on here? Like I don't think this should be like a longunning question and I believe I still have internet. So like I just don't really understand like why is this still uh generating when running Python scripts? Like I don't know if it's like thinks it needs to convert to Python 310. Like I just I have no idea. Like what is it doing? Okay, run that

command. No, I don't care about this. Thank you. Get out of my face. Run the command. Okay. Um, so it definitely found the adapter that's plugged into the computer. Um, okay. Python m inspire hands example and then it gives like the port. Okay. It definitely making start trying to do like a whole bunch of stuff. It's like uh I don't think there's any communication happening here. And what is happening down here? I'm going to stop that. Close that. Um, it looks like it's not working as expected. The hand was definitely not doing anything at all. Uh, okay. That's a whole bunch of stuff there. I don't know. This seems to be running unbelievably slow. Like I feel like normally things are like much faster but inspire hand. py alone is 770 lines of code though. Uh example so much code. I just I can't believe we're at the point now with language models where we just write so much code. It's just so crazy to me. I accept everything. Very good. Let's make it okay. Make it executable. Sure. Looks good. Let's go ahead and run it. Okay, we're having trouble communicating with the handles. Run a quick check on the serial port itself. Make sure that's working correctly. Uh, TTY USB0. Cool. Okay, let's try to check if we can see data coming from the hand using a simple serial monitor. Serial. Uh, okay. Accept. Love it. It's beautiful. Make it ex I don't know why it needs to do this every time, but okay. I'll allow it. Okay. So, this is probably the baud rate. And then D. I don't know what D is. uh 5 seconds delay maybe. I'm not sure. Sending the test p. Let's try a different baud rate. 192. Uh I'll go ahead and let that run. Baud rate. The default bald rate is 115200. So it should have the let's try a different approach. Check if we can resolve the communication issue. Let's create a RS485 hardware checker script. Okay, whatever. Um I mean everything is connected. It's definitely powered on. I don't there's no indication on the hand itself that it's powered on or the RS485 connector thing. Uh the power brick thing has a light on it. So, I know that at least it has power, but that's about it. Uh, RS45 checker. Yep. Run that. That's fine. Wait, what? Do not run this while connected to your stop. Hold on. I mean, it's literally connected right now. Uh, what did it do? Uh, okay. So, the hand is connected. Requires physically connecting. Yeah. Why would you do that? Dang. You gotta definitely run this because No, it's definitely connected to the Hangtight. Uh, you need to calm down. Where did I read that? Yeah, here. Uh, how do I copy this? Let's see. Paste. Okay, I see part of the script you wrote says this is all brand new hardware. I'm confident it works correctly. Let's not run this test if having the hand connected is a problem. The manual does

specify default uh baud rate as 115 uh 200. So let's keep that as well. Um any other ideas? Yeah. I don't want to um try using the modbus protocol. So we might be able we can try that. Uh let's actually try the Modbus protocol uh attempt that you wrote above. Uh in fact is that this one right here? Yeah. So we can accept that. Uh, did it have modbus test the modbus test. py? I should already have that. Yeah, sure. Oh, is it? No, it is in 310. Looks like there has changed in newer versions. Let's upgrade our script to use the current API. I accept all up to this point at least. Um, I likely accept. Very good. Looks beautiful to me. Trying to read error. Successfully read error. Okay. I mean, I see error and I'm concerned, but um, so it seems to think Modbus protocol worked. The RS485 protocol did not work. Um, Modbus protocol adopts the master slave request response communication mode. The protocol frame contains function codes blah blah blah blah. Uh, educate me like I'm five. The difference of RS485 and Modbus. What does Modbus mean? Um, okay. I'm gonna keep that, but I do want to ask this, but I also the hand has done nothing yet. So, I really want to see the hand move, but if it does work, um, I guess that's all it's actually trying to do. It's just trying to read the errors from the hand. So like in the protocol um I did see that like it will report any errors and like issues that it's having. So I guess um I guess because it read no errors at least it like got information back. So I think that's the whole point. I'm going to accept. I have to see the hand move and then we can ask about Modbus and RS45. Like I don't know any of this stuff. Like again, like I just I don't know how this stuff works, but uh if we can learn along the way, that'd be great. So, let's hit accept. Oh, make it executable. Run the hand. I want to open the hand. Move some fingers. Let's go. So much pain. Uh so, at least up to this point, uh getting communication at least via Modbus. Um, maybe I would have been able to figure this out. I do think this would take me a long time. No module. Let's fix the import. Um, I think this would take me quite some time. Um, and the problem is like a lot of stuff you can like look up online because someone's already been through it. But with like stuff like this, I could find like no information on, you know, interacting with these hands in particular. Uh, it's just like you're you're on your own. You need to know what you're doing and I don't know what I'm doing. Um, but potentially hopefully with a cursor we can get through that. Uh, except run the command. Oh, I heard it. It didn't do anything, but I heard

it. Um, oh, now it's doing something. Oh my goodness. Okay, it's moving. Is it counting? I'm not sure what it's doing. The hand is definitely moving. Uh, let me give you guys a view of the hand now. That's probably way more interesting than my confused face. Let's see here. Okay. Hopefully that is good for you guys. Maybe I'll do that. Okay. So, as it's working, the hand is responding to commands. The hand is making different grips and we're able to read data from the sensors successfully. Okay, let's create one more example that allows for interactive control of the hand. Okay, I'll allow it. I accept. except. Okay, so it's going to update the read me. Look at all these files that we have now. Oh my god. except honor what's happening on a serial port which could be helpful for debugging. Sure. Also Oh, okay. Okay, so just ran help. But I want interactive like I want like a CLI that we can just do things in series. Accept. Okay. Very good. Okay, your own Python code using the inspire hand modus class. Very good. Um, let's go ahead and run this. RS485 and Modbus are different things. RS485 is a physical communication standard that defines the electrical characteristics of the connection. It's just the hardware layer, the wires, voltage levels, and how signals are transmitted. Modbus is a communication protocol, a standard set of rules that runs on top of various physical layers, including RS485. Okay. Is like having telephone wires connecting various devices. Well, Modbus is the language spoken over the wires. Okay. Well, I'm not really sure why the base 485 stuff like why was that ever an attempt? Like why were we so what were we attempting to do prior to using Modbus then? We were attempting to communication communicate with the hand using a proprietary binary protocol directly over RS485. Huh. So we were just trying to make our own little packets I guess. Okay. All right. That sounds good. Um okay. So, is there a script where I can interact with uh with the hands yeah open the hand with O. You can close with force. Make a pinch. Okay. So, let's do it. Okay. Let's do it. Very good. Run the command. Okay. Wow. Uh, okay. So, the hand is mostly open already. Uh, let's go with open. So, O then C, three-finger grip. Interesting. And then we can pass like force as well. We can also set movement speeds. And then we have finger indices. Okay, this is interesting. Um, so we have speed force. So can we what I would want to know is can we set those

individual per finger or not? Um, but first let's go ahead and just let's do it. Open opening hand. Uh, okay. Close. Okay. I'm going to say nothing is happening here. H. Things had been working. No, not a power, not a connection. Really, none of this. I mean, nothing's changed, so I don't really know why it would think any of those things. Huh? Well, I can't run the commands, can I? Like, nothing's actually running, I don't think. Uh, okay. So, you run this. I'll have him. I'll have Claude. Okay. So, we'll monitor it. No, it should be here. You stupid weird. Okay, whatever. Uh, wait, it canceled. Okay, we'll open that and then run that. Okay, so I'll try to run open close. Um, okay. So then I'll just copy this. Uh, we'll cancel. I saw this in the monitor. But the hand didn't move. What? You do it. You do the commands and testing, please. I don't know what I'm doing, sir. Are you kidding me? What? Was there a reset factory? So maybe we could do that. But I don't really know if I want to reset to factory defaults. I don't know. We could try to clear. I don't know. How about not the interactive script for now? Let's try something that doesn't require my input. So, you can do some testing quicker, please. Yeah, I don't want to like have to keep typing in all this stuff. I'd rather it just like run through. Let's use the Modbus. Okay. with the hand physically here.

It's not going to be the B rate. Like I know it's not that. It's definitely connected. Let me double check the power strip, but I don't really see how anything like that could have changed. No, it's definitely still got power. So, what would have changed in the time between now and then? That doesn't make any sense. Why would the Okay, it's powered on. Try different B rates. No. Try different IDs. No. Huh? Why would the What? Like, what could have possibly changed? So crazy. There we go. Okay, we accept and we'll run that. Okay. So now the thumb is moving. Okay. I guess it's just going to wait on me. Okay. So it worked again. But I don't understand. Okay. So that did work. Very nice. What was the problem? No. I No. I mean what you just executed worked physically as well. The hand did move. So what was the problem which gave the motor okay any resistance which is odd. Okay setting speed first then force clearing errors and then okay waiting time five. Okay. After commands get the hand off. Okay. So, maybe it just needed to complete the movement. Okay. Okay. So, now with what we've learned, can you make an interactive script where I can issue commands over CLI? Meant to say over CLI, but it'll probably figure it. We've already created. Okay. Sure. Okay, we accept. Very good. Make it executable. Let's get it done. Okay, it is closed. We're going to try to open. Ah, it won't do it again. This is so frustrating. Okay, the difference right here is how they handle the parallel. Let's check the actual Modbus communication in the interactive script. See if there's a difference. If anything, maybe I would just ask it to like rewrite the uh interactive script or something because like I don't understand like what's different except I think we found the issue writing to different registers than expected. Let's check the full file. There's no angle command key, but in the set angles, we're trying to find and

access it and then trying to use it. Okay. Okay. Sure. I love it. Please work. Hey, very good. Okay. Close. Very good. Okay. Open. Close. So, we got pinch. Three finger grip. Let's try open. Let's do a pinch. How's that a pinch? I'm not sure I'd call that a pinch, but okay. Uh, open or o. Okay. Uh, speed force status clear. Reset. Three finger grip. So, how about T? What is T? Three finger grip. That's interesting. I'm not really sure what it thinks it's doing there. Um, okay. Let's hit cancel. Okay, this looks like uh Okay, good start. Could you make um a point uh command accept. What is point? Oh, it's literally just point. Okay, I accept that as well. Run the script. Okay, if I Oops. If I open Uhoh, not this again. Okay, thank goodness. I was very scared. Uh, point. Oh, no. Um, yeah, I don't think that's what we wanted. No, that's sir. Oh, man. with the middle finger. And I find this shouldn't uh shouldn't the point be with the uh index finger. Um I wonder if it thinks the index finger is what it's pointing with. Like maybe it has these wrong. I can't believe that. That's an embarrassing mistake. Okay. Should be using the index finger. Uh let's see if it fixes that. Let me fix that immediately. Dude, this is truly the robots rising up. Uh this is the first sign of this happening. Oh man, that's funny. A mismatch. Um, so this is the pinky, ring, middle, index, thumb, thumb. Uh, let's try directly setting. What do you mean? Uh let's run an open close. Um so how would I Okay, so the finger the number finger. Okay, so finger number and then the angle. So if I said finger zero 500. Huh? Right. Shouldn't that finger number and then the angle? So okay. So finger one 250. Why does it just say failed? I don't really know. They are already closed. It is not uh no finger extended or moved.

I don't understand why we keep finding ourselves in this situation where it like doesn't work. Okay, the hand is already closed. Oh my god. How many more times is this robot hand going to flip me off? Yeah. Okay. When you thought you were moving the index finger, that was once again the middle finger. FYI I accept. Okay. Closed all the fingers. Now the problem is like no finger has extended at the moment but it could be because the thumb is like literally in the way but it doesn't seem like it wants to extend either. Um okay all the finger Okay, I guess I'll hit enter. Closing finger zero. Okay, position one. No finger extended at position zero. Position one was the pinky. Okay. Position two. Position two was the ring finger. It's going to flip us off. It is what it is. Here we go. Position three was the middle finger. Literally flipping us all off. Literally flipping off like probably, you know, thousands of you right now. Interesting. Okay, position four. Uh, position four is the index finger, which is interesting. So, nothing happened at position zero, which I almost wonder if that's like the thumb. Okay. And then position five moved out the thumb. I wonder if position zero is also thumb related potentially. Oh man, that was so funny. I accept. I wonder if that's why Yeah, maybe that's why the pinch was so weird. I don't know. Except although if you're going to pinch, you need your thumb. So I don't It's going to have to figure those out as well. Three-finger grip method. Yeah, it's like all these things. Yeah, we're gonna have to work on a lot. Okay, run it. Okay, first of all, I want to try open and close. Let's make sure those work again. Open, close, point. Yay, we did it. We've done it. We have done it. Okay. Um or actually, it's just open. Um Okay. What was the uh wasn't there like pinch? What was the thing? Okay. P for pinch grip. I mean that's close. I think you want these to be touching each other and it just I think it needs to learn the positions to do that uh better. You could definitely get to the point where you pinch, but this is not it. Um what

was the other thing? There was like a three-finger grip or something. Yeah. T. And I think that's the same thing. Like I think it's trying to do this, but it just it doesn't have it doesn't understand the rotation of the thumb. And I think that's actually the problem, right? Like it understood. Oh man. I wonder if we could somehow get that fixed. I think because that's the problem. It doesn't realize that position like that zero position is the rotation of the thumb. So, um, that would be the next thing I would try to fix. Uh, but I think I'm going to stop it here because we're like almost at an hour of this. It was crazy. We actually had the hand working within like 25 minutes or something like that and then it just like stopped working. Uh, so that was pretty brutal. Um, and then now we're back at about an hour with the interactive script. But that's pretty cool. I mean, this would have taken me much longer than an hour. I think what I'm going to do is I'm going to tinker a little more. uh with everything up to this point. Um and then I I'll probably package it up just in case anybody finds this video and like wants to work with this hand and have like the Python library and stuff like that, I'll package it up at some point. I'll put it up on GitHub uh so everybody can use it. Um if you have questions, comments, concerns, whatever, you know the deal. Feel free to leave them below. Uh this was pretty cool. I I'm super impressed with with both uh cursor um and the hand is super cool. I can't wait to actually like try doing things like grabbing things and like you know uh actually doing like manipulation. So like we can do pretty much we can operate each of at least the fingers. Uh I don't think we can actually address each individual thing. Maybe RS over like the RS 485 we could in individually uh interact with each thing but I believe over Modbus it's like the whole finger and it just kind of like functions like I think that's already a higher level API. Um but I think you know already at this point we can exert a certain amount of force with the fingers and then move them to a very exact and precise like location. Uh which is pretty cool. Uh and so like even just that gives us there's like a lot of opportunity there. But anyway, that's for probably a different video. I mean, I it's it' be really tough to test too much until we put it like actually on a robot. So, anyway, I think that's all for now. That's really cool. I'm pretty impressed with Cursor and Sonnet uh Claude 3. 7 Sonnet. Uh that's awesome. That saved me so much time. I don't know how long this would have taken or I probably would have just rage quit before getting to this point. So, anyways, uh that's all for now. Sorry if that's very painful for some of you guys that know how all this stuff kind of works, but uh yeah, that's really cool. I'm excited. Anyways, hopefully at least some of you found this interesting uh to watch. This is pretty cool. I'm super excited to be getting more into the world of uh robotics and very thankful to have uh technology to help me do that. So, anyways, that's all for now. I will see you guys in a later video.