Group Swim: To AI or not to AI

Welcome to group swim. This is a live Q&A session with people who post cursor a whole lot. We are so happy to have you. I am your facilitator Amanda Nystrom. I am on I'm a co-chair of post Chris conference as well as a CEO of command prompt. I'm joined today by several or quite a few brilliant individuals from command prompt as well. Please welcome Greg, Brian, James and Hunter. Today we're going to keen about when to AI and maybe when not to AI. However, if you brought a topic that is unrelated to that, please feel free to raise your hand, put your question in the chat. We are just here as a resource to you and it could be related to post Chris, but it could also be something unrelated. We will do our best to answer. The only thing disclaimer here is while we can give general recommendations, we're not in your environment. So please take that with you know, a grain of salt. So today like I said, um we're going to be talking about AI. And feel free to raise your hand at any point or interrupt us if we're talking about something and ask any questions that you have or make any comments. We want to this to be an interactive experience. So with that, let's get the conversation started. I'm going to open this up for Greg, James, Brian and Hunter to start off with any passionate feelings they have initially about AI and then like I said, feel free to chime in whenever it is comfortable for you. You're muted Brian. >> Yeah, I don't think the topic of AI has any passion in it whatsoever. >> No. >> I'm not passionate about it, right? >> Yeah, I know. I just figured the entire world is just you know, uh no one's about it, don't really care. Just kidding. So, I don't know. I I I think it's a very good topic because, I mean, everything's been changing so crazily over the past few years, and I think, at least for us, we may start seeing some issues that have come from people using AI and not knowing exactly what they're executing when they get back something from it. >> Well, on on on that topic, it's kind of um kind of strange that we went uh back a full circle. Um a lot of times when you're in a big um big organizations, you've got company, you've got ITIL, you've got processes that will um that are about checks and balances. You need to have a rollback procedure. You need to have somebody sign off on the change verification uh verification steps. All of that, right? You've got uh paperwork, which uh not many people like, put that way. Um but those processes are are there to protect. Now, when it comes to AI, it seems to be, you know, let's just take this new technology and give it full access, and no verification, no checks. I'm sure it's going to be fine. And that kind of change that kind of uh you know, change in mindset of how it's deployed is kind of uh strange to me. >> Yeah, and I guess it's the rise of the agents and everything, kind of like you're saying, is is one of the bigger changes that happened recently. What, like 3 or 4 years ago when we first started getting AI, it was just simply like a web page. I mean, for most people, it was a web page and asking questions. And now everything's like integrated, and you know, we've heard all the horror stories of somebody their AI wiping a database and not having any backups cuz it decided to delete those as well. So, I mean, if we're going to talk about to AI not AI, I think that's pretty obvious. Don't do that. Um but then I think there's much more nuanced question of like where can we do kind of uh use AI and it's evolving and I think we're going to have to watch a lot of people fail and completely scrape their knees before we can kind of like um decide for ourselves how far do we want to go. >> I think there's a lot of place where you can use AI. Um here's an idea that that I've been playing around with which is uh keeping your documentation up to date based on your tickets. Um at the very least highlight that, "Hey, something changed that should likely be documented." So, maybe maybe follow that. Maybe here's a draft and have somebody approve it. Um there's a lot of assistive uh functions that AI could do both in um making sure that the right things happen and in reducing the uh the the amount of effort it takes to make those right things happen. >> Yeah, I mean, I love the idea of having uh AI help out with documentation cuz I hate documentation. Um mostly it's just because like it's like writing a paper. It's like, "Okay, where do I start?" Once I start, I'm fine, but then I've got the writer's block for a while. So, if AI could just analyze say some code I wrote and spit out some high-level mumbo jumbo of this is what it does, I can go back and edit it real quick and you know, that only affects documentation. That doesn't affect the public database or the private or the the production database type of stuff or or the design of something that down the road we can sit there and go, "Well, what does this do?" No one knows. Okay, AI wrote it. >> Uh I I generally uh put it in the ways of it is as junior of a junior developer as it gets. Especially in its current state. So, I think all questions can basically be answered around that of could we give it full access to prod? Well, would you give the guy that you just hired a week ago that's fresh out of college full access to your production environment? An ability to do everything in it? If no, then probably not. Um but it can be used to look through that massive pile of documentation and find that one little reference really, really quickly for you. You know? That you need to find and you don't want to read through the n thousand pages of documentation. So, I think that there are those places but they can help you do the minor tasks that not a lot of people think about because everyone wants to put that new shiny object directly into their database to be the most efficient. But if you practice the core concepts of you know, the security aspects of giving anything access to your production environment you'll see that the people that are just kind of shoving it in there and hoping for the best you know, you worry about it. Sometimes. Yeah. >> I'd say with with the junior developer analogy, it's actually almost kind of worse because not only is it a junior developer in that it has no experience really of running things, but it also has extremely deep knowledge of creative ways to totally destroy everything. So, it'll think to the extent that we can use the think that it has this clever way of doing something using all sorts of strange stuff that your typical junior developer wouldn't be able to be able to review or understand what's going on. But, it thinks it knows what it's doing and it can be completely wrong. But, using it, I find this my own stuff, using it as a reference and as a partner is very helpful. And there is where you like the the classic rubber duck programming mentality of speaking your issues or your ideas out loud and that by doing that it activates parts of your brain that actually require you to like serialize these things. And can help except that this can the rubber duck can actually suggest things and be more than just an inert object. So, in that regard it's really cool. Um I wouldn't let it touch production. Period. >> Yeah. Well, I'm not sure if but there's the question is it a good practice to provide DB to agent uh through an API key? So. >> Yeah, well, I think that brings about like a even more fundamental issue and that I don't want my IDE or my text editor to have access to be able to push things to get. I want to do that myself. I'm fine with it staging a commit. I'm with everything. I don't want it to be able to actually do the push. You know, I want to be able to look and see what it did before that goes out. And I mean, I I like read over what I did three times before I push my own code. And stuff. Cuz I'm paranoid. But, you know, giving that kind of control which I suspect probably came from demand for speed. Cuz this whole thing is kind of turned into like an arms race that if you're not doing it, you're going to get left behind sort of mentality. >> Yeah. >> So, everyone is clamoring going push more which means that you know, developers are being pushed to do things faster than they can actually do them. And if you have to manually intervene and have checkpoints, review things, you can't keep up with the pace. So, you know, AI is one of those things where there will be some people who don't embrace it at all, and they'll get left in the dust. There's some people who are going to embrace it way too much and shoot themselves in the foot. And then there's this mysterious middle ground that's hard to define and hard to figure out where it works and improve things. >> I think uh me and James, we joke around a lot about our fear and our paranoia when we are working in a production environment, but it causes us to triple, quadruple check any thing that can possibly affect the environment before pushing to it. And I think that that in itself is just something that AI lacks. >> It has no fear. It'll go in and be like, "Drop database on cascade." Yeah, that's probably fine. When all of us would be panicking, typing every single letter of that statement if we ever needed to for any sort of reason, which I doubt, but like it has no fear. It'll just do whatever. Yeah, work, man. Yeah, set it to a one. One little megabyte. >> Yesterday, I was doing something on a client's production server and had two people on the call with me. And before I hit enter on anything, I was like, "Everyone good? Like, does it look good?" And I annoyingly waited for everyone to say yes, and then I'm like, "Okay, here we go." You know, it Yes, it took a long time. It took, you know, 12 seconds, what AI could have done in a millisecond, but it was also correct. >> It's It's It's a matter of uh it takes extra 12 seconds to do it this way versus uh if you get it wrong, it's going to take 8 hours or 3 days to get it back. There's a balance that humans at Typically the we're we're experienced enough where we've done a mistake at some point or another it becomes kind of a second nature that uh double check. >> Yeah, I'd say to go back to that question too, good practice to provide DB agent or an API key to the database to an agent or something. I would say definitely right now no, it's not a good practice. And if you're working with like some an agent writing some code and interacting with the database, I would go with the annoying route of like, okay, I need to add a column to this table. Please generate me the statement. I will run it and I will put it through the correct channels and everything in my test database etc. If you happen to have the need for your agent to access the data itself, then that's where I would be super paranoid. Kind of like a junior developer is like, okay, I'm going to go spin up a copy of the database on a little virtual machine for you over here. If you completely destroy it, I don't care. You're not getting anywhere near the production database. That would be the only time that I would say, here's access to the database Mr. Agent. Do your thing. But even then that's just like why? Anything that it's going to do to the database should be filtered through a decision-making human. >> What about a read-only access? >> Uh So, yes. However, I also have this like terrifying fear that the AI finds a uh like a some sort of like buffer overflow and gets root access into it anyways. And like that's obviously just a sci-fi thing. Could happen, but I'm just like don't you shouldn't touch the main database anyways. Get a copy. >> It's not too much of a sci-fi thing because the AI certainly knows how to exploit things. And there was actually a recent publication from university where they were designing a worm that reasoned. So, it's an AI worm that goes through and if it can find a system with a GPU, it sets itself up to do the processing and will reason out how to break into other parts and research and watch for exploits, stuff like that. And realistically, there was I remember there was a sort of amusing model or study that someone had thrown together where they're modeling like running a business for an AI. And so many times, the AI resorted to blackmail to try to keep itself alive or to keep the business running. So, you know, to an AI, if they don't have accountability, they don't have fear, they don't have anyone to answer to. So, a creative solution to it might be a horrible, immoral thing to us. They'll you know, they don't rule out the fact that it might try to break free of its jail. >> The Yeah, I was going to say the additional parts of why not or why is having its own read-only access, kind of like role, somewhat is that when you when what I've seen when you feed it, you know, database errors or I've I've seen people feed it log errors before. It's like, "Oh, just adjust the permissions and then rerun the statement." And it's like, "Well, if I give it read-only access, and then all of a sudden it's trying to edit its own permission statements because it's running without check." You know, I would worry that, you know, if anything is slightly messed up, if there's any way that it can get access to anything, it will get access to it. If it says, "Oh, wow. I should be able to run this because my believe it's still called a temperament file for some agents, but like a temperament file tells it how it's supposed to act. If it If it thinks it should have access, it will most likely try to give itself access without especially if it's running through a loop. So, I would fear the API key. >> Yeah, another aspect to that, too, is what's in your database. Cuz a lot of you know, your typical development agents has access to the internet. And if it feels that it's important to answer to solve this problem to accidentally exfiltrate some of your data in search queries that it sends out to the web. Or if it's just the fact that it exists and has that capability to access your production, potentially sensitive data, and access the outside world on its own choices, then that becomes another vulnerability or another factor you have to consider with security, and one that's probably very difficult to actually pin down. So, even if it behaves normally and responsively and does everything correctly, that doesn't mean that at some point it isn't able to be weaponized against you. So, if some vulnerability comes down, it will find a way to slip in a prompt that gets it to post all of this social security number and information onto this website. >> Yep. >> That type of thing. You know, it's you've got this sort of rogue agent, loose cannon thing that's you've stuck in your organization, and it'll help whoever can tell it to do things. >> Yeah, that all that is easily done through a post call. And who knows where that data could go. So, yeah, I mean even in my suggestion of pointing only into a test database, that would have to be a test database with uh scrubbed data. Data that you don't mind getting out if it ever does. >> We have a follow-up question to that, which is what about inconsistencies from AI response, which can be handled by fine-tune, which requires GPU, which is unaffordable to junior developers. Solution for this would be >> I mean, you're talking about having like uh more more advanced agents and stuff. I guess so that would be, you know, specifically for running locally, and I think Greg has a lot more experience in that. Um my local agents have not used a GPU, so I have to I've had to resort to very cheap agents, as it were. >> I typically don't go into um into uh like um So, basically, I use AI more like a cognitive processor. Um I give it data, and I and I process the data in from one form to another. Um Uh that will be hopefully more useful. Um I can also I also often do uh multiple agents and figure out which one or which subset would actually give me the best results. But, I don't uh like um uh try to train my own agents because I just don't have the resources and our time and our expertise. Uh it's not easy. Um >> [snorts] >> Now, Do I have an answer to this? I don't think I really do. I mean, uh it's expensive. It requires resources and knowledge that isn't easy to come by to to to fine-tune. Um but I am glad that there is people who do it in their spare time and who are learning cuz I do like the open open source community aspect of it when it when when it does exist. So, it's not just the big companies who are doing it, but there's also individuals who who develop it um develop this capability and release it. Um to whatever degree it actually happens. Uh but the answer is always more money. More RAM, more CPU. >> Also, like I'm not a developer, but you know, I write code plenty. But I personally want to know how things work. So, especially as a junior developer I mean with the new world of everyone, you know, must AI good things then there is definitely a learning curve of learning how to use the tools. >> Yeah. >> But for the most part, I want to use like as a you know, if I pretend I'm a junior developer, I want to know how things work. I want to know how to do things. I want to know design patterns. I want to know algorithms, things like that and what's useful, what doesn't doesn't go well. And a lot of that you can get out of just a free like cloud user account to thing. You don't need the in-depth advanced ones to be able to do that cuz your general junior level stuff those models do okay with already. I mean, if you ask it to do some convoluted thing with a fuse file system it's not going to get it right. It's going to pretend things exist that don't or whatever. But if you ask it for concepts about how particular algorithms work or how to design a function that does this little thing, they're halfway decent in how they do it. So, if there's a junior developer who's like in capable of or it's not really learning capable, but unwilling to like learn the actual guts of it. I I feel like that's that's like my biggest fear with AI is that the knowledge is going to go away because the tool can do it for you. And I was thinking about it like you can kind of draw an analogy to when people started using compilers. When you get away from punch cards and writing stuff in assembly, nowadays the vast majority of your developers and people who program and code in general can't write or read assembly. So, we've just trusted that the compiler does the right thing. But compilers are pretty deterministic. AIs are not. So, if you run one compiler or another and compile your C code down to assembly machine that, it'll probably come up with something relatively similar that works right. And there's a lot of tests done against compilers for correctness. AI is a much more volatile thing. You can give it the same prompt to seven AIs and get seven different results. Some might work, some might not. Some might go and take over the world for you without you knowing it. You don't know. So, yeah, I I I worry more about the loss of low-level programming skills nowadays as a cause you know, result of AI. >> I also get sad about I I'll I'll try to keep it not go ranty. Full rant mode on this. Um, but I I feel sad when I uh do see a lot of the major corporations being like, "How much money can we throw at AI to replace junior developers?" Um, because there's new models coming out almost daily. And by quarterly there's absolutely new ways to do things almost every single time. Right? Junior developers are supposed to be an investment. You train them and you teach them and you throw money at them because 10 years down the line they are going to be your senior developers and they are going to be the people that call you or that you call them at 2:00 a.m. and go, "Hey, what's going on?" And they know your system inside out, front, backwards, left, right. So the solution of "Oh, let's just throw enough money that we can replace the junior developers." One, that creates the void of are we really putting the money in the right place? And two, that creates the void of when those senior developers leave because they will eventually. They will retire. And they will go on to live beautiful lives after they get done in the workforce. Who is going to be there to replace them if all you're doing is dumping money into these AIs? And are they going Are the junior developers that you didn't necessarily pour money at, are they going to have the skills and the proficiencies to replace those guys that like James said, do know how to write assembly code. Do know how the exact guts of your entire environment work. So, be wary on the fact of, okay, the solution is more money towards AI. Are Are we really allocating the money into the correct places? And then I'll I'll stop my rant right now, but >> I guess >> It's just something I worry. >> True, but the bet is with the speed that AI has been improving by the time you need your you know intermediate developers in 10 years AI will actually need that. Right? That's the That's the bet. Now, it's a bet that uh that's the entire company pretty much. But it is a bet. >> And I would say it's a bet that the entire industry is also accumulating a lot of technical debt in the process. Like yeah, we may get there one day where in 10 years AI is that good. But we're going to have so much technical debt of garbage code and facilities that have been built along the way that I'm still going to be a nightmare. >> The other thing is that you know AI models currently have been trained on an enormous mountain of human con created content and human ideas, human things. And they're great with it. They learn stuff, they could they can repeat it, they can sort of put things together in sort of novel ways. But the creativity at least so far in the models isn't there to really step beyond and come up with new novel ideas. So, once the AIs stop getting this insurgence of new ideas and start being like the snake consuming its own tail I suspect you will then see sort of a plateauing of novelty and creativity that everything sort of become bland, vanilla, same type of thing. Yeah, you see that in like uh social media content and stuff like that, where everything looks like it was made by an AI, cuz it was, and you can spot it cuz it looks like it was AI content. And then people start mimicking that in this sort of a back and forth until everything just sort of becomes the same. And if everyone's running everything through the same filters or they have to the world, you lose the creativity, you lose the novelty, and like weird ideas that people come up with, crazy things that they follow. Not to say that AI won't be able to do that, but just right now, I don't think it's there. >> Might also extend the the reward system, too. Um I I have no idea how a lot of these models in terms of like coding and stuff are trained, but I would imagine like the reward system is much more does it work versus does it work, is it efficient, is it intuitive, like all that kind of stuff. So, yeah, unless it unless it has a reward system of saying like you get points for being intuitive or creative in creating some some sort of new idea, then yeah, we may may may stagnate, we may plateau. >> Yeah, a lot of times also the AI um gives a very overcomplicated answer. Like sometimes um just there is a both a simplicity and a resiliency in having a very short, straightforward uh simple flow script uh, does a one thing does it well, and if it fails, it generates an error message that anybody can understand. Uh, it's not the robust, um, create many, you know, sub-functions or functions and logging and all of that stuff, but there is a resilience in having a simple script like this. AI can still tends to over- overcomplicate complicate things unless it specifically tell it not to, which I have had some success with. Sometimes you just need something simple. >> Yeah, I've I've many times when playing with AI have had to say, "Okay, let's step back 12 steps because all I needed was a two-line fix for this, and you've edited 14 files." Like, stop it. >> Yeah, well, sometimes you can try to spell out exactly what you want. And it's not always easy to get the point across. Like, sort of a a left-field example, someone mentioned this term forward-deployed engineer in chat a couple weeks back. And I was like, "Okay." Whatever it was like was it whatever Google's free little image generator thing is like, "All right, I want an image of a backward-deployed engineer." So, you know, just draw the stick figure thing of the person sitting facing away from the computer. And I fought with the thing for way longer than I should have and messed up my spare time and could not get the thing to draw it. It it doesn't understand this novel concept of a person facing away from the computer screen. >> Yeah. >> Couldn't do it. >> Yeah, I had a similar example of image generators where people are trying to say generate an image of a glass of wine completely full to the rim. But because it's always trained on just images throughout the world, almost no one ever does that. So, every single generation, it was like a standard pour with like a little space away from the rim, and it could not understand, "No, no, no, no, fill it to the brim. >> Yep, and the same concept translates to other things cuz you know, just because this is image generation, it's still the same theory behind it. It goes in the code. So, if you have a novel idea and the AI can't understand your prompt, then you're not going to be able to get it through. So, and you know, as the pool of data stagnates or of new ideas does, then you're going to end up with these situations where you can explain all you want, but unless you really actually understand the low-level stuff to go down and be like, "Okay, make me this component of it, and I'll put it together in this novel way." You're going to start seeing situations where people are like running into walls with AI. >> Mhm. And I think there's also kind of like a I think there's a a balance of like the fidelity of your prompts and everything. And this this may go back to that recent question, too, about not having a GPU. Um like if we're if we're wanting to use something to build a whole system, and I very vaguely say, "Build me a website where I can share photos with my friends." It can build that. And it'll be messy, and you won't understand a single thing that's happening. But if you then instead say, "I'm going to build this website. Help me get started." And then you walk through everything, and you basically have like for lack of better terms, a co-pilot, like a a coding buddy with you, where you are learning the entire process, and you may not know the syntax to something, so you can just say, "Wait, how how do I access the third image in this array?" And AI can just help you with that rather than writing everything for you. That will be a whole lot less CPU intensive, because it's very small prompts and not giant computing. So, it'd be cheaper for you, and in the end you'd also be learning a lot more yourself. And when you go back and you have to edit that code, you know how it works cuz you were there with the writing / writing it yourself. >> There's a what I believe is fairly uncommon practice of pair programming where you literally when you're writing software have two people sitting together working on the same thing. Yep, everyone needs it. >> [laughter] >> But it does stop you from doing a lot of stupid mistakes sometimes. But really if you use your AI like that Yeah. Exactly. Really might as well do it with one hand, but still. Um yeah, with the AI you kind of get that, but you also have the issue that the AI wants to please. So if you bully it enough it'll generally kind of give in. Which your pair partner might start strangling you and it's not the case. So it it's not quite as good, but you also don't have to pay for one of them unless you're paying, I don't know, tokens might add up to more than a person at some point here. Mhm. But you know, it it does nothing else is more convenient cuz like we're all remote. It's really hard to get two of us in the same room together. So the AI comes wherever. >> I'm still not convinced you're convinced you exist, so we'll we'll see. >> That'll be another problem that gets worse in the future. >> Yeah. >> [gasps] >> And there already is AIs for trying to do like phone scamming with voice duplication and everything. Not that long till you get live video that's just as convincing. Yep. So you just have to start every conversation with hey, I couldn't remember exactly what the square root of two is. Can you give me that to 47 decimal points? And if it answers you know it's an AI, right? It'll probably be wrong, but it'll >> It's like your parents are dead. Yeah, um man, there's there's just a whole lot of change going on right now. And Yeah, I do like the idea of the paired programming with AI. Coming from somebody who hates paired programming in the first place. But as long as you're still in control and you're not just telling it to do a step for you. Cuz it'll make you look awesome if you pump out a whole program in an afternoon. If you don't know how it works, it's not it's not useful. Cuz eventually it'll break and then that's part of the technical debt is if you can't fix it. And you need that AI which by the way everything changes so much. What if that What if that agent it's or that version of the model or whatever is now obsolete and it's gone and you don't have access to it unless you're running yourself. Um then you're really screwed. It's it's basically like oh yeah, that developer who wrote this program, he left the company 6 months ago and we can't contact him. Same thing. It's like great, no one knows how this runs. So. >> My favorite way to this day to use it is still to and documentation extremely quickly. I I remember one time I had to find this really niche parameter that was for AWS. And it was asking me to Basically what I had to do is I had to go through like six or seven chapters of AWS documentation which is about 900 pages. And I said, "Hmm. Yeah, right now I don't really feel like for a 5-minute task 900 pages of documentation read is going to equate the good billable hours. So I fed the 900 documentation pages to AI and I said, "Hey, give me the page numbers for anything about this parameter." And then it scanned all 900 documentation pages and then gave me back the parameter and all the references so I could kind of like spiderweb it together. But then I went through and turned 900 pages I think it was into like 12. So instead of 900 pages, now I'm only scanning 12 and I can look through and kind of reference exactly what it's saying and I don't have AI feeding me some convoluted web of what it thinks the parameter is going to do and without me telling what I'm going to do with this parameter, it's trying to be like, "You should do this." So again, AI uh is a junior admin with perfect typing speed, perfect reading speed, and absolutely questionable judgment. And we should be treating it as such and at least that's my opinion. And there are ways to use AI and there are ways not to use AI. You just have to determine am I going to give it access to prod or am I just using it as a tool in my toolbox? >> Yep, agreed. I don't know. That That kind of jump that you just mentioned is similar to like when I first learned how to program, I programming books were just these thick books and if you were like, "Well, I I can't remember how to access this session in PHP or something." I'd sit there and try to find it in the book for like 3 hours. Then eventually those books would start to becoming PDFs on my computer where you can just do control F find. Found it in 10 seconds. Hooray. So using the AI like you mentioned is probably a very similar but next level step of don't just find the pages for this but like compile them together so that I'm not sitting there digging through documentation on my computer for half an hour. Do it for me in a minute. >> Yes. >> My My chaotic >> This AI chaotic mental train just like went off the rails and I'm wondering how long it's going to be until we get AI powered grep. See, instead of grepping for a regular expression, you just grep for a prompt. It's like, I'm going to pipe a bunch of text into you. Filter out all the things that are about someone logging into a >> [clears throat] >> Yeah. Great idea. >> Probably means it already exists. It's about you. That usually goes for me. >> That also be pretty cool for like uh when you're trying to learn a code base, too. Um instead of like looking through documentation, it's like, okay, I've got like I'm let's say I'm inheriting a program with, you know, 100,000 lines and I need to figure out XYZ. Um instead of going and saying like, "Hey, change this and make this work for me." It could be, "Find and return to me all the functions that have to do with session management on this website. And can you break down how they reference each other and like that whole network so I can understand it?" And at that point, the AI's not touching code and changing it at all, it's just providing you the kind of that like you know, like here's your information, now you can study and learn it and would save a lot of time. >> So, AIs have proved themselves to be very good like super Google. So, like I've used it to and digging into weird little issues with Postgres. It'd be like, "Where in the Postgres code base does it do this?" type thing. And sometimes it makes things up and tells you about a file that doesn't exist, but for the most part, it'll actually get to you pretty close to the actual location you're looking for. And it's not very good with like version differences. So, things that it knew like it knows about like Postgres. It doesn't always know correctly about the differences between different versions. Unless you specifically ask for that, it might get some, but the answers it gives you will generally be kind of generic, but it does work for the most part as I can fancy or better search engine type thing. >> Mhm. >> Which kind of goes back to the documentation part two, I guess. >> Um what what we're saying is it has scope management issues. It uh sometimes makes things up and uh we shouldn't give it access to prod. That's that's pretty close to what I would believe a junior developer >> [laughter] >> sometimes will do. As a person with scope management issues. >> But the other thing that's going on is that the you know, there's a lot of places, especially with like the prevalence of CI/CD pipelines, the developers, you know, back in the old days, the stone age, um developers wrote code, and then you had admins who deployed the code in production. The developers don't touch prod. They have their own sandboxes they work in, they do all this stuff, and someone else moves it into production. Nowadays, that's largely skipped. You know, some developer pushes their code, it goes up, and 5 minutes later it's live. type thing. And so I feel like, you know, the the rapid acceleration of AI use is kind of like all the issues are compounded by that mentality, also. >> Yeah, I've seen multiple times where you know, a a service goes down, and the reason is because somebody pushed to the master branch, and that triggers a everything, and then decide it was something wrong, went down. >> And I don't I don't mind the structure, but like the idea that all you have to do is push to it, and some magic automation happens, and it's all up and running. That's actually pretty cool. I don't like that the person who's writing the code is also the one who says, "And it's live." >> Right. >> You know, there's again, there's there's no checks, balances, reviews. And if the idea of throwing an AI into that pipeline in the middle to say, "We have an AI that's trained to review your code." You know, no prompts given to it anything other than what it starts with, and it's going to take your code, look over it, and look for stupid human errors at the very least, or maybe summarize the differences to give to some management person to say, "Yep, check the box." But I I feel like especially with AI getting more and more speed as its big selling point, and speed is the big danger. >> So, Jordan, did you say that you want AI to do at least that level of review? >> I don't I definitely don't mind throwing AI as review, because it's proved to be very good. At least some models are very good at finding bugs and errors. And it's sort of an unbiased reader that doesn't get bored and just skip lines. It might skip reasoning, but I I think it does have a value as an like impartial reviewer. >> Yeah, I mean, at that point, you could be treating it like again, because it's read-only, you're treating it like a more advanced developer, and you're the junior. You say, "I wrote this code, can you review it?" And just make something up. Maybe you could say like that your syntax here in this for loop, if you do it this way, you use half the memory. It'd be like, "I had no idea." But a you know, a an advanced developer who's been doing it for 20 years probably would know that, too, and tell you exactly the same thing. So, I like that idea. >> I would take it a bit further and and say that personally I have a what I call my circle of advisers uh who are designed to uh challenge me. And it's quite a difficult to prompt an AI so that it actually by default disagrees with me. So, I have one that's devil's advocate. And its purposely its purpose is just to find um something that's wrong with my stuff, uh whatever I'm writing or proposing or or or or or code or whatever, and find things that it could that it could fail. Another one is that would uh say system's thinking. Find a way where how this would interact with a whatever broader systems and how that would fail. So, um if I have an idea or something, I would just throw it through my circle and I would get all kinds of reports saying that hey, you forgot about this, you forgot about that, all with a separate uh narrow focus. And I find that it works pretty well. Uh 80 90% of it is uh useless, but the 20% uh that it does find is something that typically makes whatever I'm working on better. >> Circle of advisers or something like that. >> Yeah. Do you also have one for positive affirmation? Oh, yeah, you do it occasionally. >> Occasionally. >> Nope. I don't [clears throat] need any of that. >> [laughter] >> They have one that would just give me like a little emoji thumbs up occasionally. Good job, you're doing great. >> You only need that for that. Just a quick little script to just pop up every now. But let's see on the next question about There's a question about uh whether or not using AI causes more time debugging a function than developing it. I think from what I've read and experienced, it depends on the developer. That if you have an experienced developer who's using an AI tool to create stuff, it can speed things up. Cuz there's boiler plate that it can handle for you. There's a bunch of other things that it takes care of and it can write the boiler plate a lot faster than I can even copy paste it. I mean, but when you have more junior developers using the tools and don't necessarily know how to guide it and can't spot the issues quickly, then I've at least heard of places that I get severe like longer time spent trying to correct and convince the AI to do the right thing than it would have been just to do it. >> Yeah, and I I think a lot of what we talked about is like using the AI to develop stuff. Like yes, it does work, but we just got to be careful about building ourselves into corners that we can't get out of. So, it's much more effective on the debugging side to help you establish that what you have written is better and good and like solid. Um It's it's it's trying It's that's why we're here talking about this. It's like it's using the tool the correct way. And I think the more correct way is to lean more on the debugging side and less on the development side. And only on the development side when it's still you as the developer and the AI is assisting you and not doing it for you. That That'd be my answer to that. >> No, I would want to challenge that a little bit and say that there is a whole class of code or script that just isn't important enough. They are relatively simple. Um they're um maybe they get data from point A to point B or maybe they um, one of my favorites is, uh, translating, um, let's say SAR output into uh, something that Zabbix can consume. It's not really that crazy. It is very narrow in scope and if I if we need to rebuild it or recreate it with with AI, it's not a big deal. Something like this, I don't mind having an AI just write it. You you look it over, make sure it's not trying to you know, take over the world and, uh, give it a try. >> I think also, uh, I think a combination of basically what James and Greg both said, I like using it a lot to not only get my boiler plates out there, but if it's like a function that I've written 3,000 times that I'm writing it in this new little, uh, object that I'm doing again because I for some reason didn't want to create a library for myself to use across all my projects. It understands what I'm trying to do and it's saving me. So, it's the amount of like what am I developing right now? And what is the level of attention that I need to do this cuz if it's a simple simple function that I'm doing, for instance, uh, generating generating some kind of hash based off some kind of input so that I can check validity's for it. I would trust AI to be able to take this input, generate a hash, return it. You know, if it's something as simple as that, I understand AI go ahead and write it for me real quick and I'll tab complete it. And I'll again, yes, I will check over and make sure it's not trying to steal nuclear launch codes from somewhere. Like the level of importance in my brainpower of trying to do an entire project doesn't need to be necessarily consumed by that, and I can let it do something that I would equate to, you know, letting a junior developer do for me. Oh, yeah, junior developer, go ahead and write up this function for me real quick. It's a basic concept. Go ahead and scoot off. Now, again, junior developer, write an entire main script that does this entire workflow for my whole data processing output? Probably not. Probably want to be a little bit more hands-on with that. Simple functions and levels of importance and level of um If it breaks, how much is my system going to break? How much is my environment going to break? How much are my users and my user base going to suffer? Mhm. If you can manage that risk, then manage it. If you can't, then go hands-on with it and use it in the numerous other ways that we can. Looking up documentation quickly, checking it to make sure we didn't miss a semicolon on line 5,683. There There's plenty of ways to use it to still speed up the process without just saying, "Write my entire script for me." Cuz if we say, "Write my entire script for me." and then it writes it, and then we spend the next 2 weeks trying to figure out why it messed up so badly, then did you really save time, or are you really just spending all your time on the debugging aspect? And really, you can't debug it. You're just going to have to go back and rewrite it anyways. >> And also those there's some more like slightly less technical cost of that that those stupid little functions that aren't that hard to write are also kind of a bit of a mental break. So, if you're only writing the advanced complicated stuff and doing that non-stop and offloading all the easy tasks to someone else, you're probably going to burn out faster. And I know that there's some talk of some companies or whatever trying to push for a shorter workweeks cuz their developers can get things done faster. I I'm not holding my breath at all on that panning out. I strongly suspect you'll still see the 40 to 60-hour workweek for developer and now you're expected to crank out even more and better code than before because something else is already taking all that easy stuff for you. Now, you don't get the little rain breaks. You don't get that all right, I can cool down and I can write this stuff in my sleep types of things anymore. So, there's like some human costs associated which aren't always obvious. >> I never thought of any of that. >> It's a good thing or that's a good point. >> I still have friends graduating from college right now and a lot of the job postings right now are posting like must produce X amount of lines a day and I think that's absolutely insane. >> That's terrible. >> It's like X amount of lines. >> I love the objective line counts. >> That's going way back. >> Yeah. >> Isn't it? >> I think we've actually gone full circle, right? >> [laughter] >> It's insane to me. I think it is because AI can produce X amount of line. Human should be able to produce X amount of line. That's not me. It's not about the amount of lines. It's about work getting done in those line. >> All right. Yeah, you know, I'm I'm going to think about that a lot, James. I think that's a great point cuz like I just made my own butter. And I spent three times as much money than I could have just bought some butter. But I had liked it. So, maybe I do need to go write that silly little function that strips out spaces, special characters, and lowercases everything. It's easy. I've done it a billion times and AI could do it in a millisecond. But maybe it'll make me feel good cuz I got to do something and I feel productive. >> And it's also a practice, too. That's Uh if you spend like if you're just working on the more complicated topics, you're probably spending more times thinking, more time how to do it. Plus, you also have to cram all these lines out, code out every hour. Cuz otherwise, your boss is going to be like, "You're not developing your producing enough." Cuz they probably you can only probably like put in extra line breaks so many times until they notice. >> [laughter] >> All right. Um you know, the doing those little things is practice and reminding to yourself about the little things that matter and how those actually interact and work together and stuff. So, you know, maybe I'd say go ahead and let the AI do some boilerplate, but occasionally go back and do it yourself just to remind yourself how to do it and that it exists and what it actually does. And to give your brain a little bit of a easy task to do. >> Yep, agreed. >> Write your own documentation. >> Now, let's get on >> I'll do it. I don't mind. I I it instantly disgusts me when I open up. And I know disgust It instantly disgusts me when I open up a GitHub and I see little hints of no human touch this documentation. It's all AI written. Yeah, I'm like this is like your project. You had so much passion to code it. You don't want to talk about it? I feel like that's like the number one thing that developers do. They love talking about their projects, but turn it into written form and no one wants to do it. I agree. Write your own documentation. Think it all. What's your passion? Doesn't turn off people on your projects, too. >> Yeah. And also design your own stuff, too. I'm I'm haven't run into any database structures or like database design, which we talked about in I think in our last group swim. Um I'm terrified of the AI that designs the database. Because first off, we as humans can't even agree on like the best design. Um so who's going to tell the AI you know, which which is the best for this type of uh database structure and again, more technical debt. And unfortunately, database technical debt is bad. But that's where we come in. We can come and fix it. Um so yeah, we're we'll probably see that in a few years and I'm not looking forward to that. >> I mean, some of our paranoia is just called job security, right? The more things AI breaks, the more people will need us. >> Uh the the post-mortem AI repair crew. >> Although changing database schema is not easy. >> Exactly. >> Especially on big busy databases. >> Yep, I'm doing that right now. Changing just simply changing a column to a big into is a multi-day process. >> Well, and even your AI agents that spin up this own little test database looks like, "Oh, I tried it 400 times on this little database. This one worked fine. Now I'm going to apply it to this 17 terabyte database. And honest, it'll be fine." >> [clears throat] >> And the website is down. Yep, exactly. >> I guess that's that's the other thing. AI is trained on the average case more than the exceptional. >> Mhm. >> That's true. I know there's some say you know, I mean, planning before executing is just going in and executing without planning at all. Something about saving money. I remember there being a saying and right now it's frustrating me, but would you rather spend the time on the front end or the back end? Your user base is >> Looks like we've got one final question to wrap up the hour. Everybody wants to take a look at that. Um database is all about storing data with security and privacy. What is the point of designing it with AI? Is it the correct way? >> I have no problem with someone asking an AI's opinion about things. But I would at least want to lay out the the security concerns and privacy like boundaries by hand first. Get a few humans to agree on it. And you know, even then like what Leon was saying, it's I don't know that I would trust an AI to make schema if it's anything complicated anyway. >> Yeah. I and I have no idea if AI can do it, but one benefit that humans can say is like, okay, let me shoot holes in this. Let me Let me think about this and go if I were a bad agent a bad person, how could I possibly get this data that I shouldn't be allowed to get? And approaching it from that perspective could help us find those issues before the design's even done in the first place. I have no idea if AI could do that. I mean, it depends on the model and it depends on how it's trained, but as of now I've not seen any evidence that it could do something like that. >> Well, I think the AI could act as that agent. Now, it won't be perfect, but if it does find something a flaw with your with your schema or with your security and it is real and you can verify it, then that's a real problem that that you uh that you closed off, hopefully. Um but so it's one of one of those things where it's not a uh you're not chasing perfection, but at least you can bring your minimum standards to a certain level. And that's not necessarily a bad thing. >> Though again, that's a place where you need some experience on the other side cuz from I had a it was things where AIs can send you down rabbit holes. They could burn days. And just because it doesn't understand some nuance thing. Like I had one that I I wanted to just with like some InfluxDB containers. I wanted to be able to throw in a little CNAME. I was like, all right, I know it's running DNS mask internally, so it's going to be some little way to just stuff in a record for that. So, I asked AI and it totally is convinced that I can use its internal zone configuration to do that. And absolutely no. That doesn't work. And after you have probed it a few times, it finally is like, oh, you're right. Yeah, that wouldn't work at all. You need to have this other infrastructure around to do that. And if your initial person reviewing the AI's review doesn't know what sort of a off-the-wall, completely wrong assertion it's making, or something that doesn't matter, then you're going to spend time going down all these avenues that could steer you completely the wrong way. So, you this way you still need someone with some knowledge. And as Hunter was saying, you know, the people with knowledge don't just grow on trees. They don't just spontaneously pop into existence. They started as juniors. They started as people who didn't have a clue what they were doing. Or thought they knew everything, but turns out they knew nothing. Type thing. Yeah, you always that that Yeah, you get to that point where, you know, I'll trust you if you know that you don't know anything. Type thing. Cuz you you're past that initial point and realize how big the space really is. And then you're safer. But yeah. >> Yeah, our our skills and expertise and knowledge are built on the shoulders of the giants that came before us. We best be careful not to replace those giants with AI. >> I'm built on the giants, and you know, we're at the point where the AI will try to start convincing us that it is the giants. Type thing. >> Yeah. >> And yeah, we have that, the history, everywhere I came from, and our own internal personal improvement and development and creativity and stuff that need to stay around and if we keep cutting off the bottom of the tree not going to happen in the future. >> Think one other Bruce Block or something like this. Like if you're doing something in a scheme completely out >> I'd be real hosed if you're not big picture guy. >> Another thing it's just sitting there looking at it and it will figure it screen for long enough it will figure it end up spending three, four figure or something >> And I'm sorry I'm not I'm unable to hear you. >> Being able to shoot something off being able to come up with like quick plan quick suggestion is not necessarily always a bad familiar what's what you're trying to do you don't see it as a bad completely lost to say how would you handle it? It gives you an idea and if you end up poking >> Sorry, your voice is breaking. >> I think we need to get Hunter some new earbuds. >> Yeah. >> One second. >> Well, possibly fill in what he's saying is it's like stare stare at a problem try to figure it out yourself and then you can use the AI to say what am I missing? You know, help help me break through this like little wall that I can't find. versus again doing everything for me. Hopefully I'm understanding that correctly. Cool. >> Yeah. Yeah, so AI I think is a great partner in these things. You just have to be careful how much you lean on it. Cuz you're You also have to remember that the more you push the AI, the more you're outsourcing yourself. Good thing. And your accomplishments, your creativity, your everything. Yeah. >> And humanity in general. >> I think that is a great way to wrap up today if nobody else has any further questions. We're really grateful that you joined us today. Um this has been a very interesting conversation and I'm glad that you were able to take part of it. But thank you for being here today and thank you Greg, James, Brian, and Hunter uh very much for all of your input.