Master Debugging and Performance Optimization in Visual Studio

My name is Mark Downey. Um I'm a product manager on the Visual Studio team. I'm here with my colleague. >> I'm Sisha >> and we're going to be talking today about debugging and diagnostics in Visual Studio. Um this particular course, this particular session is tailored towards essentially uh with bias towards C and C++ developers who have uh strong um ideas about um the code being incredibly important to you. Um obviously today there's a lot of code generation going on. But for from our perspective, if you think of code as an absolute asset, if your code has a ton of quality gates around it, then this is uh the kind of session we want to share with you because we've been looking at the ways in which we can use co-pilot to help when you're diagnosing issues, when you're debugging issues. Uh we believe quite strongly that we have some of the best debugging and diagnostics tools in the world. And I think we've found ways to augment them um so that when you're generating lots and lots of code, you can still be assured of the quality that you need for your for the industries that you're in. Um today we're going to be focused on two areas specifically. Um we're going to be going through my colleague Sish is going to be going through the debugger agent and the ways in which you can employ that in in your typical debugging scenarios. Um, I'm going to focus more on profiling areas of diagnostics when you want to make things faster and more performant and you're thinking about the ways that it's maybe um maybe you want to save money by making sure um that it can um uh meet the needs of your customers with maybe less hardware or less less um compute resources. So, I'm pretty um geeked about sharing what we've been working on recently u with you with the from the team. Um in fact I'm going to go ahead and hand over to Sisha now and uh and see get started. >> Hello everyone. Uh how's everyone doing today? >> So let's start off with a quick wakeup poll this morning. How many of you spend about 1 to five hours debugging your bugs in your code base every week? >> Wow. We got >> Okay. How about 5 to 10 hours? >> Okay. >> Some of it. Some of you, some of you. Okay, cool. Wow. So, that's exactly what we're here for today. Um, we will uh I'm Sisha and I'm excited to showcase some of Visual Studio's debugger agent capabilities. I'll help you diagnose and resolve issues more efficiently. Um, let's start off with the demos. Uh, the first demo that I'll be uh going through today is the bug fix workflow in the debugger agent. And this bug fix workflow, it's it's an end-to-end debugging experience with Copilot. Um, and the way it works, it takes in a bug URL or a bug description to start off with and then guides you all the way through to resolution. Uh, and it uses live debugging to do this. So, uh, it's more of a buddy or a pair programmer rather than an autonomous agent. And, uh, the beautiful thing with this is you can interact with it. you can steer the direction any way you want uh at any time in the session. So uh for the purpose of this demo, I'm going to use an open-source application uh called Tumble 3. Tumble 3 is uh Tumblr Twitter backup blog application and uh it downloads photos, videos, audios and text posts from a Tumblr or a Twitter blog. And for the demo, I'm going to use this particular bug uh which is clicking view image shows garbled or corrupted image. So what I'll do is copy this bug URL. Go back into Visual Studio. Um jump into Copilot Chat right here. First thing I'll do is select the debugger agent from this dropdown. Uh this is a bunch of agents in this dropdown. debugger uh is what we need. And this brings up the debugger agent homepage here which showcases it. It has a few icebreaker prompts in here that you can get started off readily with. And today I will use this prompt uh which is debug. And then here you can replace this with a bug URL or a bug description. So just replace that with the URL here and feed that in. What this does is it's going to go to GitHub and fetch the bug details like the bug title, the bug description, comments, any screenshots in there. Um there you go. Uh in the case of this bug, you can see it has a few um kind of a few uh steps to reproduce the issue right in there. So it pulls that off. Um >> so what would happen in the scenario where there were no no uh debug steps? No, >> if there's no repro steps then it uh looks at the code and whatever details you provide in the bug description and tries to generate the repro steps from there and um in any way it will confirm them with you before moving ahead so it doesn't waste your tokens. >> That was a GitHub repo with a pull request. Yes, it's an open-source repo that I'm just using of and u we could do a devops. >> Yes, you can do github, you can do as your devops or in case you're not using one of those two, you can always paste in a bug description. So just click on that uh thing, you can say debug and just give in the bug description or investigate my issue bug description. >> And how do we get the debugger agent set up in visual studio or It's readily, it should be readily available in that drop down. >> Yeah. Yeah. As soon as you started, open up the co-pilot chat. One of the drop down there for the agents is debugger agents. >> The reason I'm pointing this out is when we did a few customer surveys or interviews, a lot of people pointed out that they don't really know that the debugger agent exists in this dropdown. So, um it's really powerful tool. Okay. Um, coming back in here, would you like to reproduce this issue manually by running the app under the debugger or should I try to use the testdriven workflow? I'll demo the test-driven workflow later in our second demo. For the first one, let's use the first option which is launch my application under the debugger. What it does now is it looks through the you know repro steps that it has goes through the code does some analysis and then it tries to come up with some hypothesis of what's happening there. >> So so do you have to manually walk through the steps of the bug? >> So um right now yes um once it launches the debugger it'll ask you to just like reproduce the issue. So you're manually going and clicking on things. Um but we are actively working on something uh which is UI automation and soon in a month or two the agent will do this for you. It'll automatically click through things. You don't need to worry about it. Uh we're getting there. And uh the second option that I talked about which is the testdriven workflow. We'll see that in the second demo. Um there you don't really need to do that. It generates a test and just debugs through it. But anyway, uh here so let's see what it did. Um it came up with some hypothesis and it designated one hypothesis as the primary hypothesis. This actually looks reasonable to me. It says that there is a one-off uh bug in the file right in the file downloader.cs. Um and that's what it thinks is the root cause of the issue. But what it'll do now is would you like me to start debugging to evaluate these hypothesis um node it says it'll set break points uh at certain points in the code and um inspect the runtime state to confirm which hypothesis is correct. So let me say yes start debugging setting break points and running the debugger. So it sounds like it's looked at my code. It's set break points in the right places. >> It's going to gather all the kind of information I typically want when I'm debugging an issue. >> And I've just all I've really done essentially is copy and paste the the ASDO ticket or the GitHub ticket. >> Mhm. And then it uh it's asking me here to reproduce the issue. >> So um I scroll down here. um but gave me the repro steps asked me to follow that. It launched the application for me. Um I already cued the blog earlier. Okay. >> So I am going to start the download. Let's see what happens here. >> Okay, there there you go. Um it had a break point for one of the hypothesis uh that it hit. Now it's going to evaluate expressions, look at some um runtime states, variable values, the call stack, everything just to analyze um to confirm its hypothesis. And that's exactly what it did here. Hypothesis confirmed. Looked at the values. Uh >> sure. >> Uh you need the solution open here. >> Yes. >> And then it'll read through the code. >> Yeah. you've you've already cloned it at this point. >> Yeah. >> Yeah. Cuz ultimately when it's going to go look at your I mean it's only going to know to set this the break point in the right place after kind of analyzing your code and thinking about the steps it needs to get to that that bug, right? So the call stacks all that information kind of goes into that. >> So think of it like a buddy or a pair programmer. So whatever you were doing previously, you're kind of following some of that. But it's going to help you uh understand, okay, this is probably the, you know, place in the code that I need to be looking at. Let me strategically place these break points. Um get to some hypothesis and debug through them. So more like a pair programmer buddy, not not a complete another autonomous agent. Um and you can interact with it like I said. So here in this case um it looked through the values the variable values here and uh it confirmed the hypothesis. It says okay this is the root cause and it has a proposed fix. Would you like me to apply this fix? We say yes. >> So we we've seen it use the the break points. What tools does co-pilot actually get a hold of in this scenario? So um here in this particular case it was looking at variable values and call stack. Um if you um actually click on this one you can see all the tools that are available to the debugger agent. These are mostly tools that were available the regular debugger tools that you would manually use to debug your issues and now the debugger agent automatically has access to uh most of them from here. Um and you know uh some of them are basic tools. So you cannot unselect them and it uses those some of those that we're continuously adding tools in here. Uh for example this this disassembly tool is not enabled by default. Um and if you want to such tools you can just like select it and enable that as well. >> I can't remember the last time I used the assembly. just a >> yeah some of the tools that it would likely use uh we keep them off so it doesn't derail the investigation uh sometimes but um yeah and uh in this case I would like to point out the initial hypothesis turned out to be right uh with different models and even the same model different runs um and just depends on the complexity of the bug the initial hypothesis may not always be right and the beauty of this workflow is that if it lands with a you know an incorrect hypothesis at the beginning while it debugs it kind of self-corrects itself. So it it'll find new evidence based on the values and then it says oh actually you know what this other thing is what I think is the issue not that initial hypothesis. So it can go through a couple of iterations and then fix itself and get to the root cause. So uh in this case what we said uh we asked it to apply the fix. The fix is applied and it tries to use the hot reload. Uh hot reload applied the fix success. Oh um actually you know what I forgot to show how the images were looking previously. >> Uh but uh it could do that now too. >> Um oh it already fixed it. >> It already fixed it. >> It already fixed it. Uh so you know so it applied hot reload and it fixed the issue. So the images are already looking good to me here. Um but if we did not do that um and if we >> we are not guaranteeing that these agents fix problems before you see them just to be clear. [laughter] >> Just to be very clear. >> Yeah. Um so okay let me stop debugging if any of you are interested in seeing the images before uh I got got too into too much talking and I forgot to show that one but um it applied the fix and then um it's oh um yes confirm that the images are displayed correctly basically just gives you a summary of the root cause and the fix that was applied at this point. Um, and then it does ask you if you want to run uh any related tests. So to cover the broader regressions here, uh you can always say yes, I'm going to just say no. I'm done for now. >> Yeah. So in my repo for example, I always I have instructions at the root that say if you make changes here, I want to have a bunch of uh additional unit tests. And so you can have this this will respond to those kind of instructions that you have in general. Yes sir. >> When it starts making changes does it automatically like cut a feature branch or >> that's a great question. So typically uh it it doesn't do this in this scenario. Um but given the tools in v inside Visual Studio once it starts making changes you can just say hey grab these changes and just start a you know start a feature branch for me or what have you. So you can you can just at that point keep going. >> So what I did just now is I did an undo of the change a fix that it applied and I cleaned up the directory. So I can do u you know a quick demo of what the original issue was. Um going a little backward but all good. So here uh if I do a downloaded a few files in here. >> All right. >> Look at those images there. Um like it's not complete. It's garbled or corrupted. So this is the original state of it and this is what we fixed and the images you saw previously um after it applied uh the fakes and the hot reload those were the fixed ones. So, so that's how you can go uh to summarize like from reproducing the issue and um you know looking at some code analysis formulating the hypothesis and then um debug through with breakpoints and inspecting the runtime state and applying the fix uh validating the fix and uh that's the whole workflow and that's uh that's the summary of what we just did. um this would have taken many hours to debug otherwise. So the goal here is to um accelerate problem solving. Yeah, >> we have an internal uh sword. Uh you go ahead. Uh it tile. >> Yes. In the um when it says applying uh would you like me to apply the fix? It shows you the changes that it's going to be applying and then you have to say yes, confirm for it. Um, that's one point and towards the end like you saw uh it'll just say this is the summary of what I changed. If there are too many files, too much in there, it's going to show you some of that details like in a table and then uh you can always go to the get changes and then it will show you some of those changes. So there's different >> you don't use git. Okay. Yeah, it will always show it to you in the summary. >> Yeah. um >> total changes here. >> Here >> here when you can also click this total changes and then it shows you the changed files in here. >> Yes, I I undid my change to show back the issue. So it can't really see everything in here but um otherwise it would show more details. So that's our first demo. Um any more questions? Got a couple over here, please. >> What was that? I didn't >> Oh, it's the link of a bug uh in GitHub. So, you can use GitHub or u ADO bug links directly. Um and then you can paste them in there. Just say debug my bug URL. If you don't have a GitHub or an ADO link, you can always use a bug description. You can just, you know, put in your description and say like debug description. >> So it was like >> Yes. >> Exactly. >> Yeah. Yeah. Exactly. That was the root of this this demo. Oh, sorry. There's one more question here. >> Uh >> um I picked Claude Opus uh 4.6 for this demo. That's usually what I go with, but you can always use different demos. Oh, sorry, different models. There is an auto um option here too. >> Recommended. >> Yeah, >> here >> I have a recommended model. >> I uh have better >> Yeah, I have I have a recommended model for Microsoft. [laughter] I'm right. >> Yeah. Yeah. I have better success with the cloud models for this particular workflow to say, but um the others are catching up too. Um, some of the GPTs behave well. Um, I use Cloud Opus 4.6 and I've just, you know, I've never changed that drop down in a while. There is Opus 4.8. The newer models definitely do, you know, better as we go. Um but there is also an auto here um which you know the visual studio will automatically pick the model that is best um for the scenarios if you don't want to pick but my default is this >> same same to be honest was there another question here >> yeah is this bugger agents also available in VS code >> uh it is not no this is a this is Visual Studio exclusive yeah yeah this is currently Visual Studio exclusive we basically have a bunch of tools um that haven't typically existed in VS Code and we are kind of we we think and we believe there's a ton of value in those tools in situ in the place in Visual Studio. Not sure what the future hold maybe maybe they'll come um to to VS Code 2 but right now Visual Studio is the place sorry >> if you >> sorry >> so just to add on to that if you think I mean forget the AI world if you just look at Visual Studio and VS Code the debugger that was in there in both the products the debugger agent is just kind of using those same tools that were already available in those products and then bringing out you know it's doing those the analysis and the the debugging everything for you or with you. So what I mean there is not really any additional features that are added on uh visual studio code at this point for the debugger. Got yes >> I have a question about so I have the request license right now and I do see the debugger when I select the drop down >> but for me it's I don't see the second pad interactive that I'm seeing in curve >> what version >> or the Asia is selecting uh I'm seeing like a exclamation which says premium model so is it like a separate subscription >> no it shouldn't be separate description. The only thing I can imagine that is that we're on a slightly lighter version than you are. Um um so I check your version first, your version. We also when we doing these talks, um we sometime you'll see she's um got main XP in the top right. This is inside the baseball, but um she's she's got a really late version of Visual Studio. So if we could be like right on the cutting edge and you might just be a few versions behind. So but it's definitely coming to you. >> Okay, sounds good. So that interactive property I'm not seeing that second new feature. >> Yeah. Yeah. Absolutely. All right. So >> uh not yet, but I like where you're thinking. Yeah. >> All right. Is there a way difference >> in a solution specific debugger agent? So I think the way I would approach that I think I've kind of alluded to earlier. So in my GitHub repository I have a set of instructions that are infect impacting the entire session I'm in. And so and so maybe I'm not getting the question then. Could you help me? >> Oh, I see. Oh, I see. >> When the debugger agent, that's actually interesting idea. I never thought about that. That's that's really interesting. Did you get what he's saying? >> So, he wants the debugger agent and he's thinking about, hey, when the debugger agent is I want you to also know that this is the way I work like >> Yeah. Yeah. Yeah. Yeah. Yeah. I understand. Okay. Could you come talk to me after give me a bit more detail? >> Yeah, let's talk after the session. Yeah. >> All right. All righty. Um >> Okay. Let's move on to our second demo here. Um, so we're going to be looking at that second part which is we we looked at the the manual uh debugging with launching the application under the debugger. Now we'll look at a test-driven workflow where it generates a test and then debugs through it. Um, and for the purpose of this demo I will use this uh yet another open-source uh application called friend. front is more like an ECS entity component uh system and I have this bug here uh it's an unhandled index out of range exception um and it has the stack trace it says happens intermittently depending on which entities currently exist u and not every query hits it. a few details, not too much. Let me select this URL again. Uh go back to Visual Studio in the Copilot share like I showed you previously. We select the debugger agent again from the dropdown and I will use the icebreaker prompt again. Let me replace that with the bug URL again. Here it'll do the same thing. It's going to go to GitHub. It will fetch in the bug details. Um the title, description, comments, screenshots, anything in there, any information it can get from it. Okay. Here. So here as you can see it's thought process. It says repro steps are not provided. Let me go back. Prepro steps are not provided. Let me check the stack trace. So here's the bug summary. Now it's asking me, do you have any further information that could help reproduce this issue or do you want me to continue with the available information? I'll say I don't have anything else. Let's continue. So um this one particularly just makes sure it's getting everything from you before it just jumps in and tries to do you know dig deeper to um analyze the code uh just to make sure it's not using too many tokens unnecessarily giving you a point where you can say oh I have this additional detail let me just give that to you >> that's one of the things that we've been thinking about a lot is that you're kind of handing control over to something else and the idea of to token consumption's come up a lot and so we're always making sure that we're asking the right questions to kind of get you on the right track to make sure you're not doing too many unnecessary round trips. So, it's just something we've been looking at and making sure that we've got really good efficiency when it comes to caching and making sure you're not um kind of uh uh spending more tokens than you need to for any given scenario. >> And here in this case, uh this is an interesting one. It says okay I found a failing test here and uh it's going to look to see if that failing test matches and um okay this primary hypothesis there is the move next here um which doesn't skip the empty arch types. Arch type is basically grouping the um entities that we just talked about um based on the components. components are more like data that are associated with the entities in this particular case. But um let's shift the focus back here. It says okay I think it it tried to use that uh failing test that it already found in the repo but um and it came up with some hypothesis. Then it realized um here in that um the bug is clear um generic version. The fix is to not um go through those arch types essentially that have empty entities. And then um it says okay it realized that it cannot use the existing failing test. So it's writing a new test or generating the new test that will reproduce the exact uh exception stack trace. I'll create a test that reproduces the bug. So it generated this test building the solution. Um so it can find the test that it generated. Okay. And the test failed with the same exception and the stack trace. >> So this is what you expected to happen, right? So you it created a test >> for the failing issue that you have to validate then that it can actually fix the test for the failing scenario you have. >> Right. Yeah. So what it did was it it initially found that there was a failing test in the repo. So it was trying to see if it can use that. If that is appropriate, then it figured it's not the right one. >> So then it generated a new test um that matches the same exception and stack trace from the bug. >> So it confirmed by running the test that it uh matches. And now what it's going to do is uh run the debugger. Uh it placed break points um here evaluated expression. So it hit that uh exception already. Then it's doing the valuation um at the break points to inspect the runtime state. The exception is confirmed. It's confirming its hypothesis. stop debugging. Okay, it says okay, the root cause is confirmed here and this is the proposed fix. It's asking me if I want to apply the fix. Uh me say yes. So again uh similar thing in this uh case as well. Sometimes it picks the wrong test or it generates some tests um which again as it tries to run the test it'll figure something is something else is wrong with it. It may iterate a couple of times all depends on the complexity and how easy it is to you know generate those uh so but it will uh run through a few iterations sometimes and just selforrect. So there you can see the fix was applied and then it ran the test again and the test passed this time. So that verifies uh that the issue is resolved. Yes, the issue is resolved for me. Okay. Any questions there? >> Sure. >> Uhhuh. or >> yeah it is it's showing the whatever the changes are here and then I just click keep and then I'll keep my changes. >> It does have it does have >> it's basically saving the changes. >> Yeah. Yeah. >> Sometimes I don't uh it seem like [laughter] >> I often don't to be honest. It's sitting there almost like waiting for a confirmation for it, but in essence that that that is actually >> it's giving you an option to undo. >> Yeah, it's really the undo that's there. It's it's done it >> in a pointing out to UX contradiction. I like it. I like where you're going with this. Yeah. Yeah, that's fair. >> Yeah, that's fair. Good point. Yeah. Um yeah, it shows the changes in detail here similar to the get changes and um yeah uh >> great question. >> We've been focusing more on the C uh aspect, but it is supposed to be I don't see a reason why it shouldn't. >> Yeah. >> Um Yeah. Um >> I'll give I'll give you my details. We'll I'll follow up with you. >> Yeah. >> Yeah. Yeah. We >> Yeah, we'll be this is the first version actually. This isn't shipped yet. Um I'm demoing to you something that's coming to you. So we've focused initially on the C# version like I said. Uh we will be testing with the C++ as well. Usually that there shouldn't really be uh any changes on that side. Um and the mixed mode should work too. >> There's another question. Uh so we have been doing a lot of work with uh token consumption and being able to validate view that um uh so uh at the bottom right I believe there's a new token um there's a new token viewing token billing uh uh user experience I was just kind of informed about >> yeah that's somewhere um as Oh, >> I will find it. I I >> Yeah, >> on where you show the model and their dot. >> Yeah. >> Yeah, >> right here. >> No. Yeah, I I remember seeing it somewhere in a different version. >> Sorry, the little >> this this shows the progress uh at this particular >> Yeah. >> Yeah. This we just released this in a prior version. So I'll I'll get the details for you whoever >> probably the version that I have here doesn't have it. >> But the thing I think the other thing that we've been asked here that we don't have that I'm hoping we can do more work on is kind of getting a guesstimate of of kind of the consumption work that's going to happen on your behalf. Um but we definitely got the this is how much I'm consuming part of this uh story going already. So u yeah please come talk to me and I'll I'll follow up with you. And um continuing from this demo, so any any more questions on this demo here? Just to summarize, what we did was we went down the test-driven flow and it generated a test for us and then it debugged through the test. Made sure the test was failing with the same exception stack trace initially and then it went ahead and debugged that and um made sure uh after it applied the fix the test is passing. Um and um then it it just at every point it confirms with the user if they want to apply the fix if you know they like the test that is generated uh etc. So um so what I'll do next is I will actually undo this particular fix here and I'll keep the test. So the test is still generated and the fix is um not there now. So what I want to show next is um in that case when I um so sometimes you start off with bugs but sometimes you already have some failing tests that you want to debug and uh that's the scenario I want to show a few options here that you could use and um let me go actually search for the oops this is the name of the test that it generated and this test will now fail because the fix is not in there. Uh there's a test explorer here and here. Let me paste in um and search for that test. So um this is the test. I can always run the test from here. and the test fails because we don't have the fix. So what you can do is you can always debug with copilot from here and it will pretty much follow the same thing that we did previously. Uh it will jump into the debugger agent. it will place those break points for us and it'll debunk through it very similar to what we just saw and make sure the test passes after it applies the fix. So that's one way of doing it debugging test. The other way you could do it is I can just say hey debugger agent just debug this test for me here. It'll follow the exact same workflow. Um, and taking this one step further, what I can also say is find all failing tests in this codebase and debug each one for me. So, if I give this, I'm not going to demo it in the interest of time here, but if I were to give this one, it'll just run all the tests um for this repo. When I ran it last, it said there are 73 tests in here. two of them failed. One is the one we just generated and one is the other one that it picked at the beginning, very beginning as you saw. And it'll say, "Okay, let me start with debugging test one." Um, looks through and follows the exact same workflow that we just looked through with break points and finds a fix for it. Does the same thing for the next test as well. And then it'll just give you a summary of like, hey, these are all the failing tests. These were the root causes and this is the fix for each of them. Would you like me to go ahead and apply this? So, uh, so you could debug test in different ways. No >> debugger like a different is it anything that that takes >> so >> yeah uh if you don't select the debugger in this a in this here usually defaults to agent the right now the VS agent the default agent goes and tries to process it with the um information that you provided And that is usually a good case for generating code and you know resolving build issues and things like that. But anything that you want to do specifically with debugging debugger agent is the one that is tuned and has all the tools debugging tools that are available. So it's in our best interest to select that from the drop down right now. What you could also do is you could just say add debugger and then it'll do the same thing for you. It'll show you that thing. Um and what we are working on actively right now uh is also something similar to a lot of people do what you just said they are just in the >> agent and >> that's how yeah so we're acting on a handoff tool where the you know the agent can figure out like hey this is this looks specific to the debugger so let me just do that handoff automatically and I guess one thing that we're always curious about is like do Does do the users like to do that automatic switches though um for you? Um and what kind of uh frustrations you would have with it? Um if we you know we're not you know you are not the one doing the switch we are doing it automatically for you. There's a few options we could show you. I mean obviously the drop- down switches but we could do some notifications and stuff but yeah there's a lot of uh thought process to do the automatic switches just because of the the token constraints that you know we consider >> I think I would like it as an option that you can turn off turn a >> question here as I'm ask I see. Go. Wait. No. Wait. No. Wait. There. No. So with the debugger agent it you know it's a similar flow where it is going to give an initial hypothesis but again with that debugging it can see that hey it looks like you know there's new evidence this is really not the root cause. So it invalidates its own hypothesis sometimes and then it could come into that next iteration but you always have the option to interact with it. Uh it tries to selforrect if it's not right the first time. >> Okay. theory. >> It it it's certainly possible if you don't if you take away the guardrails. Yeah, actually >> and those are exactly the the the ones >> two weeks ago or something literally what he said 20 minutes 30 minutes stop it some I can't even stop it shut down relaunch it. >> Yeah. Thanks for the >> Okay, we would love to learn more about uh such scenarios. >> Right. Okay. >> Yeah, I got you. >> Yeah. Yeah. Usually with this agentic flow that the bug fix flow that we are showing in here, there's a bunch of questions it checks and if uh it is if the user says no, this is not the right thing after a couple of iterations, then it'll ask you more questions and say let's you know look at some of these things. They'll ask you some clarifying questions. So are all image in the previous case like are all of these uh you know images corrupted? Do you have specific image types that you're seeing there? Are you seeing this corruption just from visual you know when you're launching the application or you also seeing it in the uh photo viewer? So it'll just ask you some of these extra questions so it can get more information and go in the right direction. So uh we need to keep moving in the interest of time. Let's move on to our uh third demo for the debugger agent which is um analyzing I think call stacks. We um I'm going to use the ehop application. It's an e-commerce application. Um and let me get started right away. It's a little slow in here. Could be my internet. Okay, that's going now. >> So, what kind of app is this one? >> Um, this is a net Aspire app. It has a bunch of services. So uh it'll be an interesting demo for ing call stacks dashboard. Okay, let me go here and launch the web app here. Let's do some shopping quickly. Add Okay, logged in. adding some items to the shopping cart. Check out and place order here. All right. So, I submitted this order, but now uh if you see here, I have some previous orders that are kind of stuck in the stock confirm state. They're supposed to be turning into the paid state because the we've placed the order the entire process is done and um but still it is stuck in this particular state and not moving on to the paid state. the from the little bit I know from this code base the payment processor service is the one that moves my status from stock confirmed to paid. So what I can do here is let me go to this ehop application here and attach to the payment processor process and see what's going on. Uh let's hit break all. recall kind of freezes all those um threads there and I will switch to the main thread here to see what's going on in the call stack. The call stack says waiting on async operation. Double click or press enter to view the async call stack. Instead of sitting here and trying to um detangle all the sync call chain uh by myself manually what I can do now is I can click this button to say analyze with copilot and copilot does this automatically for me. it will uh look through the the different uh processes, the threads, the the async chain and uh look through the relevant code and then give me a hypothesis. I like to read some of the thought process usually uh see what's going on when I'm waiting in here getting the call stack and thread information. So instead of me having to go and look into this the advanced features of debugging which is parallel stacks and uh looking through the the task threads here it kind of does all that for me. So I don't really need to know those advanced concepts anymore. So it looked through the through the code and then it found a root cause here. It says um the essentially what's happening here is there's a request authorization service. Uh it has a pending dictionary that is keyed off by the order ID and it's waiting for a call back. The fraud check service here um brings the call back. It calls the um complete authorization sync but it's using a different ID rather than the original order ID. It's using the notification ID here for the call back. So because of that, this task is never completed and it's awaiting that handler. It just hangs there, sits there forever. And um the the fix is pretty simple. You just use the order ID instead of the notification ID here. So So the real win is I don't really need to look through all these async call chain. Uh and here in this case it's a silent hang async hang that I don't even realize until if until I actually dig in right um so copilot just does this for me it saves me uh many debugging hours by just walking through that and giving me the root cause and not buying >> that's one of the issues with async code is that it can it pass all kind of comp compile checks and then when you actually go to run it that's when you really kind of see these issues and you kind of almost need a kind of snapshot in your in your production environments or you need a snapshot inside Visual Studio. So you can kind of get a good sense of the actual state of the of this of the of the ASIC machinery. So this is what this thing kind of goes and helps you find that especially when there's lots to look at. So >> so just to summarize everything we looked at for the debugger agent today, we looked at the bug fix workflow. We looked at debug uh test in different ways from test explorer from the debugger agent and then we looked at a sync call stack an analysis using uh copilot as well. So um and these are all some of the latest debugger agent capabilities that can help you diagnose and resolve issues much more efficiently than you would otherwise. And uh now that we've looked through all these things and you know we go fix all your bugs, what is usually the next question, right? Uh how is my app performing? That's usually the next question after debugging. And that's where I'll hand over things to Mark Downey, my colleague here. He's going to walk us through something called as the profiler agent. The profiler agent can help you identify and fix performance bottlenecks in your application. >> All right. Thank you so much. So just to be clear, I want to make sure everybody knows this. I am not a performance expert and they threw me out here because the assumption is that we could supercharge people who were not necessarily performance experts. Right? So if this goes well, our hypothesis is correct. And obviously if it goes wrong, it's all my fault fault really. So um so so the the the um I've been working on a open source project um for about the last 12 years or so. Um it's called DOSS blog. It's one of the oldest um blogging engines out there on.net net and I spent years kind of changing everything from web forms theaspx thing to MVC right um I spent no time looking at the performance this code has been around since 98 kind of got really popular in 2005 I'm giving away my age here um and it kind of went into decline and I've tried to kind of resuscitate the whole thing so I've done a whole UI revamp but I never looked at any of the performance questions um I pay for my own hosting. I've got several blogs out there, several things I'm using and I want to kind of make it faster and I want to actually use less app service instances, right? And so the qu I'm hoping so I can do some profiling to help me address some of those issues. So let's dive in together. Let's look at my prompt really quickly that I'm starting with. It's fairly simple. I want to improve the cold startup time for DOSBlog runtime. I'm using that because I know that's a project name. I want to give it I want you to look at the fundamentals. This whole project doesn't include databases or services. It's just basically files on disk for every post, for every comment. I've got thousands of them at this point. I've been blog I've been blogging for for years. Um there's thousands of files. So it's just doing a bunch of, you know, uh IO. It's doing a bunch of strings. It's But this is the stuff it's doing has been around since 2005. So you can imagine there's going to be just an absolute ton of old things like lists are not modern now but there were no lists in the old cloud code. it was all collections, right? For some of you who are old enough to remember what those things are, right? And so I want to modernize that code. I want to make it better. Um, ultimately I just want it to perform better, right? So let me see if we see what this comes up with. So I actually dragged all my blog content, all the 20 years worth of blogging I've done. I've dropped it in a folder at devtemp and I've said, "Hey, point at that and start per doing some performance testing at it." Right? So act as if that's my website and start doing proven performance testing. So right now the thing it's going to go ahead and do is uh it looks like it's starting off with a performance test and it's actually saying in saying right uh right here I'm think I'm going to do a benchmark. Now why does it need a benchmark? Um the reason why it needs a benchmark is ultimately because if we don't have a strong baseline anything we do at that point is just uh code massaging. this that like you've got to set a baseline, right? And fundamentally what we've said is benchmark tool is really really good at setting bench lines. You then make a code change and then the last step is you measure again. So measure twice, change once and that process that that profiling agent is kind of locked into that process for you. So you're going to have tons of code submissions to you. you can use this to help you decide whether that if that change is performant enough and and kind of make changes appropriately. Um I found the uh I I've been doing a ton of work recently and so I'm kind of constantly pushing new changes to my blog. So one of the things I want to do is make sure the restart the cold start time is as fast as possible. So, if I'm doing three or four updates to my blog from my um open source project, I don't want to spend two minutes waiting for it to to refresh after a cold start. Uh question in the back there. >> It certainly will. Yes. Yeah. So, it's grabbing the new get package even right now. In fact, it's started actually the it's created the new project on the left hand side here. Um and right now it's having problems. It's it's created what it thinks should be the the kind of template for that performance test and it's still having some issues because you see it's um trying to build a couple of times and it's failed actually the last one right there. So it's going to run through that a few times to be sure if you know how to do this which I do not um you should just maybe maybe do do follow whatever pattern you have existing. I have none of those kind of performance and benchmarking tests in my project. So this is kind of introducing it for the first time. Um, so it looks like it's building again. Build failed. Now it's successful. Okay. So normal it typical iterations here. So I expect the first next thing it's going to do is go ahead and run a cold startup. Test one of those scenarios. I'm really interesting. I want to test as I said I want to test the CPU. I'm actually more interested in allocations is another big one. You imagine as a blog post I've got, like as I said thousands of comments and posts and so it's going to be allocating a lot of strings. You pay for that three times. Once when you allocate it, it stores it somewhere in memory. And then you've got to pay for it when you deallocate that, right? So you're paying for it. You also always want to make sure you're as efficient as possible. And so here we have build success build is green. And so now it's doing the that next step I mentioned. It's doing the collecting the baseline. It's actually going to run four um looks like it's going to run four benchmarks. So, I'm not sure I'm going to get through all of this. Um, but the four benchmarks are around those four ideas that I mentioned around, you know, what I want to actually make better. Um, let me see uh here. >> So, what's your uh motivation to do the profiling and optimization at this point after all this while? >> Honestly, my my Azure cost I'm right at the lip of my Azure cost and I have no more budget. my wife will not let me have one more penny to um so I literally I need to have that thing so that I can do a bit more work literally I the the other part of it is genuinely I mean the code is 25 years old I would say um I'm sure some of you are kind of dealing with code you haven't looked at that's sitting around it's working kind of um this is the scenario we think will help um typically what we see with teams um there's a 20 person team and there's one profile filing expert who gets to use 10 hours a week to maybe look at a few things that are absolutely critical. Um, but I've seen um around Microsoft and you know open source projects we've been contributing to where this has literally helped in a some of the most fine-tuned software out there. This has helped. So let me see how far we've got here. We're halfway through our benchmarks. We've got like 5 minutes left. I'm not sure we're going to get all the way through this. Um >> 9:15. Oh, we got 9. >> Oh, I thought we were done at the top of the hour. Sorry, you've got extra time. All right. So, >> what you're saying is what used to be like an advanced and time consuming performance investigation that's now much more accessible through the profiler agent. I don't need to be an expert anymore. >> The agent will do the benchmarking for me. So, I actually can see, you know, based on the insights it generates and what it wants me to apply. I can always apply those and I can see the difference in the benchmark. the the numbers will tell me how much it's actually improving it for me. >> Yeah. Internally, we kind of refer to this as the democratization of per performance profiling. It's actually a hard skill. It's not even so much it's a hard skill. It just takes a long time to kind of get that accreative knowledge over time. And so we are kind of hoping that we can speed bump people into this kind of uh performance uh skill set. And what I also like about this is that you can ask questions as this goes. So, if it does something and you're saying that looks relatively benign or it looks crazy, u you can ask questions about why it did that and it'll tell you, hey, this is to help the GC or this is to help the jitting when I'm when I'm doing some X, Y, or Z. Um, the the point here is I we are hoping that it doesn't just something you you kind of offload, but it's something that you learn from and you kind of develop skills and kind of more knowledge in the area itself. How are we doing here? We are running the fourth benchmark of four and I'm my hope is that we will kind of get a few options to say what do we want to work on first or next and then the goal really is to start iterating with this thing because we now we have a baseline with the baseline I can genuinely see um if the thing that it's suggesting is it actually improving or not. And sometimes, if I can be perfectly open with you, you know, one in 10 times this does a thing and it regresses, right? That's just the the nature of non-determinism, right? Um, however, the tools to me are what makes the difference here. If we measure before, we measure after, we can make sure that we're actually doing meaningful work here. All right, so we've got some baseline results. For those of you who are profilers, who work in the profiling tool, on the left left, you will see the diagnostics tools window. Again, you if you are kind of knowledgeable in performance, you probably seen graphs like this in the past. You've seen um CPU and live objects um private bytes being assigned. Um on the right, you can see the baseline results it has collected. Um it's talking about the files it's checked derializations that have occurred. Um it's running a couple of tools as Sisha mentioned earlier. These profilers have and the debugger agent have access to all the wonderful tools in Visual Studio, the world-class tools that we've been developing for the last 20 plus years are now in the hands of this this agent. And why that that kind of combination of non-determinism and measuring is to me where the magic happens because ultimately you can't trust it until you've tested it and our tools are the test part of it. Right. So, okay, let's see what we've got uh here. Uh, all right. It's saying, "Hey, I could get you some wins here in the entry clone section." Um, that we could we could skip the cloning and do a a kind of a less deep uh cache of of the details here. That's interesting. Um, it's doing some it's suggesting that I'm using XML serialize and derialize. Again, this has been around. This isn't the runtime. This has been around for 20 years. So, hey, you the way of doing this isn't probably the way we should be doing it. And I can believe that. Um, I actually really like this idea it has. Sorry, I'm I'm skipping around here. I like the idea it has for um doing this in parallel. I just saw that right here. The parallel for reach that, you know, my knowledge of of software engineering is is kind of the it's making sense to me that we do a parallel for each, right? because if I'm picking up, you know, thousands of files, why am I doing them literally one at a time, right? And that so I reckon I can save something there. Um, so my thoughts are I'm going to go with number two. I'm going to do the profile of paralyzed entry load across all day entry files. I'm going to go with that one. And you can see drops down in the the prompt right there. I'm going to kick that off. But again, it's kind of not doing it. It's kind of leading me in a couple of options. I'm still using my smarts here to to think about what is the best decision for my code. Um, last week when I was trying this out for the first time, it suggested that I not um load my comments. I'm like, yeah, that doesn't make any sense. It's a blog. Why would I not load com? But it doesn't know. I know. I'm the center of this, right? I've got I'm the one who's going to know when a tradeoff makes the most sense, right? And so ultimately what you should have here is something that's going to make suggestions, but you're the engineering mind in the room, the only one. This is a nondeterministic system. And so you use this to kind of make u make the things that makes the most sense for your code. Question, >> is it smart enough to recognize like say you're stuck in a .NET framework? Yeah. Smart enough not to make suggestion that only apply to like net 10. >> Yes. It's uh to be sure it's it's the context of the grounding of the whole thing. Part of that context is the framework you're in. Um so the goal should should be to kind of ask you to include um artifacts and things that don't exist uh forward. But again, I I it you have to understand that this thing learns on a corpus of data that is whatever's out there at the moment. And typically that doesn't always things that are older don't always age as well. It does obviously for for a period of time. Um so I imagine that it would do a fairly good job, but it's going to go it's going to naturally have learned and been taught on data that may may make suggestions that are not quite right. It it's it's really really even hard to say but but but it should take the data about the project, the solution, the language, all that information to kind of help you make suggestions. >> All right, we're running through um it looks like it's running through >> basically when it gives suggestions, it tells you gives you a little bit more details in there so you can actually pick the insight or the the suggestion that you want uh that makes sense to you. >> Yep. So versions profile and everything but like and really applicate. >> Yeah. Yeah. And not everything has been migrated. Sure. >> Those you know because those are the one building on old technology. I would filing go there. How could the optimization cheat but older >> versions? That's a great question. So, so I mean I I'll kind of have a slide on this but but we've done profiling on Visual Studio which has parts of it that are net framework kind of that that's just what it will be at least for now and there's parts of it that are net core there part of it that's C++ and so we've done made some meaningful changes meaningful improvements in that codebase that spans you know uh multiple framework versions um multiple you know framework and what we refer to as core um and it was able to make changes that were kind of appropriate and appropo for that kind for that version. So yes, I I think ultimately I think none of these things are perfect, but absolutely it does take into consideration um the version you're using. How far have I got with my Okay, running the benchmarks. Running the fourth benchmark. So my hope um from this change is that it's going to provide at least some interesting feedback on the change it's just made to entry ID cache. Um, let's actually go look at that change really quickly. Let's see what if what it change it made. See if it makes sense to us. Just having a quick look. >> Don't run the benchmarks. >> Oh, awesome. Just look like it's running for benchmarks. As a good steward of this code, I review every single line, I assure you. And so, let's have a quick look at what that looks like. And it looks like it's done something with par parallel for each. I like the I like where it's thinking about that. Um but like I said the proof is in the pudding. Um ultimately the proof is in do did do did we make a meaningful change and if we didn't we iterate we try something else we try another approach and then be beyond that behind this whole project I have a bunch of uh test code that I think um that I'm going to use to kind of make sure that it didn't break anything as well. Let me open this up a bit more. All right. It said mostly flat. Okay. I would have loved to have knocked you all out of the park with that one, [laughter] but it's useful information nonetheless. Right? That means that that wasn't the thing that was holding this thing back. That's one thing I now know now have I now have as my um kind of um grounding for the things I'm going to try next. Um, it looks like I've got a couple of more options to try. And I want to try just one more um just to see if I can uh uh make some changes, make some improvements here. I'm going to actually try optimizing the entry clone. This was the this was actually the first one in the list and I jumped to the second one. So, let's go ahead and try that one. Um, parallelization is safe and correct, but it's not really saving me a bunch of time. So, yeah, let me go down. So this data kind of helps you decide if you want to keep the changes or roll them back. >> Yes, exactly. Oh, sorry. You and didn't read it correctly. So, it actually didn't save me anything in terms of memory. Of course, I didn't expect it to save memory, but if you kind of look here, it's proposing that um the meanantime of the load time has dropped significantly, which was part of my purview here. I'm going to move on to see if I can do one more memory change as well. Uh see if we can get a bit more um improvement in terms of our allocation and the amount of memory it saved. Any questions? Yeah, please. >> Yeah. So, great question. Thank you for asking that. So the profiler agent itself is tuned for this exact scenario and part of its part of its purview, part of the things it can do is go ahead and pull that nougat package down through Visual Studio and set up an entire new project for you. And so based on the code I was looking to analyze, it created a brand new project and is exercising the line of code in the cold startup. So yes, yeah, that we've given that agent uh certain power and authority in your in your solution and part of that is adding a new project, adding new nug get packages to set up that benchmark. >> So here we have some >> please correct. Yeah. Yeah. It's a it's a new project in the solution in the in the my overall solution that has been added. In fact, if I go really quickly look on disk uh see if I can Oh no, not there. See if we can find see where it made that change on disk. kind of don't like doing this while it's doing things because I'm there's this finite CPU on this machine. So, I'm trying um so just what I'm alluding to there is that um when you are doing performance testing and you are running your baseline and then you are running the next one, don't do a bunch of stuff on your machine at the same time because you will kind of kind of skew skew your results a little bit. But my point here is that uh we've just this is the new benchmark suite that was created for me in line in my project. Does that make sense? Please. Yeah. Yeah. I I am going to I plan to I haven't actually made this part of the permanent project. If you go to github.comring/hossblogcore, I haven't made any of these changes permanent. Um my colleagues a few weeks ago suggested I should and I and I intend to. I'm just kind of walking through this with with you all but yes I intend to. Thank you. >> Follow up this private ed yeah yeah sure. Yeah. No worries. All right. So, it looks like we have uh what did it just do? Found the smoking gun. It says it found the smoking gun. We will see. Um currently does XML round trip. Um it's doing some additional pieces here. Um before and after. Let's see. And chasing this bottom of this thing. >> 45% reduction. >> Okay, awesome. Now we've got some changes right in the actual allocation we've got 45% reduction um in the allocations we've got 34% in the count of a particular type we've got a 59% reduction on strings and bytes now this is the kind of change I wanted to make now from the perspective of cost savings um my one app services instance can maybe do two or three blog you know blog post separate blog blogs um I can load rather than the onetoone relationship I have right now where I'm spending 30 you know 70 bucks that I can split that 70 bucks among three three three blog engines right ultimately for me performance isn't just about making things faster because you want them you want to do give your customers more for the same cost hopefully um performance really is about what you can deliver more of and for me as a non I genuinely this is truthful I I am not a performance expert at all but I have I have measur able improvements here um that I can actually um turn into actual cost savings and my wife will be incredibly happy uh with that. Um so um >> this question >> oh question >> Oh, oh, oh. How would you? So, it would just depend on the relationship. Um, so what this can do um if you have hot paths through your code that stretch across multiple areas, you can profile across um across those boundaries. And um so you can tell um you can basically have a a sense of you don't have to have a onetoone relationship between your benchmark suites and your projects on disk. You can have one benchmark that does multiple things across multiple projects. I hope that does that answer your question. Repeat it in maybe give me a little tweak and what what you're trying to ask. required to project >> the benchmark suite that is created that separate project is that for all the projects together or is that does it create one for each of the projects that you have I guess that's the question >> yeah it's just it's it creates one benchmark suite uh project and for that it can tackle multiple things it's trying to manage in your solution into your entire solution. Yeah. Yep. Thanks. >> Power to do balance. >> Yeah. >> The tool that let you do pretty much anything. How do you find that balance? So, >> yeah. No, the context is everything. To to your point, I think um I feel a lot better with some somebody who has uh deep knowledge of the software with almost no profiling experience. That would that's the magic to me. That's the magic moment right there because you know the kind of uh kind of uh sacrifices you're making when a code change happens whether you're you know you might be moving some part of that code to rely more on CPU when in fact it's memory you have an abundance of um your IO may be under threat and you're saying I've got to stop that and I'm happy to kind of burn more memory. So you've got to have a deeper understanding of what your architecture is and what your system is and what you need. I don't think it it certainly doesn't take that out of the equation. This does not know. Like I in my example, it genuinely thought, don't load comments until the first person shows up at my site, which was essentially what it was saying. So, it was saying offset, let somebody else deal with the cost of that change you want to deal with. Didn't make sense to me either, but it would have saved me like 90% extra extra loadup time. That's fantastic but it's useless for the actual implementation of my solution. So yes, you are still the center of this. You have to make the decisions. >> Yeah. So it can give you the different suggestions that you can look at the key insights and you are the one ultimately that's picking what is right for you. The system is not going to make those choices for you. It's going to give you those suggestions. >> All righty. Okay. We're really almost at time. Um so we this did make meaningful changes for us, right? We've made meaningful changes at Microsoft. We think we we did some improvements in optimizations in Visual Studio. The any code open got reduced by u almost 30%. We had a bunch of al 80% reduction in allocations. Azure app services itself we were profiling their their startup times and reduced the actual cost of um of of Azure app services. Rosling compiler is one of the most optimized pieces of code in in Microsoft in general and we were able to increase the performance there. If you want to know more about these comments um kind of use these links and you can uh you can take us kind of those links. Um please do um give us feedback um on this talk. We're really really um happy to engage with you afterwards if you have questions or things I didn't quite answer correctly while I was rushing through this. Um please do um provide us feedback um uh at the on the session surveys. This really helps us understand if this is the kind of content you want and if it's valuable and helpful to you. Um please do uh let us know um how we can do better for you in the future. Thanks so much for your time. Appreciate your help. >> Thank you. [applause]