The Next Generation of Developer Productivity with GitHub Copilot & Visual Studio

Thank you so much for coming to our session today, um, the next generation of developer productivity, uh, with GitHub Copilot and Visual Studio. My name is Simona and I'm a PM in the VS GitHub Copilot team. >> Yeah, and hi, I'm Leah. I'm a toy uh, PM on the VS team for Git tooling specifically. Yeah. >> So, we have a full agenda today, so we'll get right into it. This is the flow for today. We'll first talk about all the agents you have available in Visual Studio, then we'll talk about how do you customize them, make them really work for your team and your workflow. And then, uh, I'll hand it over to Leah. She's going to go deep into how you review and validate the code that Copilot generates. And then lastly, we're going to talk about also the important part about how to manage your model and tokens. Uh, So, this is the agent picker view in Visual Studio. I bet many of you have seen this multiple times where you're interacting with your chat, you're switching between agents. So, we have a lot of options here, as you can see, and today we're going to break it break it up a little bit. But, this is the first, the classic duo, the ask, plan, and agent. Um, so, all these modes, they are the default ones, um, come with Visual Studio, and um, they are like the generic the generic a- assistant you have that can be good for any type of task. And but if you give them a prompt, they're going to give you a different type of reaction and response. Ask is the very old like chat assistant. So, you can interact with the it's going to give you suggestions about how to implement something, but it won't apply the code for you. It will give you a code wrapped in a active window and you can copy-paste or apply it. Um, so, this is like the safest option if you don't want agent to change anything, um, to our code base directly. And now we have plan mode. Um, how many of you have tried plan mode in VS already? Great. So, for those who haven't, I highly recommend you to uh to check it out. I think this was available like April, March, a couple months ago in Visual Studio. And the benefit of plan mode is you really get to iterate uh with the agent with your agent and make sure the plan is perfect, every detail is aligned, and then you kick off um the plan. Um so, the plan mode will break down the different steps um in implementing a new feature. And then uh maybe there are different approaches you want agent to take, you can nudge it to change the plan. And when you're fully aligned, you kick it off, uh implement the plan. And the benefit of using it is that you have a way higher chance of getting it from 0 to 100 in one shot. Uh because if you don't plan it, maybe you'll give a very simple prompt to start with, like create this feature, and then the agent will just kind of guess how to do it. And then you probably don't like it, you have to tweak it, give it multiple prompts afterwards. But with plan mode, you can uh have a way detailed plan and have it right in just one shot. So, super powerful, um definitely uh add this to your list to try out um after session today. And now we have the beloved agent mode. Super powerful, it can apply your edits to our files directly. Um I've been living in agent mode uh since it's available. And then we also have a major upgrade coming to our agent mode, which is the preview agent. Uh how many of you were in my colleague Dalia, who was sitting there session yesterday? Great. Yeah, that was super cool session. If you missed it, maybe check out the recording afterwards. Um so, we're bringing this preview agent to VS. Uh one feedback we've been hearing about agent mode um consistently from you all is that when you're using different Copilot clients, VS, VS Code, CLI, you feel like the quality of the agent is inconsistent when you're moving around. So, we wanted we take that feedback seriously, and we want to bring the quality the same baseline for y'all. So, now in Visual Studio, we have our agent mode powered by Copilot SDK. And that will soon be the same case for VS Code, and that's already what CLI is using. So, we're going to get away um consistent experience with our agent mode. It's still in preview um now, so we really value all the feedback you have. Try it out. Um if you like it, if you don't like it compared to the previous agent, what what type of every type of feedback, let us know. It will be super super valuable because our team is still actively iterating um on this preview agent right now. Um and so that's about agent mode. And then one thing I forgot to mention why I really like it is uh when you perform a task with agent, it will do this full loop. Uh you kick it off, it will implement the code, but after implementation, it will also validate itself. It will try to make a build to make sure it's not leaving you with a mess or a bunch of errors behind. It will always try to fix its errors afterwards. So, that's something I really appreciate about agent mode. Um so, with the preview agent, we have the same thing coming to plan as well. We're going to soon have our plan mode uh powered by the SDK uh Copilot SDK. So, again, similar consistent experience across all the Copilot clients. So, that was the classic uh story trail um of agents. And then we also have a bunch of really powerful specialized agent in Visual Studio. Um they're quite unique. Um we have debugger and profiler um which will help you with your debugging process and performance improvement um processes. And my colleague Harshita did a uh a Mark did a session uh yesterday um featuring these two. Uh if you missed it, there's a recording, and I think um Yujin uh from my team is also doing a debugger demo later. So, if you missed the previous one, um I'll share the time and specific specifics at the end of the session. And then we also have Git agent where uh which Leah is going to talk more about. And lastly is Cloud agent. I feel like this is quite a hidden gem uh in VS. So, how many of you you have your code hosted on GitHub repository? Okay, then you definitely should try this out uh cloud agent. So, it's a remote agent is hosted on github.com. Um but you can kick off a task uh for it in Visual Studio. So, imagine you're doing like a big feature implementation. And then while you're working on it, you notice there's this side bug that is happening. You probably don't want to switch contacts to fix the bug because you don't want to uh distract yourself from the work you're doing, but then it would be handy to have an assistant help you fix this at the same time, right? So, that's how you use cloud agent. It's like your parallel It's a way to parallel your work. You just kick off at Visual Studio, you switch it to cloud agent, uh let it handle the side bug. It will create a GitHub issue for you in your repo repository. And then the agent on the cloud will automatically start working on it. And then once it's done, it's going to push a commit to the issue, uh and then it will notify you so you can review its code and then iterate. Um so, the benefit of it is you can parallel work. You don't even need your VS to be open to use it. If you need to go to lunch, you log off for your work, uh you can kick off uh get a a cloud agent, and you can come back to it the next morning when you're ready. Um so, that's super convenient for parallel parallel your work. Um Yeah. So, that's a quick rundown of all the different agents we have in VS, but that's not enough because Copilot still doesn't know how your team work. Like you probably have those special guidelines, preferences, um internal APIs, and all of that. So, think of the agent mode plus a powerful model. It's a pretty good It bring you to a pretty good baseline. Uh but you still need to onboard it to your team. Just like when you're hiring a new teammate, and they are probably a very pow- a very capable, very competent software engineer. But that it only knows general knowledge about software engineering alone, but it doesn't know anything specific to your team, to your project. So, you probably have a bunch of onboarding docs for them to go through to get familiarized with the code base so they can start working in it. This is the same thing with Copilot. It is powerful enough at a baseline for general task with agent mode and the basic model, but then you still need to onboard it with all the customization features. Um, so we have a bunch of these features available in VS for you to try out. Um, first is the very classic uh I think one of the first customization features as well, custom instructions. Um, who have it set up in your repo? Oh, wow. This is uh smaller than I expected. Uh okay, I'll also add this to the list to try out after the session. Um custom instruction is super powerful. It's good for the basic guidelines, um the guardrails you want your agent to follow on every task you give it to it because how it works is when you are interacting with it, you send a chat message to it. Every chat request, your custom instruction is appended to the system message. So, the model will read it and follow it every time. So, this is good for Yeah. >> Do you need Copilot instructions? >> Yes, yes, yes. >> Easy. Okay. >> Yeah. >> more people might use it by that name. >> Good point, good point. So, yeah, so the file name for it is Copilot-instructions.markdown. Uh so so so by that, how many of you have it set up? Okay. Slightly more, but I think with 50% of the room, we for the rest of y'all, uh definitely worth checking it out. Um So, that's the custom instructions. Um you can also scope it to different file and file and directory. So, if you have some instruction just for this repo for this folder, um you can do it in the front matter. You can define it so so it can scope um to whatever file directory you want. And now we have prompt files. Those are also like I think available a while ago. You can save all your favorite prompt. You don't have to copy paste things around and you just have them saved into a Visual Studio and you can use them as slash command. Once you save them, you just slash and then the name, it's super convenient uh and neat to use. And then we have agent skills, custom agent, and MCP servers. I think these are relatively new. Maybe MCP server is like the oldest, but I think all these three combined together make your agent super powerful that you can way easily to automate a task um you have. A task or a workflow. And now we'll dig deep into these features. So, analogy on I gave you first, kind of how to think about um these customization features is this one. Uh skill is the brain and MCP are the hands for agent. Um myself, sometimes when I look at all these features I got confused. Like they're all markdown files, right? What is the difference between all of them? Um I think this is a very helpful analogy for myself. Um you can think of skill is the part that decides tell your agent what to do. It gives you your agent the knowledge, um the expertise, and then allow your agent to perform a similar task more consistently uh one one time and another. And then MCP actually connect your agent to all the outside world so it can have so it can reach out and then use all the tools outside to complete the task. Um so, here's the message. Skill teach the agent what to do and MCP let the agent uh actually uh act it out. And first uh we're going to talk about skills. So, skills are really good um here are some example. You can teach uh testing patterns, complicate workflows, domain knowledge, team practices. And I think the best use case for it is uh skill is really good for procedural task. Like you probably have some complicated task like do a um performance check, and then there are multiple steps involved in this task. First, maybe do a scan. Second, how do you analyze it? Third, how do you report? Um so, for those procedural tasks skill is super important. Because if you don't give it a skill, just tell it um do the scan perform a scan for me in the app and the model is just going to guess take a best guess how to do this performance scan. And because we all know model is non-deterministic, so every time you tell it to do the same thing, it might guess a little bit differently. And the result you get is going to be a little bit different. Um yeah. >> So, what's the difference then? I guess how does it interpret the skill differently than just a saved prompt? >> Yes, um good question. So, skills um uh I think to some point uh if you have a really short skill, then they can be very similar. But sometimes skills can also skills more than just a markdown file. It's a uh folder and then you can you have a skill.markdown in there or you can also have a folder of references. So, maybe you have some really complicated skill, you need some examples that the agent can read it. So, you can have a folder for to store examples or you can also say have some script that you'll want your agent to run via the skill and they can store those scripts in the folder as well. So, it's like a more powerful holistic uh version of that. And then also skills are now triggered uh oh yeah, prompt again. Uh compared to custom uh instructions, skills are only triggered when the model figure out oh this is the right skill for the right task, I'm going to use it. But then for a custom custom instructions we talked about earlier, those are attached every single time. Um so, these are also different in that way. Um yeah. Um so, you can think of skills as reusable capabilities uh you teach your copilot to to do. Um and then maybe oh yeah. >> So, the skill that I already saved, can I do {slash} comment on it? >> Good question. Uh we don't have that feature yet. So, the question is uh for skills I have, can I {slash} command and use it? Uh we don't have it yet, uh but you can always do like use this skill and put the name in it. That'll be a very obvious for your model to figure out. But if you feel like the {slash} command is going to be very handy and useful, definitely submit a uh feature request um on the DevCom tickets and then our team will um prioritize it. >> So, we cannot create our custom slash command. >> Uh you can do it with the prompt files. That is the custom um yeah, slash command. >> So, does it show that it's using the skill the >> Yes. >> You can tell like it does it show the code by like instruction but it also shows the >> Yes, it will show that. And then I have I have a live demo later and then we'll see that uh in action uh in life. Um so, maybe for those who haven't have skills set up, you might wonder, "Oh, sounds cool, but where do I get started?" Um so, don't worry about it. Our team, we have been shipping a bunch of really cool built-in uh skills in Visual Studio for common .NET and Azure workflows. So, we have some built-in skills in VS ready for you to try out already and these are all built by the experts uh in .NET and Azure uh SDK team. And we're going to see that in the demo very soon. So, here's idea for the demo. Uh I'm going to perform the same task, one without the skill, one with the skill, and then we can see how the outcome is different. Uh for time's sake, I already ran the uh uh did the run that without the task skill. And I'm going to redo it with the skill. This is my VS. Can you all see it okay? Okay. Larger? Um let me see. Uh Is better? Yeah. So, this is my um app. It's called National Park. It's just a personal app for tracking which national parks I've already visited, which ones I haven't, and then what I'll visit next. Um it's a desktop app built with WinUI 3. Um so, the task I'm going to give it to is to scan this app and find performance improving op- improvement opportunities. Um do not use any skills and this is the result. And then later we can compare it. Um and I'm going to use the same model. I'm not going to cheat with a better model. So, here I'm using Class 1 F um 4.5 preview agent. I'm going to do the same thing um just remove the do not use any skill part. Copy this. Create a new chat thread. Put it in here, and I'm going to make sure that I have the skill turn on. So, here is the your tool picker with window, and then the this is a tab for skills. Um, I have a bunch of skills set up, um, but then I'm going to use this built-in skill, uh, which is analyzed on that performance, and it's turn on. That's perfect. Um, going to send it to my agent. So, while it's loading, a few things I want to walk through. So, on the skill, um, page, you can filter skills, search. Um, you can also create new skill from scratch using this button, this plus button. And, um, you can put in your skill name, you can select where you want it to be stored, and then, um, it'll go with Uh, okay, let's see. It's thinking. This is actually a good example of the question you were asking me, is it going to show up? Um, so, technically it's supposed to show up, and then it's reading the skill. Oh, perfect. It's doing it. It's just doing a little bit late. But, this is where you see it. It will load a message saying this skill is being used, analyzing done that performance. Um, so, the mo- usually, if you're having a pretty decent model, it should be smart enough to pick up the skill automatically, but if you feel unsure, you can always put the name for it. Um, so, it will actually use it. Also, on this, um, tool picker, I have skills set up at different levels. So, the workspace are the ones stored in my repository. User are the ones that are stored in my user profile. So, these user ones are going to show up for whatever project I have. And, these are the built-in ones we're talking about. Um, so, we have done that skills like the performance one and we also have a bunch of really cool Azure skills. And then a lot of the Azure task are very complicated. Like how do you prepare um your app ready for deployment? It involves multiple steps in there. So, these are perfect for skills. Um and then for each skill, you can turn them on and off with the toggle here. And then with this three dot button, you can open the file and then you can to edit it or you can also open it in file explorer. And while I think this is almost done with running, let's take a quick look at the the skill I'm giving it and then see if it's actually following the steps. I'm going to open this. Uh and then turn it on. Bigger. This is my .NET performance skills. Uh you all also have it on your VS. Um And basically scan the code for performance and the patterns and it does more than 50 It check more than 50 and from patterns. So, you have here it's a step-by-step workflow like the procedural task I was mentioning. The load file, detect signals. So, it does a really comprehensive scan um over your app. And then scan and report. And then lastly, uh we define a very particular format for how we want the result to be so it's easy for us uh the human to read it. I think this is also important. Sometimes when the message is unstructured, some I have ADHD so it's hard for me to focus and read such a long message. So, it's helpful that when you have this output defined, um so you can parse the information more easily. And the prioritization rules. And let's see. I think maybe the Copilot is ready. Let's see if it's actually following the skill. Yeah, great. This is exactly what we saw in the skill. Uh critical, moderate, and info info level uh and time patterns. We can take a look. From the most critical ones. Uh seems like I have something old and squared. Um performance wise this is definitely not good. And also I really love that it's getting uh the entire app so it's telling me all the instances that it find this. So there were eight instances and then tell me all the different uh lines we hit it. Uh and then also give me a suggestion um suggestion uh solution how to fix it. That's very convenient. Um and Um now let's compare it with um the threat without the scale and see what is the difference. Back. So I think it's also finding similar um issues like the link queries on every service call. Um I guess this is a really bad issue so uh both of them find it but the difference is you know here it only mentioned it. Um here it didn't really tell me all the different instances that we hit this. And it gave me a more generic fix um because I didn't really ask it to give me the full code. So it's not really blaming on the agent but just like we don't I don't give it enough prompt. Um and then I think overall it find similar um results but then um the other one is more holistic um review. >> And the format is more consistent as well. So every time you run the seal you get the same sort of result back in the same sort of format. So it's really easy to know exactly what you need to look at versus in this version it might not be laid out as neatly as you'd like it. >> Yeah exactly. That's a really good point. Uh thank you Aliyah for bringing it up. Like maybe after I fix this issues I will run the scan again right and let it uh evaluate do I have more of this issue left. If I don't have it consistently then maybe it's just guessing like throwing me different issues different time. But then if it's doing the same step one and another time then I feel more assured. Oh all the issues you find previously are covered. Um yeah, plus they're really great color. So, I hope um this side-by-side comparison show you the value of skills and make you want to try it out a little bit um after the session. So, you will you will have all the built-in ones ready, so definitely check them out here. Uh one thing you might notice while I was just scrolling scrolling around is all of these skills are turned off like the built-in skills are turned off by default. And why we're doing that is currently the .NET and Azure teams are actively evaluating the the value of these skills. Um so, they're running multiple tests on a benchmark to see if the skills are uh improving actually the quality. Because every skill added, how it works behind the scene is that the front matter for each skill, the Let me scroll up. This this section, the name and description, uh will be attached to a system message. So, the more you have, it's going to cost you all more tokens. And we're very aware of that. We don't want to waste our tokens for nothing. Um but that's why evaluations are now. Uh if you're interested, you can check it out yourself as well. For example, this is the it's on .NET GitHub IO skills. I think it will show up if you just search .NET skills. And then this is the skill we're looking at, the analyzing .NET performance, and it's comparing the quality when um doesn't have a skill, the vanilla versus isolated like had a skill, and then plugin. So, introduce a skill via plugin. So, you can be more assured, oh, are these skills actually improving my uh response quality? And then you can see here, generally, they are. Like the blue and green lines are usually above um this dashed uh vanilla line. Yep. Switch back to the slides. So, that's a lot about skills. And then MCP. We already touched on it. The idea is super simple. MCP gives your agent access to tools and the information outside the editor. So, once you have your MCP connected, you have access to docs, APIs, and data. And I think the magic really happens when you have both skills and MCP server set up. Um so, your agent is fully capable, it knows how to do a task, and it have the way to do it. So, it's a capable agent that can automate your workflow uh from one end to the other end. Like, first end is starting the work to the end of wrapping it up, maybe submitting a GitHub issue, uh pushing the PR, um all of that. So, I'm going to We're going to continue the previous demo, and then but this time we're going to add some MCP tools to it. Um and I already Let me copy my pre-written prompt. Okay. Switching back. So, now we have We found these issues. Why not we just let Copilot help us fix some of them? Uh I want uh this previous one, the run with the skill. So, here is what I'm going to tell it to do. Um this emoji got copied somehow. Um so, my code is hosted on GitHub repository, so I'm going to have it create a GitHub issue as a type of task and initiative to document all these performance finding results, and then implement a fix. And let's do the most severe one. I think that is the link um scan. Um this one. Uh I'm going to copy this one. Put it here. And then after the fix, add a comment on the GitHub issue uh about how you approach fixing that fixing that. So, how I'm going to give my agent access to my GitHub account is via MCP. So, here again this beloved two-picker icon, and I have already previously added my GitHub MCP tools. One thing is really important to check is always make sure like you're online like you authentication worked out. Sometimes I forgot to check and then it crashed. I was like, "Oh, why isn't working?" And then wasted a round of conversation. Now I've I'm connected. I'm going to send this. My agent. Um so here I'm combining the skill about how do you create a GitHub issue with the uh GitHub MCP tools. Uh as you can see here, it's reading the skill of how to create a issue. Uh and then we can also look at it um through here to read the contents of it. Um I think probably every team have their own preferences of how these bugs bugs and tasks should be managed and I think it's always good to have like a issue uh skill. So if you have you're not sure where to get started, this could be a good idea um to play with. So here in my skill, I'm telling it there are three types of issue um I want to manage, the bug, the feature request, and task and initiative. So this well fit well nicely under task and initiative and then for each of them I'm defining the style like the title, the the body, and then also the labels. Um he'll do that. And let's search back to see if there's anything he did. Oh, yeah. So here I'm getting this ask um to uh read um perform uh to ask for my permission to read and check for existing performance issues. So I'll allow it to do this time. And I'm only seeing that because I'm under the interactive mode. Uh when you're under interactive mode, uh the agent will ask always ask for a permission when you're performing some task, but if you really trust your co-pilot or you're doing some very familiar workflow that have you have done it multiple times before, uh you feel safe of handing it over, you can also do autopilot here. Yeah. While it's running, I'm just tell some quick small features that we recently introduced. Um, I think it's really cool. Um, right now, I think as you're more and more of you are onboarding to agent mode, uh, you're going to have a lot more uh, messages like explode uh, in your chat thread. Uh, I have a lot um, so now you can rename um, each of your thread and then give it a name so it's easier for you all to identify them. And I think one thing that we are also thinking to prioritize is you can pin the issues so you can find your favorite issues quicker. Oh, here um, agent is back with this request um, to create um, the issue. Oh. I think my screen >> My name is Zoom out. >> Yeah, Zoom out a little bit. >> Like it sometimes the output really long. >> I've a really long issue to create. And then going to do I'm going to do it a lot of the session because I am trusted to do um, the push to the issue. And here is how you can see it is using the MCP tool. It gives it. So this is the MCP uh, create. And now it's want to add this issue comment saying I'm doing a handoff um, um, to the agent to to fix the first task. So the MCP GitHub add issue comment. And one also quick knowledge about MCP tools um, is that though they're super powerful, you want to be aware of how many tools you are giving access to your agent. For example, here the GitHub uh, MCP that comes with 48 tools automatically. But then um, the reason that I only hand handpicked a few and not giving it all is if you give it all the tools, how it's going to work is similar to skills. Like all the meta data information about a tool like it's going to send over to our agent. So imagine if you have 40 plus tools enabled, it's going to append all of them and that's going to use up your tokens and contacts window. And then also I think there's some research out there if you give it too much tools, your agent is also going to get more confused about what you want it to do. And then sometimes it will pick the wrong tool or it will try different tools. So it's always if you know so if it's always better to fine-tune the set of tools you give to your agent if you know exactly what they're going to need. And something very neat I really like is once you select these tools, when you turn them off, turn them on, it's going to remember the ones you selected. So just uh simple small features that I find joy when I figure them out. Um great. So here the agent is done I think with fixing the performance issue. And now it's doing verifying and then build the project. So this is the thing I mentioned I really love about agent mode is it will always build and check and then fix all the errors if there is any and not let left you with like error window of like 10 plus or even more errors. And here other quick feature I will highlight is in this example because we're only changing one file so you only see one file listed here but then you can but if you're doing a more complicated change you can have like five five files or even more. And then that makes this change summary view really handy. So what it does is basically it's a multi um multi-file summary diff. So if you have multiple files, all of them are going to be loaded in one tab. So you don't have to actively back and forth between tabs. You can just manage all of them from here. So here you can collapse and open um the files you have because I only have one here so it's probably not the best um case uh but then it's going to be very handy um when you are scrolling through a bunch of changes um made by our agent. Great. I think it finished um finished and now it's pushing the final command. A comment on um how it resolved the approach. So, that's kind of like the end-to-end automation I was talking about. Like you can really hand over a task to our agent and then trust it to follow through the entire loop. And then you can do really complicated tasks when you have multiple skills. You can make them work together and do a really complex a complex um workflow. And then let's actually check the GitHub issue. So, we um are honest about how our agent is doing. So, this is my repository. Let me refresh. Here it is. Nice. So, this is the uh style of the body I wanted. Um its objective, scope, the implementation plan. Super detailed to-do list. I love it. And then here are the handoff comments it was creating while it was um completing the task. And then you can take this um to the version of your team. Like how you all love to manage the issues. How do you manage um the progress um and then make it work for your team's workflow. Okay. Now, let's come back to the slides. So, yep. So, this is the skills in MCP dual. Don't forget about them. Definitely try it out. Set it up after the session um today. And then one quick thing I want to highlight is custom agents and uh how they bring a bunch of the customized feature customization features together. MCP, skills, the custom instructions. Um instead of like asking your general assistant to do accessibility scan, uh it will be uh you can also do a design a custom agent who will just ask act at a as a domain expert in this area. So, I think the benefit of custom agents is uh you probably have those very specific tasks that requires very deep knowledge into an area. And then if you will feed all those knowledge into your keep feeding them to your general agent, I think it will do all right. But then if you have more of them, it's going to use a bunch of your context and token. But instead, you can separate my workflow. And then for those very deep knowledge, just have one expert for it. So So you will create this isolation between our agent. And then you just turn to the specific agents when you have a very specific task for it. And then you can use all the MCPs and skills you set up for it as well. Just like our general agents. And now when you are starting to set up all these customization features, you might think, "Oh, where do I store them?" One way you can think about it is who need to access this information. So if it's just for you, your own preference, your own workflow, then put it in a personal level, that will work perfectly. It will be available at every project you're working on under your user profile. Um but if it's something that your teammates might also benefit from, you can put it into your repository. But then these are going to be pushed and shared for everyone. But if you work in a really big company that has multiple repositories and you want to share them across all the repos, then you need to check out the org level organization customizations. And that's also where customization shine is that it can scale at an enterprise level with you. And that maybe will be applicable for many of you here as you work as a professional developer. So the flow is similar. You define the skills, custom instructions, the custom agents you have. And then once it's ready, you can package it and distribute across repositories. And then And then after that, every developer under that org will automatically have access and use these features by default. A quick note is that this is limited to folks who are under a GitHub organization subscription plan. Um So here are the features we have. First is org level custom instructions. So, for the GitHub admins uh who are the admin for the organization, they can set up the instructions on the box here, and then once it's saved, and then you're working on a repository under that organization, you'll see this file automatically show up when you're interacting with it. But it is it will show up as a default instruction down marker. And similarly, we have the org level custom agents. Um same same deal, the admin can create a secure private repository to save a bunch of agents for your team. And then once it's uh saved, and then we'll pull it um from the cloud, and you'll see these available uh as organization agents uh automatically appear in your agent picker list, and then you can just start using them. And we also know for enterprise uh another super important piece is safe and control and security. Um all these customization are super powerful, so they can be very bad uh if there's some ill intention customizations. Um so we uh also have all these admin control plane on the GitHub admin page where you can set policies, um allow certain tools and access uh uh for MCP servers, for example. Like we mentioned how they're powerful, they give you access to all sides, third-party uh tools, but if it's connected to a uh ill intention um server, that can be very bad at the same time. So, with this, the admin can define which are allowed and not allowed MCP servers. So, that's uh a wrap uh for now for the quick agent flow starting with how to pick your agent, and then how you can customize it to make them really work for your team, then how you can distribute them across multiple repositories uh with governed enterprise control. And I think next I'm going to hand it over to Leah. >> Yeah, thanks, Samona. So, we just talked about how you can set up all your agents and customization so that you can have Copilot generate things and work in the way that you want. But, the next step is just as important of reviewing and validating your code. So, with the review stage, here you're got a lot of code that Copilot is generating, but it's becoming the bottleneck now because with so much more code coming in, there's a lot more to review. And at the review stage, you're still going to still want to keep the same level of quality you've always had, so it's going to really slow things down. But, in Visual Studio, we've been working on adding a lot of things to help and make that review easier. So, the way I think about review, there's sort of two different stages you can think about. There's the local review stage. So, if you're still really hands-on in the code in Visual Studio and working closely to get your code to a really good stage, you're going to want to keep and make sure that code gets to the right quality level in there. But, when you get to the PR stage, there's also a lot of more scrutiny that you're going to want to apply to your code. So, a lot more things to take a look at and consider there. But, I'm going to jump into Visual Studio to show you some of the features we've got in the works for both of these flows. All right. Let's see. That's connecting. All right. So, here I'm checked out in a branch where I've been working on a fix for a bug in this National Park Tracker app. And I've already got a bunch of changes in the works in the get changes window here, as you can see. But, before I'm really confident to commit it, it would be really good to get another review. And we can really easily do that by sending these changes over to Copilot. And we can make that really easy right here in the get changes window. So, I click on this button, and it's sending everything I got here over to the GitHub Copilot code review service, which is a service we're leveraging directly from GitHub that's going to take a look over all these changes and see if there's any opportunities to improve. And that's going to be sending me back some comments that I can take a look at and choose from there if there's any that actually want to apply here or if there's any that makes sense. >> I think the local review is super convenient because sometimes I always feel nervous about pushing in command, making it public. What if I made a really obvious mistake in there? And now you got your back. A copilot is going to check it out. >> Yeah, definitely. >> Yeah. >> Yeah, and it looks like stuff. Oh, yes. Yeah, the code review button that still works for non-GitHub repos as well in this case cuz this is in Visual Studio you're able to use that on whatever code you have as well. Right. And here, yeah, it looks like the review is done. So, I'm going to go ahead and open this comments up. And let's go ahead and take a look at this first one. So, here it's noticing some duplicated code in my file here and telling me not to instead of having that to in multiple places I need to maintain, I should put it in a single helper which makes a lot of sense. So, I can go ahead and actually ask Copilot here to take this comment and generate a suggestion on it. This also works regardless of where your repository is as well. And here it's just taking that comment, looking over the entire file as well to make sure that it really does have a good handle of what the available context is. And it'll make a suggestion on what to do. Let's see. Oh, is it diff loading? Oh, no. We got it here. >> Okay. >> Yeah, we can see that it added this helper method here and it found that duplicated code that was originally in the setter method here and replaced it properly. So, I'm good with that. I'll go ahead and keep it. I do want to note also that uh other than kicking it off in get changes, I also really like being able to kick it off in the get agent here in Visual Studio. So, if you go through the Copilot chat, change your mode to the Git agent, you can go ahead and easily kick off that same review flow. It's going to call that same tool in the background to do that local code review. And bonus here is that you can also iterate on it. So, if you really want to dig in deeper and guide the Copilot to a specific direction, if it's a little bit different from the comment, then it can apply that for you as well. Let's uh If I'm happy with all my changes here, I can go ahead and commit. I'm just going to go ahead and >> I love that you see all the Copilot comments under each file. It's really easy to navigate um through with them. >> Definitely. It's really easy to take a look really quickly to see if there's anything I want to look at. So, yeah, I've generated my commit message. This is also pulling from all the custom instructions to Copilot instructions file like someone showed you earlier. So, you can get it exactly in the right format consistently every time. I'm going to go ahead and commit those. And at this point uh I didn't do another review, which it might have been good if I had a bunch of edits I did after that first review and want to double-check, are things still in a good state? I can still do that. I can still review the commit that I just made, kick that off. It's calling that exact same GitHub Copilot service as well in the background. And it'll run and take a look to see if there's any comments there. >> That's great. So, whenever you want to review the either before you submit your commit or afterwards, you have a way to start it. Yeah. >> Yeah, so you get control like you can call in Copilot wherever you are in your flow working on your code. And I'm going to switch over to a different branch, which I actually have set up in a separate work tree over here. This work tree support is recently added, so I suggest you go and try that out as well. >> Oh, could you explain a little bit to me what is a separate work tree here? >> Yeah, here uh work trees are something that we recently added to Visual Studio. It allows you to separate uh and have different parallel work streams at the same time. So, if I have different branches I want to work on, different states I want to manage at the same time, it's really helpful for that. >> Oh, so I don't have to fully save, I can just switch and then >> Yeah, so if you have like unstashed, uncommitted changes still on the branch, you can easily switch to another work tree and things will still be preserved. >> That's going to be so convenient. >> Yeah, it's really nice. Let me just uh I don't know why I'm getting all these errors. >> Error? >> It's uh oops. Let me reopen get changes cuz I want to show you what it looks like to review a pull request now. Yeah, we can see VS detected the active pull request for my current branch here and it's going to open that up. And while it's opening up, I did want to note that it'll also detect all the other active pull requests I have in my repository as well. So, if I need to take a really quick look over that, it's really easy to switch between those. And let me open this up in a bigger view. So, over here, we've got the nice familiar pull request page, really similar to what you see on the website. You can check all the status, the timeline, all the comments right from the comfort of Visual Studio. It's really easy to do that. And uh over here, I'm going to jump into one of the comments so I can kind of show you what uh flow might look like if you're iterating with Copilot as well. Since I have this in the GitHub repo, I did pull in Copilot as a reviewer on my pull request and it's able to take a look over everything, leave comments, and have me review them. And from the comfort of VS, I'm able to iterate with it as well. So, I'm able to talk back to it, ask it to perform another task for me, or take a look at another thing. I can easily see that it picked up my request and in a few minutes, it's able to come back to me with a commit of all the changes I've requested it. And I can also see that the commit is here in the pull request view as well. So, very convenient to do pretty much everything you need to do with a pull request if you're trying to either work on as an author or as a reviewer, take a look, and see how things are working. >> That's cool. No more context switching. >> Yeah, I I guess this is also another hidden gem. I think a lot of folks may maybe not aware that you can access your PR directly in VS. >> Yeah, and another thing I really like with the PR integration is being able to iterate it on the side with chat as well. Cuz if I open up Copilot chat here and I want to talk to it about a certain PR cuz I can go back to this list view here. I can choose any of the active ones I have over here and easily add that over to the chat. >> I'm sorry, it's not. >> Yeah, it's taking a bit. >> You want to maybe switch a different thread and see? >> It's just taking a while to load the entire window, but yeah, this is just a way to add the specific context of the PR to your chat, so it's okay if you don't see that. This uh just lets Copilot know exactly which PR you're looking at. It adds all the required metadata just so it knows exactly which one it is. And if you have your MCP server like what Simona showed earlier, it's even better. It gets a deeper look at all the different changes and all the comments and able to iterate you through much more closely, which can be really helpful if you're trying to work on the PR locally or if you just wanted to have a conversation with it on the side that you don't want public on the PR. All right. And those are the features that are currently available in Insiders right now if you want to try those out. But I'm going to switch back to the slides cuz I have some features that are still in the works that I want to talk about in this space. And the yeah, the Visual Studio team is really excited to be able to show these as well. Let's see if that's showing up. A minute. Right. Okay, so the first thing I wanted to talk about is really in the space of reviewing your code, so we talked a lot about like the different features you can use to review your code if you're working on changes locally as well as working on a PR. And that PR stage is something that we are really invested in in the Visual Studio team. We've got work to help make that view even easier for reviewers to understand by grouping all the changes semantically, so it's easier to know where you want to direct your attention when you're taking a look at the code. So, this one is very early stages right now, so we're excited to make more progress on that. Then, the next thing is more at the end of the review flow. So, you've got your changes reviewed, you're ready to merge your PR, but then you might get hit with merge conflicts, and that part can be really irritating. So, we've got a flow where we're trying to use uh Copilot as a agent to help you resolve those conflicts more easier, and make sure that you're confident in accepting your resolution. So, I've got a video to play from my colleague Anna will show you where we're at there. >> Hello. My name is Anna, and I am an engineer on the Visual Studio version control team. Today, I will be giving you a quick preview of something we've been working on to help make Git conflict resolution a little easier. For my setup, I have already cloned an open-source GitHub repository, and I plan to mimic an actual merge conflict that happened in this repository. In order to do that, I have opened Git changes, and I will merge this bug fix branch into main. This will open up the merge dialogue with the addition of a new auto-resolve button. I'm going to go ahead and click on that button, and when I do, if we end up detecting any conflicts, we will actually launch an work in progress Git agent to help resolve those conflicts. So, I'm going to wait for the agent, and I will come back when it's done. >> This is cool. So, the merge conflict flow is very similar to how you interact with a regular agent, but then it's designed just for this case, and you can pick the model you want. So, you can decide, you know, how difficult is this merge conflict and give it a more powerful model if it's a very big problem. >> Now that the agent is done, the agent has provided us with a resolution summary, so we can further understand the reasoning behind the changes. Additionally, you might want to dive into the actual code that demonstrates the changes that have been made. You can see them on the left-hand side with the live file, but these do have the conflict markers that makes it a little difficult to read. Luckily, we already have a merge editor that helps users visualize merge conflicts, and so we're actually going to leverage that in this scenario by clicking on the file link, which will open up our merge editor. You can tell from the merge editor that the left side or the incoming side was taken, and this has been reflected as well in the results file. Now, after reviewing, this looks good to me. I'm going to go ahead and stage conflicts for resolutions. >> This is really cool to show how VS is further integrated with the merge editor as well. >> Once conflict resolutions are staged, it's suggesting I go to open changes and they open the Git Changes window and review and commit all the changes to complete the merge. >> So, that is a overview of the entire review flow that you can do right from the comfort of Visual Studio and how you can have control at each step of the way. So, you have a lot of options for how you want to do things. And now I'm going to switch things back to Simona. >> Yeah. Um and now we're to our last section, but last but not least, super important. Like we're talking about agent, you cannot talk of you cannot like miss models. Great agents need great fuel. That means picking the right model for the right task, uh and sometimes even bringing your own models coming in uh when necessary. So, with the um GitHub Copilot subscription change um to the use base usage-based billing, uh VS has added a lot more features to help you gain more transparency and control over your usage and your spending. So, here is the context window. Um uh you can click from this little donut ring button on the top right um of your chat box. How many of you have already uh clicked on it and played around? Yeah, some of you. Great great. So, check it out uh if you haven't. Um so, here you can see all your um context window, the usage. So, the context window you can think of it as the model's active memory. So, all the knowledge, all the context your model is actually uh right now picking up and understanding. But as you talk with your agent, you're going to feed in more context and then the window is going to slowly fill up and then you'll see this and every different model have a different con- con- context window size. And you'll see this thing slowly fill up. And then when it's getting full, that's where the summarize conversation comes handy. Uh you can summarize it, compact the context window, and then keep continuing um your conversation. And then when you expand this, you'll see a very detailed breakdown of Okay, I'm really bad at using this clicker. Okay, here. I guess you have a breakdown of what is going in. Like what is your model currently using? For example, we're talking about system message and how the MCP tool definition they are being expanded here. Uh they're appended here. Remember I was talking about uh be mindful about how many tools are enabling because this number can go really big uh when you have too many tools enabled. Uh and then here the system prompts, custom instructions. So, customizations are powerful, but you have if you have too many of them that are not actually related and super long, they can take up a bunch of your context window uh and consume tokens as well. Um and then here is the user context, um but IDE messages, files. This kind of give you a very um clear view of where you're spending your context and tokens, and then you can trim it down um to your need. Another feature we have is uh the usage meter. So, actually that help you um um that manage your usage. So, you can see this Copilot usage window from two uh spaces places. One place is from this top right button here. From here, you can click and right see how many usage you have left or you can also go through uh your Copilot badge on the top right corner of Visual Studio and then there's a Copilot usage uh option there as well. And from here, uh you'll see how many uh tokens you have uh you have left. Um if you're if you're on an under enterprise plan, there's a chance that you see the message saying you have no monthly limit. Uh that basically mean all of you and your colleague are using tokens from a shared pool. And once that depleted, uh it's going to be done for um all of you and then you need to uh ask your admin for more tokens. Uh but in other case, you can you you can get assigned with a certain number of tokens and you see that meter of how many is left um and when will they reset. And you can also set a warning for yourself to see how many tokens you have left and you can be more mindful about using them moving forward. So, all of these refresh every month. And then automatically you're going to get a 75% notice when you have hit that. And then you can also customize that. It's in the settings tools options. It's under the GitHub Copilot chat. There is a place you can set how many percentage you want to get notified. And then you'll see that yellow notification bar on top of your chat window. And then when you fully used up all your tokens, you'll see this red warning have you have finished your monthly limit. But you can always upgrade your plan or like reach out to your admin about it. And next is models. So, this is ever-growing list of models. There are more and more and more of them coming to Visual Studio. Something I think we're really proud our team is recently we're able to ship models at the same speed as VS Code. Like when some of the really recent models such as the GitHub Soul, Terra, Luna, we also had it immediately once it's out. Yeah. >> Sorry, back to the other >> Oh, yes. >> This file um on the enterprise usage right now it looks like only admin can see >> Mhm. >> the user's count. >> Yeah. >> for usage. That is a big problem for us. Is there a plan to like allow somehow that to bubble down to the users? >> Mhm. Yep. That makes sense. Yeah, thank you for the feedback. Though our team don't directly work on this, but I can definitely share this with the GitHub folks. And then let them know it'll be super helpful for the each individual users also see their tokens and spending. Last one okay. So, okay, great. Okay, I'll definitely take that feedback to our GitHub friends. Yeah, thank you for that. Yeah, so back to the models page. So you're going to see more and more models available in VS. And then how to manage them is also becoming a problem. And super I think super recently we just shipped this model management view, which is super cool that allows you to favorite models you love and then pin them and you can see them in a quick view. So Let me quickly show that in VS. So here is your model picker. And you can see here only three of them are showing up for me because these are like my favorite ones. But you can always click this to expand the list. You just pin this super easily. Pin whatever you want. And then next time when you open it, you only see the ones um um you have already picked. And then you can also go to this manage model view. Which will show you all the models. What are their capabilities? How much context size they have? So this will reflect on the context window. Like how much of context are being used. And then some of the models they support different thinking effort from low to max. And then how much they're costing you as well. So and then if all of these models won't work for you, there's always option to add your own model and bring your own model to Visual Studio and use them in your agentic workflow. Yes. >> Well, yes, the first question is will auto only pick from your in models? >> No, so how the auto works is basically GitHub they have this algorithm this router behind the scene which will route your task to the model for best availability. Also for like it will detect how complicated your task is and then match that complex level to the right rank of model. So, it will just pick from all the all its pool. Yeah. >> Second question, is there a way to turn off certain models with doing >> I don't think that is available right now, but I can definitely check with the auto team on GitHub and then ask them. Maybe afterwards you can share email and I can follow up offline. But those are great questions. And auto have like a 10% discount on your token spend. Um, yep. >> Um >> Yeah, maybe we'll go this way. >> So, there's a I just follow up on that. Is there a way at the enterprise level limit They do limit that list. >> Yeah, great. Great uh great input. I'm not aware of that. Thank you for sharing. Yeah. >> Two thoughts. One is I'd love a select model skill that would drop >> Mhm. >> Probably, I don't know. >> Yeah, that's a great question. So, for custom agents, we are defining them in the description field there's a a place you can define which model it have access to. So, that's super handy. So, imagine you have a custom agent for a very simple task like managing your issues, you probably just give it the cheapest model. And then you can limit it to that model only. So, you can do the isolation and then give your complicated task to a complicated agent. >> Is this app model is this general purpose? >> So, I think this uh app model provider this feature is available for a while. So, you can see it in um the B I ask already. But then we have more more work coming up with the B I okay and I have a really cool demo video to show you next. Yeah. Question? >> Yes, about the skills because it's already having the title and the So any way either or in the CLI that prompt it'll pick up >> Yes. Yes, so models will So basically how it works is we'll send model. Here is the prompt. Here is what user is asking and here are all the skills you have available. And then model will decide which skill to pick up and then you can add in some models are better than others so they are more accurately like picking up skills. But then you you feel you're you're not trusting it you can always tell it directly use this skill and then put the name in and then for that case usually 100% it will always pick it up. What is for like Yeah, let's follow up. I'm not sure exactly where to let's talk about it. >> when you were demoing it you were explicit check box this skill. >> Oh. >> So is that needed or just recommended? >> Oh yeah, so that is when I was turning on skills only for the task because again it's really like if you have too many skills it's just going to use a bunch of token and context. So if you want to save on your tokens you can be more selective of turning on skills when it's necessary but if you are not worried about it um then you can always have them turned on. >> Yeah, the toggles are more so for making the skills available to the chat in the first place. So if you turn them off it won't ever pick those up. But if you turn it off then there is a chance that it will pick it up because it will be aware that that skill is available. >> Yeah, so that decides if we're sending the skills information to the model. Yeah. >> Did I understand correctly you said if it request for a particular model >> Uh no that's for custom agents. Yeah, not for skills yet. Um But I think you can always Yeah, I think that's only agents right now. Yeah. That's right. Okay. Um so, check out Oh, let me go back to the slides. Yeah. Um Check out the uh models view. Super cool. And then we're going to talk about BYOK as um audience asked. So, I'm going to hand it to my colleague Damay and she has this super cool demo with her. >> Many regulated organizations cannot use GitHub hosted models today due to security and governance requirements. Instead, they choose to deploy and manage approved AI models in platforms like Microsoft Foundry behind private endpoints. With the bring your own key feature for Visual Studio Copilot, developers can use those organization-approved models for their AI-assisted development workflows without requiring GitHub sign-in or GitHub hosted model access. Now, let's take a look at a common enterprise scenario. My organization already has a model deployed and approved within Microsoft Foundry, as you can see here. Now, as a dev in Visual Studio to utilize the Foundry model, I can click on manage models, add a model provider. Here I'll select Microsoft Foundry from the list of providers, sign in with my Entra credentials, and then find the corresponding Foundry resource and project that hosts the deployment that I want to add. From here, I can select the deployment that includes the graph model that we saw in my previous screen to be used within Visual Studio Copilot. All right. So, let's take a look at a real development scenario. Now, here I have a simple expense calculator solution. As you'll see, out of the 10 tests, one of them is failing. Now, before I ask Copilot to help fix this, let's head to the Solution Explorer. Here you'll see I have a set of custom instructions and a skill to help guide the agent as to how I want it to behave while fixing specific tests. For example, in the test failure debugger skill here, you'll see that I expect it to produce the output in a specific format by listing out the failing tests and write towards telling me why this fix is safe. Now let's head back to chat and I'm going to give it a prompt to actually investigate the failing tests and then fix it. Now as the agent does its thing, I also want to call out that I am not signed in with my GitHub account as you'll see here. This entire experience is powered by the Grok model that was hosted inside of Microsoft Foundry as we saw. And additionally, I also want to call out that this experience is built using the new agent preview mode which is built on top of the new Copilot CLI SDK. Awesome. Now you'll see here that the agent actually went ahead and made the fixes and using my skill, it actually listed the failing tests right to why this fix is safe. Awesome. Now let's actually go ahead and ask it to run the tests again. Awesome. You'll see here that all the 10 tests are now passing. In just a few minutes, we used an enterprise governed model hosted inside Microsoft Foundry to become more productive with AI assisted development inside Visual Studio. We worked without GitHub sign-in, used agent mode, leveraged custom instructions and skills, made code edits across multiple files, and successfully fixed a real development issue. The bring your own key feature will be available and enabled by default across all Visual Studio SKUs including community, pro, and enterprise. For the pro and Enterprise SKUs, administrators will be able to manage or disable this capability through a policy. We're excited to get this experience into the hands of our customers and see what they build with it. Thanks for watching. >> So that's the demo of BYOK and then the difference between what is going to be available soon versus what is already in VS is that the the one already in VS require you still have a GitHub Copilot subscription. So you still need to pay this monthly subscription and then you can bring your own model. But then this one doesn't require a Copilot subscription and you can just bring your model. Then you you can use um all the agent harness um for free and then just you know the cost of your own model. Uh but then a caveat is that uh with BYOK you only have I think agent available to you for now. Um the model won't work for uh other Copilot features such as um the inline suggestions like completions, next line suggestions, and things like that. Yeah. >> Next slide. So yeah, another something really cool I'd share with you all is something um about CLI to VS. How many of you are using Copilot CLI? Yes, some of you. Uh if you already have a GitHub uh subscription then you automatically get access to Copilot CLI. Personally, I really love it and then all of our colleagues will love it because um it's in the terminal. Um so it's like faster. Um so it it does not have all the UI um and all the tools you have as IDE but then it's a very convenient uh terminal window tool. And then if you are a big fan of CLI you're going to love this news because now you can easily transition between CLI and VS IDE. I'm going to hand it over to my colleague Rachel. >> AI has changed how developers work and increasingly that work spans multiple agent sessions, tools, and surfaces. So, in Visual Studio, we've been exploring how VS can fit into that workflow, no matter where you start. Here, we find ourselves in the middle of a Copilot session in the CLI. And after making some code changes, we've reached a point where it would be nice to step back and take a closer look at the code in Visual Studio. With a simple slash command, we can launch VS to pop open the IDE and transition seamlessly into Visual Studio. Typically, we would have had to launch the Visual Studio installation through a separate independent process. But here, we're able to transition seamlessly into Visual Studio with the very same session loaded up and ready to go. And we can continue the conversation from right inside Visual Studio, and even move back and forth between the CLI and Visual Studio as we wish, with their conversation progressing across both surfaces, regardless of whether we sent the last message from VS or from CLI. This enables us to have the best of both worlds, and we can work wherever best fits our workflow at any stage of our development. In Visual Studio, that means being able to easily access, debug, and review our code from any session, no matter where it starts. For example, in Visual Studio's chat, the file paths link directly to the files in Visual Studio, and that enables us to navigate and review the code files seamlessly in one place. This is just the beginning of the Visual Studio team's explorations, and we're super excited to continue investing in cross-surface workflows. >> Great. Thanks to Rachel. So, this is early exploration. Uh we don't we don't have it in VS yet, but then it's coming I'm coming to VS. Oh. English one more time, I think. Thanks. Great. So, if you enjoy this session, if you want to learn more about VS Cop- and Copilot, and we have more sessions for you to uh check out. >> Yeah, the first one on the screen here is from my colleague Sebastian, who's a developer on the Visual Studio team. He's going to walk through a bunch of the different features that the Visual Studio team uses themselves in Visual Studio to build Visual Studio. >> Yep. So, check it out. I think it's always super cool to see the behind-the-scenes how the team is using the product uh on a daily basis. And we also have a session, which is a quick 20 minutes read around by my colleague Yojun, and she is also a developer uh in the VS Copilot team, and then she's going to talk about the debugging flow. Uh so, if you missed the previous session about debugger, I want to learn more about it, this will be a great session to check out. Yep. So, thank you for staying with us, and uh we hope some of the things we showed uh excite you, make you want to try it out in VS after you um get home. Uh any questions? If not, that's all. Feel free to come over later for more questions as well. Thank you, everyone. Enjoy your session today.