Intro to Customizing Microsoft 365 Copilot with Agents

[Music] uh thank you very much for everybody who's joining me today I'm really looking forward to diving in and sharing some of the goodness about what we can do with Microsoft 365 co-pilot um specifically we're going to look at how we can do some customizations and make co-pilot uh a little bit more tailor just for us and our organizations um I like to think of this as the way that you can really get value uh out of Microsoft 365 co-pilot um we've all experienced that $30 per user per month with a an annual uh commitment um for co-pilot uh for Microsoft 365 co-pilot licenses and there's a new Option about a month or so old uh called co-pilot chat where it also supports a consumption model that allows us to um pay based on the number of pay based of U uh based on messages uh that are sent um I have an article where I walk through all that stuff with the the consumption model but um I'm going to focus mostly on just why you would do uh this different um why you would want to customize co-pilot uh and uh hopefully answer any questions that you may have um so this webinar is primarily uh targeted to full stack developers or web developers um this is not uh particularly uh for uh makers or um low code or no code uh people um however you will still learn stuff uh if you are not a developer if you are not a a full stack developer or web developer uh you will learn stuff if you are a um a maker a low code no code person if you're a business manager if you're just a decision maker or something all those things this is still going to be useful to you because it's going to I'm really not going to dive into any code here if questions come up I've got some ready that I can pop out and show you um but the whole point here is for you to walk away understand how co-pilot works and then how uh where the limitations are um out of the box and where customizations are really going to come into play uh for you to create tailored experiences that's where you're really going to get the value out of this uh so let's go ahead and get started now I want to uh first start off by just introducing myself my name is Andrew Connell um I'm a longtime uh web developer in the Microsoft 365 space I've been doing um primarily SharePoint development going back 20 years uh but in the last five plus years I've been doing a lot more work on like uh Microsoft teams development in the last year or two uh more work with SharePoint embedded um also work around I do a lot of work around SharePoint framework um and spending a lot more time too with Microsoft 365 co-pilot and specifically around not the out-of thebox story but more focused around the um the customization story of uh how we can build custom agents for Microsoft 365 co-pilot um I got a little bit of contact info here where you can find me um uh I'm on mostly active on uh YouTube LinkedIn and uh Blue Sky um also you can see there are threads in GitHub well I live in GitHub these days um but mostly it's like LinkedIn and uh blue skies where you can find me but I've also got my contact info there uh where you can also learn more uh and get in touch with me um I have a company called uh Voos where I focus primarily on uh really making you the best Microsoft 365 developer that you can be uh and I do that through on demand courses free video free email courses um articles uh webinars like this free webinars like this um workshops um group coaching programs like eight-week long uh uh application development uh accelerators for both teams and um SharePoint framework so lots of different options that are there um I'm also a I think a 20y year or 21y year um veteran of the uh Microsoft's MVP program um where I've been a MVP been recognized with their annual award every year since I think it was 2005 I think it was 2005 um it's always hard to tell like when you know how old are you or how old is somebody always messes me up uh okay so let me go quickly just kind of explain what are we going to cover uh in this webinar um I'm going to start off by just explaining you know Microsoft 365 co-pilot and understanding how it works because if you understand how it works you're going to you're going to have a better understanding of where you can hook in and change how things work or to customize it and really make it your own um we're going to talk about the two different ways uh that we can um customize and extend uh our co-pilot experience and by creating custom agents uh and there's really three main ways we can do that um three or four so one of them is by creating Uh custom instructions for our agent so that we can set the tone we can uh tell it if we want it to be funny if we want it to be uh professional and engaging not creative there's a whole bunch of stuff that we can do we can basically give it some really good instructions uh second thing uh that we can do with that is we can add capabilities to it and by adding capabilities those are things like uh giving it knowledge uh so it and I'll explain where we do that but that's essentially like let's fill its brain up with some additional stuff and what really makes this stand out from using something like chat GPT uh that we had that we have from open AI uh third the the third thing we have and part of that knowledge there's a different ways we can do that we can do that by um giving it additional uh content uh from our organization that does not live in Microsoft 365 uh we can also do it by um giving it access to or only filtering it down and scoping it down to only letting it access ACC uh specific types of content um inside of either SharePoint sites or um one drive and there's additional capabilities like crawling a site on the Internet or um uh by giving it the ability to do uh like a a code based um a code thing it's called a code interpreter or being able to generate images the other thing that we can do with this is by we can give it the the teach it how to perform certain skills uh and that is not changing its knowledge but that is going to be more teaching co-pilot how it can reach out to an external service when it we've trained it that it knows how to do something and when it deduces based on the user's uh prompt it can deduce that hey I know how to go look up inventory information or customer data and when it does that when it realizes it needs to do that it leverages the skills that we teach it to reach out to a uh an API uh to get data real time and essentially we're on the Fly we're implementing What's called the rag pattern the retrieval augmented generation pattern which that's essentially saying hey based on everything you know and all of your knowledge here's some more stuff that is relevant to what the the the user just prompted you about and use that as additional context for generating your response the way that we do all of this is we do it through things called agents so we can create a custom agent give give it its own instructions knowledge and um skills and that allows us to create create personalized or very uh specific scenario specific implementations uh of these of Microsoft 365 co-pilot so I'm going to talk about all of these different things when you would use uh certain things when you would not use them and then also we'll get into like when should I go with a declarative agent or when should I go with a custom engine agent now I'm going to focus primarily on the the story that we have uh for developers which is using the Microsoft teams toolkit to create both declarative agents and custom engine agents but with that being said there are other options we can use something called the uhu co-pilot studio uh to be able to create agents as well it can't do everything that we can do with the teams toolkit and it can do some things that we can't do with the teams tool kit so the VIN diagram between what's possible and what's not there it it has a significant of overlap but there are some things that are exclusive to each one um because this is primarily for developers though I'm going to focus really on the the um the Microsoft uh the teams toolkit uh development story with this um now when you registered for this webinar I asked you what do you want to walk away from this when you when you come to this webinar what kinds of questions do you have that you want to ask uh and you want to get answers to so the way that you can do this um the way way you can do sorry not the way you can do this some of the questions that you asked some of the questions you sent me you may see your question up here so if you registered this morning uh or um uh since like well if you registered today uh then I didn't get your question yet uh I I did this uh slide late last night um so this is kind of how like I summarize things so there are some general questions here so for example uh you asked for some use cases and business applications for Microsoft 365 co-pilot I'll do that because I've built some agents for my company and that I use that make me much more productive and so I'll give you some scenarios here um the next question somebody had were another category of questions was around governance costs and complexity considerations uh so we'll talk a little bit about that um some people asked about some very specific functionalities so creating multiple choice questions from PDFs connecting to apis and databases SharePoint integration Auto tagging and automated weekly report generation uh I would love to go into all of these the second one connecting to apis and databases we'll definitely cover that but those other three things remember this is an introductory webinar on how we can customize co-pilot so those things are more scenario specific and while I don't discount them uh it's hard to do those and in an introductory one but if you've got questions I already see one person has already posted a question in the webinars uh uh Q&A panel I will answer every single question that you have I have this webinar are scheduled for about 90 minutes I do not plan to take up the entire 90 minutes I'm I'm probably only going to talk for about 50 minutes 45 50 minutes maybe an hour um I don't have a ton of slides uh my my goal here is for you to learn something and to walk away with some questions answered so I have plenty of time for questions and I'll answer them if I see a question come in I will answer it as it comes in um and I will uh if if it's relevant to what we're talking about right then so I see the question hear about csvs and uh PDFs and stuff I'll come back to I'll come back to that when when the time is right um but I will answer your question another another category of questions agent capabilities and interactions consuming list elements uh and working with documents not sure what you mean by list elements if you mean uh from a SharePoint site I'll talk a little bit about that um agent agent communication and handoffs well handoff is something we heard about last year and it's kind of not being done uh in in co-pilot these days but uh um the agent to agent communication is more like autonomous based stuff and it's possible but it's uh it's still early on doing that and it's um I I wasn't going to go into that much detail with that today so just cards on the table uh the other question somebody has using SharePoint list data I'll talk a little bit about that as part of a limitation but then I'll show you how you can get around it and then finally customizations with and implementations so like integration with AI Azure AI services or when to use different types of agent extensibility and the differences between the teams toolkit and visual studio um uh options we're going to go through some of that stuff we're going to go through some of that it's not going to be an exhaustive teams toolkit and co-pilot Studio comparison uh to be quite Frank I'm doing a workshop on that an entire work workshop on that uh at a conference later this year uh with a colleague of mine um it's a full day workshop so I I can't and the whole thing is about co-pilot studio and teams toolkit so clearly I can't do that in just an hour of our session but uh if you're interested in that I can I can tell you more about that if you like just post a question in the chat okay little bit of a preamble here I'm going to be completely give you complete cards on the table with this this entire webinar if you're joining for this and you're trying you want to learn about what's possible and how does it work and what are my options for extending and customizing Microsoft 365 co-pilot this webinar is for you now to be completely honest at the end of this webinar I want to share with you how you can work with me to get the most out of your Microsoft 365 co-pilot Investments but you this is going to be like 95% educational I guarantee you that you will walk away and get an Roi on your time a return on your investment of your time joining me for this webinar you will get an Roi on this today so I just want to be heads up with that I am going to share something with you at the end but I don't but if you're interested in it but it's going to be done at the end right before I do questions okay with that being said let's dive in let's talk about how Microsoft moft 365 co-pilot works so first of all what is it now you've probably seen a picture like this on the right Microsoft has revised this picture in some of the more recent uh presentations they've done personally I think the new one is a little bit harder to understand but what co-pilot is is it's an AI uh powered productivity tool that's going to help your users stay in the flow of their work across all Microsoft 365 applications and that includes things like Outlook Microsoft teams word Etc and it does it by grounding your prompts and the its responses but not the prompts but grounding its responses with your organizational data and what they Microsoft refers to it as Microsoft graph data you'll understand why they say that in a minute but essentially the difference between using Microsoft 365 co-pilot and using something like chat GPT is that chat GPT has a wealth of knowledge that it has accumulated from all of its training and by the underlying models it uses from from the under from all the training it's done when the models were actually trained um when it comes but that's about it when it comes to Microsoft 365 co-pilot it also has all of that knowledge because it's using the same models that chat GPT uses but the thing that it also has is out of the box it also has additional Knowledge from your organizational data inside of your Microsoft 365 tenant that includes chat messages inside of teams emails uh calendar or sorry uh meetings uh transcripts for meetings files and and SharePoint and and uh one drive Etc now there are asteris and like uh tangents to each one of those things I just mentioned we'll get through some of that stuff today uh but just know that that's like the additional stuff that it has it gives it's basically think of it using co-pilot or chat GPT but also having my domain expertise and knowledge of my organization um so it offers like I said here in the second bullet point it offers knowledge such as emails chats Etc the big thing that it does though is by having access to all that content is that it can then summarize uh and um uh present in a in an easier way to understand uh information related to The Prompt from its underlying knowledge plus all of its knowledge that that it has from your content in Microsoft 365 so over here on the right hand side this is kind of like what co- how co-pilot looks and you're going to see a picture similar to this uh towards the end of my presentation because it's going to really show you where you can go through and have make some changes so again let me just make sure that we're clear on how this picture is set up on this the bottom part right here when you hear Microsoft talk about foundational models these are the open AI based models that chat uh that chat gbd and uh Microsoft co-pilot are founded on so these are the large language models that have a bunch of underlying knowledge of from being from crawling the internet uh that it can use to rationalize overall of this content then you also it also has access to all of this additional knowledge so this is going to be out of the box data coming from Microsoft graph data which is going to include all of the stuff that you have that graph has access to like transcripts from meetings emails files and and one drive and SharePoint uh Etc it's also going to have access to all the content in data verse so all of your power apps uh that you've invested in um you're going to have access to that and data like from like Dynamics 365 sales uh is also going to be available to us uh in addition it can also have access to the to the internet and you'll see how we can customize that and then finally we also have we have all these skills and these are things that we can teach co-pilot to not only just be able to summarize content like up here we're just summarizing we're not able to act on it we just have read access to it whereas these skills have not only read access to maybe large data stores like an RP or an inventory or or a CRM system that doesn't live in 365 but it also is going to give us the ability uh to even perform actions on those different um uh services or those different endpoints so all of this is controlled using this thing called the orchestrator so the orchestrator is the thing that gets the prompt from the user from the user experience it rationalizes over that trying to figure out what is the intent of this prompt what is the user asking and it then is going to go collect some uh data from its knowledge sources and then it's also if it deduces that hey the intent of this prompt is to go get data from an inventory system I can go reach out and go get data from that inventory system and add that as some of my additional content so it's going to take all of the knowledge that it already has plus additional data that it's accumulated from these real-time services and then it's going to send it down to the foundational model which has all of the foundational data rationalize over the response and then and then um or over all that data generate a response and then return that back to the user all right that's the gist of how this works now I just jumped two slides no just one slide okay so the challenge that we have with everything I just showed you is that while it's great out of the box copilot only has access to all the content inside of your Microsoft 365 tenant that's what graph has access to show me an organization today that has 100% of their business inside of Microsoft 365 and I'm sure somebody can show me that but they're very very few and far between and I'd argue like is it are you really going to get value out of Microsoft 365 co-pilot uh and spending that $30 per user per month license to get access just to rationalize over all of your data in Microsoft 365 well you you could but think about all of the other sources of data that you have in your organization Maybe maybe policies that are not in SharePoint maybe transcripts from courses you sell like I do maybe uh an Erp system uh for inventory management and supply chain uh CRM system for customer management and relationship management uh maybe um uh other uh sales information financial information this stuff doesn't live traditionally in 365 and so maybe you want to be able to give co-pilot access to all of that information so that it has a full picture of your company and all the data so when someone asks it a question it can have full access to all of that or I can create a scoped thing where it's just a a um like a customer or a lead co-pilot that I can ask it questions about potential leads that I'm working so the way that this works is that I can add my own knowledge I can add my own skills to this I can also give it special instructions so that it can only do specific things this is what co-pilot gets out of the box right now the way way I customize it is I'm going to go create my own agent which is still going to have all of a lot of all of the same stuff but my other agent is also going to have additional knowledge and additional skills and maybe there's like a nice little veneer over my other agent over here that co-pilot's talking to so this out of the boox co-pilot is going to talk to my custom co uh agent uh and it's going to be able to respond uh based on the instructions that I've told it or like you're only interested in speaking about these specific things like I can't change the system or the base instruction set that the outof thebox co-pilot has I can just create my own agent and then co-pilot will use my agent all right so that's the gist of how the extensibility stuff works oh boy all right now this is how co-pilot under the covers works so how does this work so follow along with me here okay up here at the top in step one here what co-pilot does when a user goes to in the the business chat they're going to send a prompt to co-pilot and ask it a question or say something and then in step two you see right here step two co-pilot is then going to pre-process the prompt that the User submitted through an approach called grounding and what that does is that's going to improve the specificity of the prompt so that you can get answers that are more relevant and actionable to the task that you're trying to perform so the prompt can include text from input files or from content discovered by co-pilot and what co-pilot then does is it's then going to send this prompt to the large language model for processing co-pilot only has access to the data that the current individual user that submitted the a submitted the prompt that they have access to right so it's all based on your existing Microsoft 365 ro-based uh access control then in step three and you'll notice here in the picture that step three is not only coming back here and responding from getting all the data from co-pilot and then sending it back in addition you also see it's getting data from maybe potential plugins or actions uh and also from the um from search is then going to send all of that over to uh The Prompt over to the large language model all right and have it go through and rationalize over some of that stuff in step three this is where it's taking the response from the large language model and then it's going to do some additional postprocessing of that data um and then this postprocessing piece which you see like listed right here this is going to include additional grounding calls to Microsoft graph and do some things like responsible AI checks to make sure that I'm not saying something or that copilot's not not being asked to do something that is not considered responsible like maybe hate speech or self harm or violence or something like that um it maybe is doing other things like Security checks and making sure you allowed to do what you're asking me to do uh it's going to do maybe compliance maybe privacy reviews and also some additional uh some additional stuff that it's going to do it behind the scenes so after step three once it goes through and it's checking for additional data based on what uh Copa responded so step three is still going back to graph and getting more data that it needs it then in step four that you see is going to um come back over here to our large language model it's going to return that response to the app where the user can then review and assess the response and then that's where the user is going to get all of their um get the response back from uh the co-pilot um uh uh gen cont content generation or um uh when it generates a response is what I'm trying to say took me a second um okay so that's just the gist of it the thing is that I want to do as a developer is I want to over here in this piece where I'm going through and adding in my own stuff or maybe adding in my own data when it's going through and fetching this additional data I want to control that I want to be able to give it different data sources not just stuck with Bing or dataverse or uh Microsoft fabric the the entire data platform I want to give it some other custom stuff that it doesn't know about out of the box that's what this is all going to be about today okay so one thing you need to understand let's think about Microsoft graph for a second you might have heard of this thing called the semantic index so what is this when I when I said previously that copilot has access to all of your graph data it does but it the more specifically is what I want to say is is that it's accessing a semantic index that is sitting on top of Microsoft graph and what that is is a vectorized database of all of your content now there's actually two of those there's one for your organizational data and then there's one for your data so there's like an index that is has individual uh Communications and stuff like that that I can only see the stuff that I've been involved in I can't see everything else I can only see chat messages that I've been involved in with somebody else but I can't see when like two of you go have a chat message that's going to be in the user index but it's not going to be something that I can see uh unless I've been given permission to it so these two indexes these two things the tenant based index and the user index they sit on top of Microsoft graph and uh they now are going to give co-pilot additional uh access to content inside of Microsoft 365 above and beyond its foundational knowledge this this first bullet point you see uh right here this large language model Knowledge from from the initial training it had um so then it also has access to all the data inside a graph via these this uh semantic index and then as I've talked about before it's also got access to uh apps inside of Microsoft 365 and uh El has access to uh content um on the internet so this is going to help you interpret our user queries and produce more contextual relevant responses and it starts with prompts in the Microsoft 3 65 co-pilot uh experience or what's called bis chat with the uh data from Microsoft graph that's used for grounding which basically says when I say grounding if you're not familiar with that that's that's me basically saying hey when you respond I want you to respond in context to this right to to um uh uh to this context here so like for example if I said uh if I asked you a question of uh you know what's for dinner tonight um each of you are going to give me a different response because you're grounded on what your plans are this evening with your family or spouse or um significant other whatever or solo right so it's all that all that grounding is all your context that's what grounding is all right um and it does this by searching through billions of vectors which are just mathematical representations of phrases uh to figure out like this phrase is closer to this one so vectors if you're not familiar with this or you heard this concept of uh embeddings and uh in vectors in Vector databases an embedding is taking some text and generating a mathematical representation of it uh where basically you can think of like I'm going to take this block of text and I'm going to put this block of text on like a three or a three-dimensional like uh chart and then if I take some other text it's going to give me a new mathematical representation and that's going to put it on the same chart but it's going to help me see like where are they related so for example if I did if I generated embedding uh for the word dog and for the word cat those two are going to be somewhat close together they're both domestic pets they're both um they're both animals um but there's some differences between them now if I also had an embedding for a puppy and an embedding for kitten well puppy is going to be closer to dog than it is cat but it's also going to be closer to kitten than it is um than it is cat right maybe not as close as it is to dog but these mathematical representations the all of these embeddings we store them inside of what's called a vector database and then we can run prompts against this to say like here's some other text I'm looking for and I'm looking for Content that is related or closely related or based on whatever the query is how much related or how closely related based on the response that I get back that's a fundamental um AI concept is this whole embeddings and uh and vectors okay so that's just a little bit of like a I like grounding right and getting our and understanding how co-pilot works and just understanding how how all this stuff uh how like just the gist of uh doing queries with AI works so now let me talk a little bit about what an agent is now an agent is the phrase that we're that's being used across the industry um not just a Microsoft thing uh in fact Microsoft used to call these things something different but they have settled on the word they used to call them just separate co-pilot so if you remember about this time last year or summer of 2024 uh Microsoft was talking more about creating your own custom co-pilots well the industry now says no you're really creating an agent and an agent is what's doing all of the work so what Microsoft used to call like individual co-pilots now we now they're calling them agents and what the way it works is in Microsoft 365 co-pilot it is going to call agents so it is has its out of thebox experience and then for us to create a custom experience we're going to create an agent or lots of Agents so an agent is a couple things it is a scoped version of the outof thebox co-pilot I have a co-pilot for working with leads I have a co-pilot for working with students I have a co-pilot for um answering questions from my students uh where I have knowledge that I've created a knowledge blob inside of my tenant of all of the transcripts from my um uh my courses on the SharePoint framework and then it also has all of the content uh that I've written in the transcripts from my YouTube videos that are published on my YouTube channel and uh content on my on my uh my my uh like the articles I published on voan uh so that when a student ask me a question when I ask my when I ask co-pilot to use my student like you know um uh my my uh it's called ask AC um just I go by the initials AC so Andrew Connell go figure um when I ask AC I ask my agent a question or I take my customer's question my students question and I plug it into that agent what that agent does is it's going to basically I've told it to not be creative to only look at the knowledge that it already has don't look out on the internet and it's essentially saying look at all the body of work that I've produced over the last X number of years and come up with an answer and oh yeah best thing about it is is that because all of that data is well tagged it's going to respond with you know this is how a web part works and to get it to work as a single page app here's what you have to do by the way I talked about it in lesson number five of my course at around two minutes because it knows that from the transcript so that's how I can go through I can give that more uh tailored information to my students right now I get a response back from co-pilot and I usually go through and I modify it a bit uh so it's not all machine generation and I make sure that I correct anything that's there but it sure makes it easy for me to go through and figure out like I know I've talked about this in my course where did I talk about it let a machine help me do that a little bit still all my information um the idea behind this too is that this is designed to automate and execute some very specific uh business processes on my behalf exactly what I just described right so real world scenario it's going out and looking at all my content that I've given it to go through and do and to doce this stuff or as another example maybe I want you know I have I'm a classic virtual business I have a email-based marketing system I have a a learning management system where my students uh can both buy my courses and then watch the courses that they have access to I have a Community where people can ask questions um I have a CRM where I use to uh track customers and track leads and uh and do a lot of my sales um and then I have um all my payment processing information from V various data stores like stripe and PayPal and you name it I have another agent where instead of me having to go to all these places to figure out what what do I know about a particular customer um I can just plug in an email address and the agent knows oh here's all these places that I I can go find customers and it goes out and calls all these different apis and pulls it all in and then sends it back to the large language model for processing and not going through and getting creative and just saying hey this is a student uh this is this is somebody who hasn't bought anything but they register for some of your free courses they've asked questions in your community uh they've um dropped comments on uh on your YouTube channel for this video and so I know like okay John Doe they're interested in co-pilot extensibility and they used to be interested in teams development and SharePoint development yeah that's a life cycle I'm familiar with so that that's the kind of stuff that it's helping us do now where do you use these agents there's two different ways of doing this there's an an in context experience where you at mention the agent in the chat experience to interact with it in a conversation think of this as just a bot an old school bot that we're used to working with inside of Microsoft teams so this agent really I'm going to at mention the agent and that's how I bring it into the conversation and it's going to get access to what I ask it when I send it when I mention it in in the conversation it treats that as my prompt second of all there's another way you can do this and this is an immersive experience and this is think about this is like a one-on-one chat where I'm just having a private chat conversation with the bot right that's what it is with an agent I'm having an immersive experience where everything I type isn't going to co-pilot it's going to the agent itself and it's only going to work with that one agent so when I give it an email address my custom agent knows I'm looking for a I'm looking for data from a customer it's not going out everywhere else and saying you can send an email to this person like no no no no no I'm I'm doing research I'm trying to figure out who this person is now there are two different types of agents that we can create we can create declarative agents and custom engine agents I'm going to talk about both of those but before I do that we need to talk a little bit more about what is the extensibility uh landscape and what do I have available to me um as a developer so one of the things that I can do here when I go to extend co-pilot um uh just saw somebody's question here just see uh let me come back to that piece in just a second uh same person asked a question earlier this one's about organizing or labeling data so let's talk here a little bit about how do I extend uh co-pilot so the way it works is that if you look at my picture up here up here in the top this is the user and what the user does is they're going to ask a question or they're going to prompt co-pilot so what does co-pilot do well again it's got all of its you know out of the box stuff that it has access to the ability to to use actions we used to call those plugins um hence the puzzle piece um and then it also has access to all of the like the foundational learning uh training that it has and uh data inside Microsoft graph uh through the semantic index and other knowledge that we create or that we add to it so what we can do as a developer there's two main things things that we can do one is that if we have like static content right so articles or transcripts from courses or policy manuals and procedures and stuff standard operating procedures we can take that content and we can give it to Microsoft graph and graph will then index that content into the semantic index so that now when co-pilot goes to ask a question my agent is going to be able to say I only am interested in these data sources so your foundational knowledge plus these different these are called connectors so I'm going to add in my own Microsoft graph connector of an additional knowledge source so here's more data that I'm teaching you all right notice one thing with this this is a oneway action that is going into the index so this is not a read right thing I am S I have to create a process myself as a developer I have to create a process where I'm going to load that content into the connector and then I have to wait for a certain amount of time for Microsoft graphs indexing process to um vectorize or create embeddings and vectorize that data and then store it inside of our Vector database inside of the um uh semantic index all right um the other thing that we have uh is um we have these things we have um uh let's say for example I also am working in an environment where I have additional uh systems in my organiz ganization so these are the like the Erp CRM Salesforce or sorry uh sales and marketing could be Salesforce uh Pro Payment Processing you name it so I basically have these databases right and what I want to do is I want to give co-pilot access to some of that data so notice here this is a read write operation so notice the arrows are going in both directions what that does is I'm simply teaching co-pilot or teaching my agent you can do these things and when you need to do those things these are the apis that you can call that map to those specific tasks so if I say I'm looking for information that John do or I'm looking for um sales information from John Doe well it knows well you're asking for John Doe so I got to figure out who what his email address is so I'm going to go to a search against the CRM system or the email um Marketing System to figure out who is John Doe and so it's going to grab one or a couple matching email addresses and then it's going to go back and run uh go to one of my other um actions that it knows about to say I'm looking for sales data so I'm going to go to PayPal's rest API and and uh Stripes rest API to go find information about what they uh what they have access to so these are this is the way these are the two different types of things that we can create now I see uh shine had a question here uh saying that when he was working with uh co-pilot Studio he wasn't he didn't see graph connector as one of our knowledge areas uh to use and if that's available now or if it's coming I don't think that that is something that we have access to inside of co-pilot studio um I don't know what their plan is on adding it this is one of those things that maybe outside of that ven diagram and is only available to us over in um uh over inside of a uh a declarative agent uh furthermore uh this is one of those things that um let me go through I'm just going to give you a thumbs up on this just give you a thumbs up showing that I've answered it uh and then uh this may be one of those as well uh where there's a way to go through and and to create a connector through the Microsoft admin Center I'll talk about that in just a bit and that's going to allow us when we do that that's going to make it available for also for search inside of um Microsoft 365 which may also automatically make it available to co-pilot but just keep in mind there's I don't think that we have a way to create a custom agent in copilot studio and say I only want that data it just has gets access to either all of that data or none of it and when I say all of it I mean like if I create like three connectors then it's all of the outof thebox data plus the semantic index plus all of these other connectors does it get access to all of that stuff I'm not entirely sure with co-pilot Studio there's things that copilot Studio does do that are helpful but you really get more uh power and control when you're working with the teams toolkit with the teams toolkit I do get to build uh connectors and we'll talk more about that in just a bit okay um so someone uh see uh tals asked uh apart from read WR when do you use a graph connector to extend data versus action to an API great question Tom all I'm going to get to that so what I want to do now is let's actually dive in to each of these different things that we can build and let's look at like some what makes the most sense okay so when it comes to connectors there are two types of connectors um you have Power Platform based connectors and we're and you you might be familiar with those based on uh stuff that you've done with like um uh power apps or with um uh Power automate if you're using co-pilot Studio you have access to all of those Power Platform connectors now when it comes to Microsoft graph connectors this is the thing I'm going to focus more on going forward the reason why is because those Power Platform connectors they it does give us real-time data to get to outside uh uh sources um that are outside realtime sources but and while they're available in copal studio they're not available in our custom agents that we that we want to create so I can't use these in agents that I create with the teams toolkit graph connectors though I can so that's what I want to focus on for a second these graph connectors they add organ they allow us to add organizational grounding data to the semantic index in our tenant so it gives co-pilot the ability to access and summarize diverse data sets from multiple different data sources not just the data sour sources that we get out of the box in an out of the box co-pilot experience so this way I can use co-pilot as a research Aid as I've described already and looking when my customers ask questions um about something with the SharePoint framework or teams development two things I teach on a lot um it also makes data that does not live inside of my Microsoft 365 tenant it makes it accessible to co-pilot because out of the box if I don't create an agent and I don't create a connector that data that lives outside of of Microsoft 360 5 and outside of one drive and SharePoint it's not going to be available to me it's not going to be available to me this again it just adds another source uh essentially to Microsoft graphs semantic index there are a bunch of pre-built connectors that we have in um Microsoft 365 in the admin Center uh where you can create an instance of one of these and then connect it through the web interface uh it's under the search and intelligence admin Center um and when you do that uh it's allowing us to kind of like have a more WI wig kind of approach to go expose some of that data um so that's definitely something that you could do but if you don't have one that already exist or I want to shove data into it that's something you would do the these things also can be set up to also uh crawl data from that external Service uh on a on a specific schedule if you're familiar with something called the the um oh God what was it called the BDC the business data connector uh that we had in m in sh in uh SharePoint uh years ago uh that's similar to what these connectors are okay so a graph connector what is it ideal for so to this is going back to your question what is an idea what is a good example of this these are really good for long form SL unstructured content so articles policy manuals not things that are relational uh it's also good for Content that is not frequently changing because I have to push the content up to uh Microsoft 365 up to the graph connector to get indexed um it's also really good for when I don't like when I have unpredictable prompts when I don't know what the person's going to what the users are going to ask a question for c one of the things that the large language model do a really good job of is like looking at vast amounts of data and then synthesizing it and condensing it down in addition it also can summarize content so or it's also good for Content you want to summarize so these are all really good examples things like document repositories so maybe I have content that doesn't live in inside of one driver in SharePoint maybe if you have a knowledge base or content on a website or a Content management system like a web a web content management system uh my case transcriptions for courses uh video based courses that I have um I have email courses so the content from those is also uh inside of my uh inside of different graph connectors so a bunch of different options uh with that now one of the things that's neat about these graph connectors I think that was the only thing I was going to say on it yeah so the other thing that's nice is graph connector so going back to I can't tell what your name is uh based on this you asked a question about 10 minutes ago um you said can we organize or label so I think it's Anja 24 uh you say can we organize or label that uh data so that we can turn on or turn off access to dat within the semantic index and the Trant index the answer is yes the way you do this is that when you create the um the process of creating a graph connector what you end up doing is you you create a schema inside of graph and part of that schema has a bunch of properties and then there are also some well-known properties that you can say like hey this property actually maps to this well-known property and that's what helps co-pilot um like use common common uh responses so like it always wants to know what a title is and I may not have a property on my data set called title it may be name or it may be like product name or something so what I do is I create a property on my schema called Product name and then I map it to a a well-known label called title so that when co-pilot synthesizes its data and it needs to reference it it'll use that uh that label um I think they're called semantic labels I forget off top of my head but as part of my process for when I gra when I take that external content and I give it to graph one of the things I can give it is not just the data the raw content not just the metadata the properties that should map that I've created in the schema but but I can also give it an Ackle an access control list and what that does it says who does and does not have access to this content um that's how I can security trim uh content out and keep people from actually accessing it uh the other thing that I can do with this too which is actually pretty cool is I can even Define um um activities so let's say for example I have a standard operating procedure and in my system my external system where people use that that content maybe it or better better example here because this is how this is what I'm working on right now in my course um a web hook gets not uh will be emitted when someone watches a lesson from one of my courses so I have a course on the SharePoint framework it's like 41 hours long about 200 different video lessons it's a lot of content what's the most popular uh stuff that's in there well if it's if it's content that where I talk about a similar thing a couple different times but one video is more popular than others what I would like to do is give that as a signal to to the index to let it know like this piece of content is more popular than that piece of content which is a more gives it more relevance so what I can do is that I can periodically uh update the semantic index and tell it different activities when things are either um accessed they're read or they're shared or they're updated and those are different signals that I can pass in that the semantic index has access to but also like I said and to answer your question I can um also go in and I can security Trim in or out that data now that's on the that's on the the um the connector side okay when it comes to the and because it's all done this is all done with code and and you're importing it uh and using the Microsoft graphs rest API to do that you have control over all that content and where it's all going to go um it's different though when we come to actions so let me come back to actions in just a minute so I think I answered your question if I didn't then please come back and ask it again oh the other one you had another question here you said you want to import CS v files connecting to a database as well as importing large PDFs uh to create multiple choice questions so ah you're the one that asked that question uh and are there different data size limitations yes there are so here's a great it's a great question uh you know what I'm going to come back to let me give you some of this right now so you're when the question you asked there about uh the work accessing uh csvs yeah you can totally do that because csvs are one of the outof thebox connectors that you can configure in a uh in the Microsoft admin Center or the search and intelligence admin Center uh so one yes you could definitely get that now do you have the ability to go through and do like security trimming and stuff like that and add akles not really because you're just using the out of the box stuff if you write your own which is actually very easy to do uh then yeah you can totally do that um the other part here where you talk about creating like uploading these 150 page PDFs and their data size limitations absolutely now these are always changing so I don't want to quote you any numbers and have you say that you know take these to the bank um I there is it is documented in the Microsoft uh documentation um I just don't remember the most the most recent numbers but think of it like this if you have a very big file and you it is inside of a document library in SharePoint or inside of one drive or a very big file that you then give to uh the semantic index so what you're going to do with that large PDF uh let's say it's a 1 gig PDF can I take all of the content from that 1 gig PDF and can of just shove it into the semantic index yes and the semantic index will also take do the exact same thing when it crawls content from one drive and from SharePoint document libraries but it doesn't crawl all the content it only crawls some of the first part of the content so for example there was a time when I think it was like if you had 150 page uh document only the first 20 or 30 pages was going to be indexed so you're missing out on a lot of additional content when you do that out of the box right but there are other options like for example if it was really important that all that content got indexed and here's what you could do what you can do is you can take that data and you can you can embed it create the embeddings yourself and store that in a vectorized database that's called Azure AI search that's one of the Azure AI services that are available to us and then when you create your agent your agent agent can then either use the rest API for that you create to go get data from Azure AI search or you can just and and then you can query that to pull that data back and however much data you have in there I mean you just use more resources but because it's consumption based you just spend more on Azure AI search and now you have access to all of that content and you can export that you can then get the responses by if there's a prompt you embed you create an embedding from the prompt you pass that embedding as a query over to aurei search it looks at these big uh files that you've uploaded big blobs of content that you've uh imploded imploded imported into Azure AI search so now you've got this big you have a database that has indexed everything it's not in the semantic index but it's in the Azure AI search index has indexed everything I can query that and take the results I get back and hand those that as part of my um uh what my plug-in is going to do or my my is going to do from one of the skills that we're going to look at in just a minute to give me additional data that is something that we can do with co-pilot Studio where co-pilot Studio does allow me to connect to an Azure AI uh search um uh implementation uh a resource that's been created one I've generated for this uh so that is something that you can do it is if you just use the out of thebox stuff with the semantic index it only indexes a certain amount of content though right and so that's a big blind spot that a lot of people don't have with this when you just stick with the cop out of the box experience you can run into challenges with this so um but this is what this is where you really get the value out of this because you can if you have like large PDFs or like I've got large transcripts from videos that are two hours long um is all of that going to fit in the in the index I got to go look at the numbers again right but don't assume that if you have a large file don't assume it's all going to get indexed right the number that sticks in my head is like the first 20 or 30 pages are only indexed in a 300 page page or 150 page uh Word document or PDF right it's that's what sticks in my head but again don't quote me on that number because I know there was updates uh at that they announced that ignite from changes and Microsoft is trying to get these numbers bigger and bigger it's like um they're always working to go through and improve on this so with that being said all right let's talk a little bit now about the other option we have and that is working with actions now what these do is these allow user uh this enables the user prompts and interactivity interactivity that we have this allows us to do more uh with external systems all through a natural language um interaction so what you end up doing is you give co-pilot access to realtime structur data so these are the Erp systems and inventory and sales and uh CRM systems large data stores large relational data stores right and I want to not only give it access to that content but I also want to give it the ability to perform actions on that content I'll explain how I do that in just a minute so what it also does um is that this is going to allow us to have something that use it that has multiple actions uh to um or multiple agents sorry one agent can then talk to multiple different uh endpoints as I mentioned earlier uh based on what I've asked it to do and what it's been trained to do um it then once it gets its results back it can then summarize its response based on all the data that got back because the data gets from a semantic index is just like it's basically here's all the stuff that I've learned as part of my training I need more so I go to the semantic index and I I I query it for additional stuff from um my organizational data that's in the semantic index which is a which is what is basically Microsoft graph and then I'm like and you need more data so I'm going to go out to some real-time sources like inventory or sales or CRM and I'm going to get more responses back and now that's an addition data source and I give all that back to the large language model and it knows how to rationalize summarize and synthesize uh all of the all of it its response based on all of that data I've provided it furthermore it also lets me act on data so if I tell it that it knows how to do something like for example in my case remember I mentioned I have all these different systems four or five different systems with uh different customer data in each one well when I'm in my CRM system I would like to be able to jump over to my sales system or I would like to be able to jump over to my learning management system to see what courses somebody has watched uh or what they have access to um or i' like to go look at my sales system and see that data so what do I do well it's a little bit of a mess today on I'll be honest but what I have is on each one of these on some of these systems I have the ability to add custom properties and so inside of my learning management system I have like three different URLs for how do I get to this customer in my CRM into my email system system and into their payment processing stuff on PayPal or stripe or whatever over in my CRM system it's how do I get to them in email or payment or over into my LMS so what I do is that when I find when I ask co-pilot hey uh or I ask my agent my my uh uh lead and customer agent I give it an email address and I'm like what do I know what about know at John do and it pulls all these different things back and it's like ah we got an email looks like this we got a name looks like this these are coming from CRM these are coming from here and what I'll do is I'll kind of look at all of them these tiles that all show up on the um that all show up in my app or in the agent and I then check specific ones and I'm like these are all the same people and what the agent then does is it goes to each one of those different endpoints and it updates all of those URL properties uh and syncs them all essentially syncs them all together so that they all I now know whichever system I'm in it can pull them together it also makes it easier for it in the future because I've given it instructions to when you go look for somebody and you find who that person is now that's their URL inside of like CRM go look for that same URL on that property on these other systems and without without trying to do some lookup make a very specific like wear Clause instead of like you know a like kind of condition okay all right we've been going for about an hour here got I got a couple more things I want to get through and share with you on this so uh I I talked about uh these actions for Microsoft 365 co-pilot when do these make sense these are good compare this to to connectors earlier these are good when I need to get access to realtime data so I don't have to index it this content is never indexed and ingested into the semantic index I get it real time it's also good for frequently changing data relational data and large large scale data sets so for example like the question I had earlier 150 p uh page PDF or a 10 gig file so give you some examp examp at the bottom here what's cool about this too is that the these skills that I'm defining it this is where it allows me to really open up access over to uh stuff that's inside of different Azure AI Services I could use Azure AI document intelligence I could use Azure AI search uh and there's a whole bunch of other services that Microsoft provides as part of um Azure Ai and the new Azure AI Foundry uh like ecosystem uh thing that we have there so this is where stuff gets really cool with these different actions of things that I can do and all these are just rest in points that we're going to essentially advertise over to um uh advertise over to co-pilot now those are the two things we can build connectors and agents sorry connectors and actions so connectors are I'm giving you more knowledge actions are I'm teaching you skills to get data real time how do I use these I do this through an agent now I do this through an agent there's two tyes of Agents we can create one type is a declarative agent now what this is is really just a series of yaml files and Json files to describe my API endpoints uh they could be secured endpoints they could be anonymous uh it's also going to help me describe uh what um actions are available to me and what they can do from those different endpoints uh it's going to allow me to Define um uh the instructions that I want uh the agent to have it's going to allow me to Define the uh different capabilities that it can do in other words what knowledge sources it has access to Etc these things are always going to use the same orchestrator foundational models and the different Azure Azure AI sorry in the different AI services that power the out of the box Microsoft 365 co-pilot and business chat so all we're doing is we're basically just customizing and either giving it a scope with instructions or we're giving it access to just specific data or specific real-time skills this allows us to give our users a custom experience without the need to write a bunch of code to do this furthermore back to the question I had earlier about like the the um permissions and stuff I explained how you can give akles to the um uh to the uh Microsoft graph uh connectors when you're uploading content I can do not the same thing when it comes to actions but I'm going to know who the user is when I run the when I send the the the prompt over to when copilot sends the prompt over to my rest API to go get data from my CRM system and in the code I can figure out does Andrew have access to look at all customers or just the customers that he's worked with and I would only return those back so basically can Implement my own security trimming of what I do and don't have access to um these can uh these agents can be created and used privately in my organization or I can create these and publish these for anybody else to use them and that's what Microsoft is doing now creating these like sales agents and uh a meeting agent they're creating these public ones and they're making them available to us to show you what's possible the end users can see these declarative agents on the right side of Microsoft 365 co-pilot both inside of like office.com chat and inside of the business chat experience uh if you have the Microsoft 365 co-pilot so the intended use for these declarative agents this is a custom chat experience all right now what do I need to do this in order to use a declarative agent the person building it and deploying it and using it or other people who are going to use it they have to have a Microsoft 365 co-pilot license that is the $30 per user per month uh US Dollars um in uh with an annual commitment okay now the other thing that we can do is well what if I don't want to leverage all of Microsoft stuff and maybe I want to use a different model for example the co the large language model that Microsoft uses today is the um uh the uh GPT 40 what if what if I didn't want to use 40 what if I'm writing content and what if I want something that's a little more suited for long form articles right well that the Claud models from the anthropic uh from anthropic are a lot more suited for that or maybe there's models that are better for generating images and generating like infographics I can't change those I can't change the foundational models when it comes to uh Microsoft 365 co-pilot out of the box and I can't change it when it comes to declarative agents because I'm still using the same orchestrator the same um what was I going to say the same orchestrator the same uh uh foundational model the same Azure AI like Security Services and and content safety and all that stuff but with a custom engine agent that's me basically saying I just want the chat experience I'm going to implement this myself and instead of paying Microsoft a license for co-pilot I'm going to incur the cost of running all of my own Services think of it this way well I'll show you I'll show you a slide in just a few minutes but what's cool with this one is that not only do I get a lot more control and power because I can use all these different a AI services and I can use my own models and can even set up my own orchestrator and I can even set up my own content security policies right um I've heard stories of people who are uh working in like medical research and they're doing like research on breast cancer and sometimes their prompts and co-pilot are uh flagged as being inappropriate because it has the word breast in it but clearly that's a research thing that's a that's viable stuff to ask ask about but also clearly that could be used as like you know sensitive content so is it sensitive well not in a medical scenario but in a let's just call it other scenario yeah it could be totally inappropriate so if you want more control over that you can't change those dials and everything that that co-pilot has right that's out of the box but if you do a custom engine agent you're in full control it's like hey look I want the exper I want users have the same experience they do with other agents and co-pilot but I want to be in control of that data they get back all right what's cool about these custom engine agents you don't need any license to use this because when these things run you are getting the the chat experience from co-pilot and the users are getting the same experience they do from like out of the boox co-pilot and declarative agents but the services that run them and Implement them we're not using what Microsoft gives us instead as the developer I'm responsible for paying for those services so I would have like models created and deployed in the Azure AI Foundry I would have maybe my own Azure AI search where I'm not using semantic index but I'm using Azure AI search just to index and vectorize all my content Etc so these are really cool these are very powerful gives us a lot more control but it also means by more control I'm responsible for paying for it myself but I've heard a lot of stories here about where people are actually saving a ton of money by creating custom engine agents instead of doing declarative agents when does it makes the most sense it it all depends on the scenario now the question then comes in as I just said what am I supposed to do it depends what should you build now Microsoft has a slide that looks similar to this I've changed some stuff because I don't think that their stuff really reflects exactly what's going on so here's how this works over here on the far left hand side this is the business chat section this is what you get at think of this is out of the box co-pilot what do you get so these Bluer things this is like the base stuff so I'm getting access to the foundational model GPT 40 I'm getting access to the orchestrator that Microsoft has created cre for Microsoft 365 and that's the thing that does all of the AI resp responsible AI security trimming and it knows how to go look at a prompt and knows to go to um go to uh this a action for a skill and go get data from this uh graph connector for knowledge uh it knows how to do all that stuff it then is going to be available to me in all of the Microsoft 365 apps and to use it I have to be a licensed user uh from it says Microsoft identity but basically that's just inra ID right I have to be a licensed user in my tenant right um it also does have access to though to data additional data from the semantic index now what if I want to go custom well what I can do is I can create my own custom prompt instructions and this is a text file that basically just sets all the ground rules so if you've used chat GPT and you're like you know you're an expert at SEO for articles um I need you to look at all I'm going to give you the content of an article I want you to suggest five uh potential um article titles um each title should uh uh factor in or incorporate one of three of the most popular um emotions to get someone to click on the title to go look at it I think it's a curiosity um fear and I forget what the third one I always forget the third one but it's like curiosity fear and um uh like excitement or something like that if you use one or two of those it's more likely to get a click um that's why you see all these Sensational titles on YouTube and on different articles uh so you can say hey give me some suggestions for this so that's like what a custom that's what a prompt is when you're telling it kind of like you can do this is what you can do the rules about this are you should do three two these are the three emotions you should use one or two but not all three uh the titles should be between 50 and 60 characters long if they're going to be articles if there are YouTube videos there's different rules um I also descriptions so and here's some uh keyword you should use very popular uh keywords examp and give some examples Etc so you give it all of that kind of stuff you can put all that stuff in your instructions you also can create actions right and those actions that's where I can go to those real- time uh data sources so we talked about CRM Salesforce Etc uh we also then can add additional data uh as a graph connector so we can have additional data that's going to be grounded here these other things that you see like triggers what are these things so triggers are going to help co-pilot take proactive actions and facilitate um uh um different automations right um so an example would be like an event or a message that prompt that is going to prompt co-pilot to do a very specific thing uh that's like it's going to say like hey look this is going to trigger me to go get data from this specific system I Can't customize that so much with a declarative agent I can't do anything with that when it comes of the business chat but in my instructions I can tell co-pilot that you know how to do these things and that's coming from this Plugin or from this action um the orchestrator is then going to be responsible for overseeing and synchronizing all of the pre and post different processing of the users prompt and the across foundational knowledge foundational models and knowledge and all the different actions now when I want full control over stuff that's where a custom engine agent comes into play and notice here the big thing is why why am I not having to pay for a license for it because I'm not paying for or their orchestrator or their foundational model I'm responsible for creating my own model or using my own model I can bring my own model or I can use models that are inside of azure Studio maybe I don't want to use AZ Studio maybe I want to use Google Gemini maybe I want to use the stuff from AWS or anthropic or from a hugging face I can do that with custom engine agents um the semantic grounding I'm the one that's going to be responsible to go get all that different data sure I could use an API to go query data from the semantic index or from different data sources uh different uh graph connectors that I've created um or stuff that doesn't live inside of a connector and that is data that lives somewhere else that's been indexed like for example back to the question earlier as your AI search um I can Define my own triggers I can go through and build my own my own implementation of how I'm going to go get all this data the way that we can do this with plugins or actions inside of declarative agents we follow a specific schema and a specific pattern um because that's what the orchestrator is going to expect but when I'm in control I can do whatever I want right I can also go through and build my own custom user experience with this I can have it be people that are in Microsoft intra and licensed or co-pilot or I can just make this work on my public website this is all up to me because it's all running not inside of Microsoft 36 D co-pilot it's running in mine it's running all my resources when a user goes to use these different things they can't tell a difference between either one of these all they see is a list of Agents they have access to and they can at mention them or go in the immersive experience this is simply an implementation detail that we as developers have control over so custom engine agents that is not a co-pilot Studio thing that is something that you do with teams toolkit okay all right so so what do you need to do to be able to do this stuff well this slide here just kind of walks through all of the different things of things that you can doing uh that you need to be able to start doing development uh for Microsoft 365 uh uh co-pilot extensibility so you are going to need to have a Microsoft 365 tenant with co-pilot uh there are some limited preview options available in the Microsoft 365 developer tap membership uh most of the time you're going to end up needing a base 365 license so there's a like God there's a ton of them that you that are available here but like you can even get something as low as the um Microsoft 365 Business Essentials or business basic license which is $6 per user per month with an with a annual commitment and then you usually also need this co-pilot license but the new chat the new co-pilot chat um experience is has a metered option where I pay per message um I have an article on that uh if you're interested just post something a question in the chat I'll pull a link to it I can drop it in the chat for you to go take a look at but I show you how to walk through that I will tell you though in my experience and from what I've seen the metered approach for using co-pilot um in the article I kind of walk you through some scenarios in my opinion it is really only useful in a developer scenario uh or for very very small companies uh in small usage of co-pilot um you'd be surprised it's a penny per message but one prompt is not one message like grounding data uh is going to be I think like 25 messages right and that's per prompt and you may have multiple grounding things that it's having to go through so it may have to go back and forth to graph a couple times or to get a couple times so you could be spending like 25 Time 4 for a prompt um the cost can run away very very fast but in a developer scenario that's good because it's easier to get to control that and all the it's all done through um the Power Platform um admin Center and you basically connect it to a resource Group in Azure that's in an Azure subscription uh and that way we can do meter billing because co-pilot uh all meter billing is done through Azure whereas all uh license based stuff is all done through U Microsoft 365 um one thing that does kind of suck about this is that if you are one of those fortunate people uh that has a uh still has one of those developer tenants from the developer program um you cannot add a co-pilot license to this you can though configure it for uh the consumption pricing though I've done that um also what about co-pilot Pro that other license that's out there you can't extend that this is only a Microsoft 365 co-pilot thing for graph connectors uh you're going to go into the Mac the Microsoft admin Center and into the search and intelligence section and there's a there's a setting in there where you need to you need to set the graph connector that you want to use use inside of your agent you have to set it that it is enabled for inline results so that's something you have to turn on um also indexing content is subject to available index quota within the tenant there are limits on how much content how many items and how much data you can use uh inside of a connector and across all of your connectors uh furthermore when you're devel when you're developing agents and uh actions what we used to be called plugins um when you're using the teams toolkit you have to go configure uh in the teams admin Center you have to go configure your site uh your teams configuration for allowing uh side loading of apps when you're doing the development of it because really these agents they're all installed as teams apps and you can almost look at these as they're just Bots because really that's all they are they're just Bots um they're just infused with AI okay so let me let me kind of wrap this up I got a bunch of questions that I will come back to but before I do that couple things really quick um of all the question you guys asked as I said I went through lots of this stuff and I hopefully have answered a lot of your questions and I hope this was useful to you uh for those of you who join me you're going to get an email from me later this afternoon uh with uh an option to leave a uh some feedback in a survey I really would appreciate it I'm trying to do more of these webinars more often I'd love to know what webinars you're interested in I've got a few that are scheduled uh for the next month or two um that are up on my website on vos. webinars uh again that link will be in the email that you get um I would love your feedback let me know what you liked about this what did you not like about this give me some constructive criticism what do you want to see more of what do you want to see that I'm not talking about right now let me know I would love to go through and do more of these um there is one more thing though that I did say so I told you that at the beginning of this webinar before I do questions I told you at the beginning of this webinar that I was going to show you how I going to help you how we could how you could um work with me on learning how to build more of these agents so so if you likeed this if you liked what you saw here if you give me just a minute I'd love to tell you how I can help you so I have a workshop a two-day workshop coming up next week uh it's on March the 12th and the 13th it's a virtual Workshop it is about 3 and a half hours uh a total of um uh it's a total of three hours uh per day uh of content on building declarative agents in that we do a lot of overw we do overview in our very first session um that we go through each one it's like it's a it's an hour and a half session and then we do a 30 minute break and then an hour and a half session we do that we do that both days on both Wednesday and Thursday of next week um and you can see all the timing and detail right there uh we in the first the first session there's four sessions so the first 90minut session it's a lot of what you saw here but more time for questions the second uh session on the first day we create graph connectors so I show you how to create a connector and we go into a lot more depth on it um the third session on uh the on the morning of the 13th uh that for that hour and a half we go through building connect we build actions so we build rest apis I show you how to export how to expose uh different endpoints for those and even how to create custom um visualizations for your content and then we wrap it all up at the last session and create a declarative agent that uses all of this stuff uh that we've just created and learned for the previous two days um this Workshop is uh I've ran this earlier in December my students that were in this I got great response from this I actually intended to run this sooner than just uh than next week uh but it just didn't happen um so you can see here Muhammad loved it he said he uh loved the expert guidance and he's now confidently able to and on track to build his first agent uh early this year same thing from uh xay uh he said he loved the time that we spent on this initially like what I did today um in higher level cont racts and why and how the extensibility of co-pilot uh Roode is a longtime customer of mine he said that just in a few hours he's now at the speed in uh this more in a position to evaluate design and start creating declarative agents for his clients it's well worth the money and the time when we went through this uh Mark Rackley you might know this name too uh fellow MVP um he was with me on this uh as well he actually went to the live version of this I did in person at a conference late last year uh I'm actually doing this workshop at uh not only virtually next week but I'm also doing it at two upcoming conferences at the Microsoft 365 um uh Community Conference in Las Vegas in May and I'm also doing it at the techcom 365 conference in Seattle in June you can learn all about that stuff if you look at the the footer of the email that you get later today there'll be links for those with discount codes that you can use I do those as full day workshops uh in-person workshops at those conferences um so what kinds of things would you like learn if you came to this Workshop where you're going to learn what you can and can't do with declarative agents how to provide a custom uh instr how to create Pro sorry how to create and provide custom instructions for Microsoft 365 co-pilot uh how to use existing content in your tenant including sources such as SharePoint online um uh sites and and one drive how you can add unstructured content through graph connectors um how to teach Microsoft 365 co-pilot new skills through creating custom actions how to Leverage real time and relational data with existing or new apis and then how we can use different developer tools like vs code to create custom agents without an expensive co-pilot Studio license those co-pilot Studio licenses it's $250 per month for 25,000 messages if you think you're buying one of those message packs you're crazy I see people buying lots of those it's not just 200 bucks a month okay now last thing I just want to tell you here when is it it's next week it's Wednesday March the 12th and Thursday March the 13th it's from 11:00 a.m. to 2:30 to 12:30 is the first session and the second session is from 1: to 2:30 every session is recorded and you get access to it after the session um after the session is concluded um if you're interested for those of you who join the webinar today I have a discount code here for 15% off you can scan that QR code or you can go to that short link at the bottom they both go to the same spot and you can register for for that uh for the workshop uh next week I've kept it it it's only available to the first 200 students that register there's plenty of space available right now uh but I would expect this webinar is going to grab a bunch of them and plus when the recording goes out um as well it'll be a um uh I I expect it to go through and get pretty close to filling up if not sold out um so the link for this uh that you see there at the bottom and the discount code they will also be in the email that you received from me later today and the recording for this webinar will also be available in the coming days now with that I want to thank you very very much for joining me and sticking with me uh over the course of this entire webinar I do want to run through your questions so if you um let me let me take a look at what we've got uh submitted here and I will take the time to go through and make sure that everybody is going to walk away with all their questions um answered so let me just start from the bottom just kind of scroll through this real quick um I've got to okay so uh couple more questions so this one was from 27 minutes ago by anaja 24 um so can we use and connect to SQL Express across data with co-pilot and does SQL Express have a vector database capabilities so the answer I don't know about SQL Express having Vector database capabilities in fact I don't even think SQL Server does um so can you connect directly to SQL you can through one of the out of thee boox connectors that's in the search and intelligence admin Center uh but uh even if you can't you can always create an action that would talk to a rest API that then accesses data inside of uh SQL um I'm 99% sure that SQL does not have a vector uh is not have a vector database option what does have a vector database option there's a lot of different solutions that are out there but um Cosmos DB uh is Al does have a vector database uh solution in fact that's what chat GP or open AI uh uses for chat GPT uh okay so I'm going to Mark a little thumbs up on these questions as they come in just uh for those of you to see uh when they've been answered um so next question can co-pilot connect to semi structured data like Json yaml not out of the box but through an action yes um instead of your arest API talking to sorry instead of Your Action talking to arrest API that talks to a data store it would just look at yaml or Json in fact I do that as well because all of my courses have a have a markdown file associated with them and the um with the the lesson description and a bunch of notes under the description is in the body of the page and then the top of the page has all this front matter which is all in yaml uh and so I extract that data and I make that available as part of um one of my rest apis uh to my agent um so you said thank you very much for a great introductory course can I walk us through what we would get when we attend this course building declarative agents that was 16 minutes ago so I think I've already done that um also if you go to the uh the the website uh the link to it that I uh if you go to my website um here just show you really quick open up Chrome drop this over here if you just go to my website and if you go to courses and you go to co-pilot the only course I have is this uh is this um uh this workshop and if you just go right in you can click on this link and it takes you straight to the registration page so the course is the I didn't mention this the um the workshop is just $199 that's less than I charged for it back in December um because I really want to get to as many people as I can with this and so uh the 15% discount will knock off about $30 um off of this um off of this Workshop if you join me next week uh so if you go to the site though uh there are there's a whole bunch of information on the site about what you learn what you're going to get and there's even a bunch more testimonials that I didn't go through um that you can read from my some of my students uh that attended this uh last December um someone asked is it legal isn't developer for non-production purposes I don't know what you mean by that I'm assuming you're going you're referring back to my slide when I said how you can uh use uh co-pilot and how you can do development um you can use a a a one of the developer tenants in the from the Microsoft developer program you can use one of those for creating custom agents uh you I mean it's not supposed to be used for production stuff I would not recommend using production stuff because the those developer tenants like Microsoft monitors uh through like automation what you're using oops yeah so Microsoft monitors what you're you what you're doing inside of those agents sorry inside of those tenants and if you're doing things that are not that are not development things then they may not renew your developer tenant um so that's I'm not saying to to to get around that by using it for production that is not at all what I was trying to explain I was talking about how do you do this stuff how do you build agents I'm talking about the developer experience so it is legal like because that's the intention I'm not suggesting you use a Dev tenant for production purposes that's not what I was suggesting um however most of you may know this for those of you who don't the developer program is basically Frozen right now and they're not giving they're not Distributing any of those tenants in fact that was the subject of my article uh that I published this week um about uh a solution for giving us ability to do that and this is just my proposal that I published in my newsletter and there's an art there's a link in here too that kind of walks through um why it's frozen why you can't do stuff um my what I what I propose is a a paid developer program but in the LI of that today when I'm telling my students if they want to do this so like if you join me for my workshop next week and you don't have um Microsoft 365 co-pilot license what I tell you to do is I would go buy a uh a Microsoft 365 Business Essentials license that is $6 per user per month you get your own tenant it is an annual commitment so that's going to be $72 uh for the annual commitment it's paid out monthly $6 a month uh and then what I would do is if you want you with that it's a commercial tenant so you can you can use RVC fit um but you can also use that uh for your production purposes you can also buy the $30 uh co-pilot license or and I talk about in these two articles down here about co-pilot chat uh where this new Option of the consumption billing you can set up consumption billing to be able to try this stuff out and there's zero commitment with that um this is kind of an overview on what this is and I I think I look at this consumption thing is a great option for developers in this second article I have here this is walking through how to set it up in your tenant so I would uh that's what I tell people to do so like when I teach this like when I'm teaching my teams accelerator where we're going to cover this one in one of our um one of our modules I tell people if you want to follow along with the homework this is what you're going to need to buy I would do the consumption one you're not going to spend much money on just testing things out and plus there's no full commitment all right okay I am going o I've gone over by about a minute or two on my workshop uh time here I'm going to go ahead and wrap this up now um I want to make sure that I'm respectful of your time so thank you thank you thank you thank you everybody for joining me today uh and learning a little bit about what we can do with um uh co-pilot Microsoft 365 co-pilot extensibility and creating custom agents I hope to see you next week in my workshop uh where we're going to create custom agents custom declarative agents uh and if we have time we'll even talk a little bit about uh custom engine agents but that is the the focus on it is primarily on declarative uh agents where we'll go into a lot of depth on all the things that we can do so with that keep an eye out for an email from me later today with a summary of what we went through today uh a link to where you can provide feedback a uh where you can also give me some feedback on what uh webinars you would like to see uh and then furthermore where you'll find a discount code for 15% off next week's Workshop so with that I'm signing off I'm Andrew Connell thank you very much for joining me and I hope everyone has a great rest of their day [Music]