EP 9: Microsoft Power Virtual Agents: A Step-by-Step Guide

Transcript for this Episode:

INTRODUCTION VOICEOVER: This is Tech UNMUTED. The podcast of modern collaboration – where we tell the stories of how collaboration tools enable businesses to be more efficient and connected. With your hosts, George Schoenstein and Santi Cuellar. Welcome to Tech UNMUTED.

GEORGE: Welcome to the latest episode of Tech UNMUTED. Today we're going to talk about a really interesting topic, Power Virtual Agents within Microsoft 365. We've been playing around with these for quite a while now. We've launched two internally so far. It continues to be updated. We've talked about this. The 365 platform, day by day, week by week, month by month, there's always something new that's been added to it. What we're going to talk about today is how we've used this internally, but walk through the steps of how you would set it up as part of that discussion. There's been a couple of added features even in the last couple of days. Santi, you'll go into that. One of them, in particular is around grounding. The initial grounding, when we were working with these, we grounded it to our website as one of the setups that we did. You can now ground it to SharePoint. For folks who don't know grounding, Santi, you want to just give a high-level overview of what that means?

SANTI: Yes, sure. Grounding is basically pointing, if you would, for lack of a better term, pointing the artificial intelligence to your specific data. Unlike ChatGPT, that has billions and billions of parameters, you can still use the ChatGPT engine if you would, but you ground it into your specific data set. That's what this is involving.

GEORGE: Awesome. The other thing I'll point out, if you're listening on audio only, we're going to try to keep this as descriptive as possible. This is another one, you might want to listen on the audio first and then take a look on YouTube and watch some of the things because we're going to drive some stuff on the screen with this. Santi, if you want to jump in.

SANTI: Absolutely. I'm going to go ahead and share my screen. Here's the chatbot we created, and we actually called him Mark Eting. By the way, George, it was a hit when we rolled it out. We rolled it out as an additional member of the marketing team, and a couple people caught onto it. Others actually tried to contact IT, wondering why they can't find this person in our active directory. [chuckles]

GEORGE: Again if you're not getting the joke it's marketing with a break between the K and the E, so Mark Eting.

SANTI: Mark Eting, as you can see there from his welcome message down below. He is the lead knowledge manager for Fusion Connects marketing team. We created Mark Eting to answer frequently asked questions that the marketing team gets. We have resources out there, and we're always being asked, "Hey, where can I find X resource?" It's usually the same question. The intent here is to create an intelligent bot that can answer those questions.

We created Mark Eting, as you can see here, right away there's a box that talks about creating generative answers. Well, generative answers or using large language modules, which is what ChatGPT uses, is leveraging in essence AI to come up with a response. We're going to show you a little bit of that. I just want to first just navigate through the actual power virtual agent once you create the chatbot. It's actually very easy to create a chatbot by the way. Just click Create and drop in the website that you want to use as the grounding of the knowledge base. That's it and then it brings you to the screen.

Let's go ahead and look at a few things here. I'm going to start off in the setting section, going to the bot details. This is where you would come and give your bot a name create an icon. Give it a persona if you would, you would do that here. Channels is where you can make your bot available. You can now come in here and say, "Hey, I want to make this bot available in Microsoft Teams." You can make them available in a webpage, for example. We're using two channels, Teams is one of them. There's a particular teams channel and Mark Eting has been assigned to that channel, so you can ask him a question just like you would anyone else. You literally would do an at mention, so @Mark Eting and say, "Hey, Mark, where can I find X resource?"

GEORGE: Again, on this, the grounding is what matters, right?

SANTI: Of course.

GEORGE: This isn't ChatGPT, just trying to find an answer and build an answer out. This is, again, we grounded this against our external website initially. This capability to do it. Again, SharePoint, has just recently launched, we will test that out over the next couple of days. One, see how it works, and two, see what it has access to, because there's a wide range of stuff on our SharePoint instance. We want to make sure that it's accessing and presenting in the way that we think is appropriate and-

SANTI: Appropriate and safe.

GEORGE: Yes.

SANTI: Then we also use him-- Speaking of SharePoint, we use that custom website icon there. It pretty much creates like an eye frame. On our marketing page within SharePoint we've inserted Mark Eting so that it's easy to find him. These are channels and this is how you can decide where to make your bot available. There is some new security features which we're going to be looking into. Right now, we're not leveraging any of these. These are fairly new. One of them is authentication which is interesting. Basically, you can either leave this bot open, meaning that anybody can use it. That's great if you have a bot that's public facing and he's maybe engaging with customers.

If you want to identify who is the user trying to use your bot, then we can activate this authentication, which means that the user would have to sign in using their credentials, then the bot will be able to identify and refer to the user by their name. It's a much tighter experience at that point. Again some security features here including some web channel security features. If you want to use the bot in a customer facing webpage, but you still want to add some security, you can do that. This would be that section to do that.

I'm going to jump down to AI, because this is what everybody wants to talk about. Is how do we make this thing smart? To George's point, one of the things we've done is, as you can see here, we have grounded the data to our website. What does that mean? That means that when you ask the bot a question, it's getting the answer from the knowledge base or the data source that you've assigned. In this case, it's the public-facing website.

What has happened, and this just happened, is now available, is now we can now add an internal source. Now, if it's internal, that means it's sitting behind a firewall. There has to be some kind of authentication that takes place. The way they're doing this is they're using Azure OpenAI. As you scroll down here, you'll be able to see here where it says, "Add knowledge to this bot from Azure OpenAI on your data." This is where you would go. This is the box and it would take you to a set of instructions on how to configure this. There'll be some type of a token or some type of an authentication that takes place so that the bot now has access to internal data.

That is how they're pulling us off, is they're basically leveraging Azure OpenAI and then tying that into probably your active directory, then giving the bot access to scan a specific page. George, what we don't know yet, meaning we don't know because we haven't configured it yet, is, can we limit this bot? Because when you say SharePoint, SharePoint could be a massive data verse in and of its own.

One of the things that we want to confirm and then come back and maybe even do a follow-up at some point is can we pinpoint to a specific site within the SharePoint. For example, can we limit or ground the data to, let's say the marketing portion of SharePoint alone. That is what still remains to be confirmed. We're still looking into that.

GEORGE: You need to think about that from a broader context. We shifted to 365 about two years ago now. We had a different platform for the internet. We did a very deliberate move of content. It wasn't a lift and dump from the old site to the new site. What's out there today was to a great extent curated for the site and built for the site by the different teams that have their content out there.

Our data is very clean, but many organizations may have had SharePoint instances for 7, 10, 15 years that are out there that have all kinds of data, much of which they may not want to expose to employees. It may be there if they search, but to send the bot out to dig more intuitively and act like a person sort of filtering the data, you're likely to end up uncovering things that you wouldn't want exposed or exposed in the way that they would be returned by the AI bot.

SANTI: More to come on that. We're going to dabble into that a bit. This is a brand-new feature, just became available. Another one that's a new feature that's also available is the generative actions. It is not just the response, but what actions does the bot take. Now you'll be able to, as it says here, you can enable the bot, again, using Azure OpenAI to select a chain of reactions or steps that it can take so that it can tie in plugins to complete tasks and walk a customer, if you would, through a specific flow. That's interesting. I like to see how that would pan out, but this bot is really taken off. They're very focused on making this as smart and as conversational as possible.

Another new feature that's available, and there it is, intelligent authoring with Copilot. Again, brand new feature available to the power of virtual agents. What that means is as you're trying to develop your list of topics, because while the AI can go scan the specific data source that you have designated, there might be topics that live outside of that data source that you want to include. Well, now in order to maximize the potential response from your bot, Copilot can help you with that topic.

You basically give it a description of what you're trying to accomplish, and it'll generate the topic and the potential responses or questions that people may ask, and that becomes part of your topic list. I'll show you that in a second here. Then finally, again, coming soon, brand new. Now they have a tone. Now they're really trying--

GEORGE: Nice.

SANTI: Yes. I like this. You'll be able to take your bot and give it, again, tying it back to that persona. For example, Mark Eting which is a "knowledge manager" for the marketing team, we probably want to have a good conversational tone or friendly tone with him, but you can see here it can be very formal. I can see that for law firms or those types of businesses that seem to be on the high-end professional side. A technical tone. There's an empathetic tone, which is great for that customer facing bot. Especially if it sits in a section that is specifically there for customers who maybe are calling there for a complaint or they need help. I'm looking forward to seeing how this works.

Again, it's not available yet, it's coming soon. As you can see, a lot of these things have just been added to the power of virtual agents very recently. We're still dabbling ourselves into this, but bottom line is, right now this is our source. For topics that fall outside of this data source, let's take you over here to topics real quick. We've gone ahead and we've created our topics, and you can do that too. You can come in here and say, "Hey, where can I find, for example, questions around branding, or do you have any company brochures?"

I'm going to click on one of these just to show you an example of a flow. One of the questions we get asked a lot is, "Hey, where are the battle cards?" As you know part of marketing, we try to help sales with the right information when they're talking to their customers about how to position certain solutions and whatnot. That's a very common question. "Hey, where can I find the battle cards for this, or the battle cards for that?" Let me just pull up the flow.

GEORGE: Basically, there's two flows here. There's the backend, the AI, the ChatGPT element looking at the source data. The grounding, which is our website.

SANTI: Correct.

GEORGE: Then a second set that you've built manually to say, I know I've gotten this question over and over again. I'm going to build out a standard response to it that gives very deliberately the answer we want. That you might want to mention a little bit, like you and I and everybody used to constantly get questions, where's this, where's this, where's this? Although you could search on SharePoint, the search generally gives-

SANTI: Of course.

GEORGE: -you pages and pages and pages.

SANTI: Of course.

GEORGE: What sort of the before and after look like, do you think from people asking you questions versus now they're using Mark Eting?

SANTI: Well, I will tell you that first of all, they're getting more precise responses. Here's a perfect example. If you say, I need battle cards, but you don't tell me what kind of battle card you need, I'm probably just going to send you over to the marketing site, to the solution library. Then you'll have to search which solution you want the battle card for. In this particular case, Mark Eting can actually drill down a little deeper. For example, there's a couple of phrases up on top, "Hey, where are the battle cards or where can I find the battle cards?" That's all Mark Eting needs to trigger a response.

In this case, the response is actually a follow-up question, which solution are you looking for? Now there is a multiple choice that they need to pick from. Depending on what they choose here, it will then point them to the resource. If they happen to say, "Hey, what I want is SD-WAN, then it'll literally give them the link to where the specific battle card for SD-WAN resides. Now the reason we created this topic is because this particular topic lives outside of the grounded data source, which is our public-facing website. I foresee this topic no longer being needed as soon as we're able to ground the bot to the SharePoint site, because it would index the site and it would find this on its own.

Think of topics as those things that fall outside of what the grounded data is able to provide, or if it's something that you want to be very specific on, you can go ahead and create that topic. Tell you what, why don't we do this? Let's take him for a test drive. He is new to the team, so I think it's only appropriate that we welcome him on board. Let's just say welcome to the team. Let's see what he says. Oops. [silence] Well, and he says, "Hey, I'm excited to join the team. Let me know if there's anything I can do to help." As you can see it pulled up on the right, the trigger and the topic that he's using. See that.

GEORGE: Obviously, we only see this because-

SANTI: Correct.

GEORGE: -we're the back end of this. User wouldn't-

SANTI: Correct.

GEORGE: -but this is…

SANTI: Correct. This is how you're able to test the bot, because it's showing you what is the flow that he's following. I'm going to ask him a follow-up question. I'm going to say, what is your role? As you can see, it shows you what topic it's grabbing from, and it's showing your response and there's his answer. Of course, everybody is going to ask Mark Eting if he's real. You know that's going to happen. Right?

GEORGE: Yes.

SANTI: So let's ask him, are you real? This is really a good opportunity to really give him a good persona and maybe even add some levity to your response as we've done here. He says, "Yes, I'm real. I'm very real. I may be part of artificial intelligence, but I am as much a part of the Fusion Connect team as you are." It just makes it very conversational, very interactive. You can get very creative with these topics as you can see here.

However, I want to ask a question that I know is not residing inside of the topics list. It's residing inside of the fusionconnect.com website, which is where we've grounded this bot and I'm going to refer to blogs in particular. I'm going to say something about, let's see, we talk a lot about Microsoft Teams calling and Operator Connect. Let me see if I can ask him something. Do you have a blog about Microsoft Teams calling services using Operator Connect? It's a very specific question.

GEORGE: Specific but complicated.

SANTI: It is complicated.

GEORGE: Small….in there. It's a blog post. It's calling services, it's specifically Operator Connect.

SANTI: Let's see what he does here. I think what's going to happen is-- He wasn't clear on the question and so we got this response. I say none of these. He's going to ask me to just try this again. By the way, it may be that we got that because we're specifically inside the topics pane and so he may be trying to search for in the topics pane, but I'm going to rewrite the question.

[pause 00:21:40].

So, what it's doing here, Perfect. This is great. What happened here? If you look over the topics pane it says, "Hey, I have an unknown intent," meaning I just could not find this question in the list of topics. Then what am I going to do? I'm going to then go to this prompt which I'm going to generate a response based on my data source, which right now there's only one. It's a public website as you can see here, this is where you would increase your data source.

Then it went ahead and generated an automated response and it actually gave me multiple links. There's a perfect example. By the way, he was very descriptive, because I just asked I need a blog on Operator Connect and his response was Operator Connect is a delivery method of Microsoft calling services that enables Microsoft Teams to make calls. You can find more information about it in the following blog post. That response is not something that I pre-crafted, right? This was--

GEORGE: That's a great answer, right?

SANTI: It's a great answer.

GEORGE: That's a great answer externally, if it's on the--

SANTI: Exactly.

GEORGE: At some point we will deploy this on the public website. That's the answer we wanted to give when somebody comes in.

SANTI: Absolutely and it's accurate. The answer is very accurate. Operator Connect absolutely is a delivery method. What's interesting about this is that it generated, again, this is why it's a generative answer, it generated this answer using ChatGPT in the background, leveraging or grounding the data based on the public website. It did this because it couldn't find a specific topic around what I asked, and so it went to the source to generate this response and it gave us three links to three separate blogs, which is great.

Anyway, that just gives everybody an idea of how Power Virtual Agents work. I'll tell you what George, it actually is pretty simple to set up. You start off with just asking "Hey, can I get a new bot?" Then entering the website that you want to use for the grounding. Then you spend time in the settings section that I just demonstrated. The key here though is until you publish the bot, it's not live. You can test it in the sandbox, but until you actually go through the process of publishing it, it won't be live.

I think Microsoft is really focused, like I said, on this Power platform and Power Virtual Agents is part of this. The fact that just from last week till today they made so many updates already, you can tell AI-- Again, Copilot is now in Power Virtual Agents. This is the future. Microsoft is betting their future on this. You can expect to see these smart bots popping up all over the place powered by Microsoft now, which is great.

GEORGE: It's so easy to use. Again, we've talked about this repeatedly in the podcast. Anybody can do this. This isn't complicated and everybody should do this, if for nothing else than to understand how the various services within 365 work as well as other things. We had this on a previous one as well with Sam Husbands who joined us a couple of weeks ago. Mark's, photo was generated by Midjourney.

SANTI: That's correct.

GEORGE: We went through a couple of iterations of photo options and that's where we landed on that one. Before I hand it over to Santi to close it out, the thing I would ask folks do, comment, send us an email, reach out to us. If there's more of a deep dive on certain elements of this that you want, we'd be happy to run another session on it. We could even using the platform that we're doing, run a live session on this and report it and also repost it. Again, one of these days we'll talk about the platform which is running this off the webinar capability within Microsoft Teams, which is a premium feature at the moment from Teams. Santi, if you want to do the close out.

SANTI: Absolutely. With that said, this is the most difficult part of the podcast for me because I am forced to stop talking about the things I love. Listen, if you want to remain connected with our podcast Tech UNMUTED search for us on your favorite podcast platform and subscribe. This way you'll be alerted next time we release a new episode. Until next time, remember, stay connected.

CLOSING VOICEOVER:strong> Visit www.fusionconnect.com/techunmuted for show notes and more episodes. Thanks for listening.

Episode Credits:

Produced by: Fusion Connect