Event recap: “Bots in the Real World: Truth, Hype and Demos”

A lot of us long for the day we can kick up our feet and let Cortana manage our professional lives. Or we worry that our jobs will be stolen by bots.

This month at the Melbourne Office 365 Business User Group (BUG), they took a shot at cutting through the hype and getting to something practical.

Facilitated by Engage Squared and hosted at Microsoft’s office at Southbank, they had 90+ registrants and a full house on the day.

Host Jack Hendy (an adoption specialist at Engage Squared) kicked off the session with…

“What’s new in Office 365”:

  • Cortana can be tagged in emails to help arrange meetings (Your personal assistant, just @Cortana)
  • New Q&A functionality in Yammer (Lets you mark answers to questions)
  • Microsoft Whiteboard App (Allows you to add reactions, integrates with SurfaceHub and has “ink beautification”: Bad handwriting? No worries)
  • PowerPoint-designed slides (Make your company’s branding consistent using custom templates and automated suggestions)
  • New customisable and unified Microsoft Search (A reason to use Bing)

AnswerBot: Going with the Flow

Their first presenter, Jason Soo from law firm Hill and Wilcox, introduced us to the basic Question and Answer (QnA) bot, enhanced with Flow. The bot solution leveraged Flow’s approval workflow, allowing the bot to update its QnA knowledge base with new answers to new questions.

This bot/Flow duo:

  • Captured questions from a Yammer group (this could be any ‘front door’/entry point, such as an intranet or website)
  • Answered questions parsed through the Azure QnA Maker
    • If the question is “like for like” or the answer has a high degree of confidence (an adjustable setting in QnA Maker), the response is posted in Yammer
  • If the confidence score is low or no match has been found, an approval flow is triggered which then presents a subject matter expert with the question, a potential answer and an option to submit another answer
    • This can be extremely powerful for enterprise-wide bots, as subject matter experts can be tied to different knowledge bases (owned by different portfolios/organisational groups), all connected to one bot
  • The answer, once accepted in Yammer, is automatically added back to the associated knowledge base

Apart from the ‘learning’ AnswerBot is capable of, she can be used to drive uptake and draw audiences to platforms like Yammer.

Jason also showed how he was able to set up AnswerBot (Flow not included) in 5 minutes using an Ignite video found here.

Codee: Journey from FAQs to Teams Governance

Next on deck was Adrian Tan and Sarah Aquilina from the Department of Health and Human Services (DHHS). Codee started her journey as an Office 365 FAQ assistant and is currently on the road to becoming much more, starting off with Teams Governance.

How do you transition an organisation of 13000 people[1] off Lotus notes and onto Office 365 while keeping up with basic (but necessary) queries?

The answer was Codee (formally known as BAE).

Codee is a basic QnA bot (living in Teams) that also leverages the Azure QnA Maker. Sarah, a non-technical stakeholder, showed how she was able to use the QnA maker to manage and improve the bot without needing help from the rest of her team.

The related cost of Azure consumption and virtual machines in the cloud was also discussed. The group agreed that different business problems called for different levels of investment and the free options provided by Microsoft to trial bots made it easy to get started. Generally, the ROI associated with the automation of answers to high-volume, repetitive questions was seen to pay for the initial outlay quite quickly.

Adrian, the program manager at DHHS concluded that paying the monthly cost to maintain Codee allowed his team to focus their efforts on activities more impactful to the business.

Codee is currently being developed to have the ability to create Teams at DHHS and asking for Teams names, owners and settings during the provisioning process.

ZELA: The Truth and the Hype

Elaine Van Bergen finished off with the (not so cold) hard truth: bots are just apps.

They’re apps that can be used to supplement other content in areas such as websites, Yammer and Teams.

In its more advanced stages, a bot:

  • Can be used to greet and attempt to assist users (and refer them to humans if sentiment analysis or the specificity of the questions warrant it)
  • Can be used with voice recognition or integrated with IVRs to provide feedback and activate workflows

While these examples are exciting, Elaine emphasised that while it’s easy to create bots and get immediate returns, any complexity or customisation will find you needing developers. It was also recommended to run basic bot projects before delving into more technical projects.

Elaine then introduced ZELA to the group.

ZELA is Microsoft’s internally built bot that assists with high-volume, low-risk queries to the legal department which previously lead to bottlenecks. ZELA helps with:

  • Answering self-help queries such as “where can I find…”
  • Client requests for support
  • Answering general, front-line questions from clients

The group agreed ZELA was a good example of fast value-for-effort and was intrigued by its use in a department as critical as Legal.

 

The session concluded with a prize giveaway, thanks to sponsorship by Logitech.

If you’re in interested in future Business User Group meetups, please join them and us at the next meeting!

Melbourne: https://www.meetup.com/Melbourne-Office-365-Business-User-Group/

Sydney: https://www.meetup.com/Sydney-Office-365-Business-User-Group/

 

 

[1] DHHS, ‘People Strategy 2020’, Department of Health and Human Services, Victoria, 2017, p 15, https://dhhs.vic.gov.au/sites/default/files/documents/201708/People-strategy-2020_2017-07-03.pdf, (accessed 4 July 2019).

Doing UAT Right

Of all forms of testing, user acceptance testing is often the most essential to get right. Why? Because it saves you money. Too often I find organisations underprepared for the task of doing UAT properly and I believe this is largely due to the organisation not knowing the true impact of NOT doing UAT properly.

In 2017, IBM found the cost to fix a bug found after a build was 4 to 5 times higher1 than if the bug had been found in the design phase. More alarmingly, it was 100 times higher if the bug was found after deployment to production.

diagram cost of bug fixing over stages

4 Steps to getting UAT right

  1. Define your UAT team
    It’s important to avoid restricting your UAT team to project team members. The most important group to include in UAT testing is the “real” end users of your solution. And the key word here is ‘users’. This is crucial because they’re the people who will use the solution daily. Every persona and stakeholder group should be included, which means that people from each group should be selected to join the UAT team. Prepare that UAT team early and enable focused UAT to happen by booking in the UAT resources ahead of time. Consider booking time in your UAT team’s schedule to enable them to adequately perform UAT.
  2. Confirm test cases and acceptance criteria
    Agree upon user stories and their acceptance criteria during the design phase of your project. Each user story should cover a specific use case or scenario of the solution and therefore lends itself to being turned into test cases for your UAT. The test cases are normally a set of actions which the UAT team member can carry out to verify if the solution has worked as intended. The acceptance criteria define what is considered to be “working” in the solution.
  3. Develop a UAT plan Consider the following when writing your plan:
    • When will UAT start and finish?
      – It is important that a finite date is applied to the UAT period to avoid perpetual testing.
    • How will the test results be collected?
      – Write down how the communication between your project team and your UAT team will take place while testing. Doing this prevents the project team from the nightmare of receiving emails, word documents, spreadsheets, screenshots (or no screenshots) and endless discussions through email and other channels.
      – Collecting and managing bugs in a central repository will also mitigate the risk of duplication of bugs, as well as assist with the management of bugs through the correction and retesting process.
    • How will you label any bugs found (bugs, feature-requests, usability, training, etc.)
    • Who will triage the results?
      Remember: the triage process can bury you. Plan ahead and appoint someone to manage the triage and confirmation of defects.
  4. Conduct UAT
    Provide the UAT team with the list of test cases (as demonstrated in table1 below) as well as for instructions about how to log a bug well (i.e. the bugs should include the account used for testing, the device used for testing, screenshots, URLs, description of how to reproduce, etc.). Consider holding a UAT ‘briefing session’ to take the UAT team through what is expected of them throughout the UAT period.
    With the UAT plan in place and the UAT team readied for the task, the UAT can begin with the tests being performed and the results recorded. Were the tests successful, or did defects result? Any bugs raised then need to be triaged, confirmed as defects, corrected and re-tested.

    Table1 – Sample UAT test caseSample test case:
    As a user, I can see news on the home page
    ID Acceptance criteria description Pass/Fail/other Comments
    1234 Verify that you can see a news carousel on the homepage
    1234 Verify that the news carousel auto rotates
    1234 Verify that you can click on a news image and be taken to the news page
    1234 Verify that when you click on the navigation arrows in the carousel, the image displayed changes.
  5. Close and sign off UAT
    Once all testing is complete and all bugs are corrected, the project team should conduct a UAT closure meeting. During the meeting, you should look to tie up any loose ends, and formally close the testing period. Signing off or closing the UAT period means that you can move your solution into production.

Many development teams, including Engage Squared, apply an agile approach to projects with continuous integration and continuous testing. Agile’s incremental approach allows more feedback, flexibility, and of course testing, so that every time a feature, fix, or function is added or changed in the code it’s checked for bugs. In turn, we have found this helps avoid preventable bugs – ultimately saving time and money.

 


1 https://www.researchgate.net/figure/IBM-System-Science-Institute-Relative-Cost-of-Fixing-Defects_fig1_255965523

Working with the AI Spark partner program

Engage Squared recently had the opportunity to spend two days in Hobart with our partner LiveTiles, as part of their AI Spark Partner Program.  

AI Spark is a Microsoft and LiveTiles accelerator program for partners working with the LiveTiles Bots platform. Through the program, Engage Squared will be able to build AI capabilities that we can use to help customers develop and deploy AI solutions. 

LiveTiles Bots is a ‘no-code’ bot builder powered by Microsoft Azure’s bot framework and language understanding intelligence service. It takes a huge amount of the complexity and required technical know-how out of the bot-building process, allowing users to quickly create an AI tool relevant to their needs. 

We’ve been hearing more and more of our customers talking and asking about bots and AI, and it’s clearly an area set for a huge amount of growth, with enormous opportunities for businesses to take advantage of the technology to transform the way they and their people work. 

We’ve learned so much about bots and artificial intelligence over the past few days, being able to work with a no code solution that integrates with so many SaaS based applications, utilising complementary technologies such as Microsoft Flow has clearly outlined why AI is of great focus to Microsoft. 

Using the LiveTiles bot platform means the approach to building bots becomes less about investing technical effort and navigating technological boundaries, and much more about understanding what our customers are trying to achieve and supporting them with the design thinking process that is necessary to deliver a successful bot scenario, and the creation of custom connectors to the services and platforms they use.  

This flexibility is a testament to how LiveTiles have made it so easy to configure a bot and connect into an enormous number of data sources – basically any that have exposed API’s. 

The wide-ranging two-day AI Spark workshop started off with a review of the underlying Microsoft bot framework and suite of AI services, followed by a detailed walkthrough of the LiveTiles Bot product, and how it overlays the technology to provide a far simpler interface for users.  

We talked about the value of bots and some of the many real-world use cases that they can help with, from HR Bots returning your leave balances, to a public facing bot that helps customers check the status of their order and answer complex questions instead of calling a dedicated support number.  

Through this process, it became apparent that the quality of data and inputs in a production scenario was something that needed to be reviewed and be given thought to spend the time working on retrieving quality data, and putting it into a format that works well for a bot interface.  

We also discussed some of the many lessons that LiveTiles have learned from working with customers to develop bots. For example, often there is a temptation to want to jump straight into building an “uber-bot” that can do everything, but typically a much more successful approach is to start by thinking about your most valuable use cases and starting with one or more simpler, targeted bots that do one thing well – this approach is quicker, more reliable and more likely to be a success. From there, we can start to build on this success and look to more advanced things like starting to combine bots’ abilities. 

We also got a sneak peek of some of the many enhancements and innovations that LiveTiles are continuing to work on to make the platform increasingly useful and powerful for users. Bots and AI are set to be a major growth area and both Microsoft and LiveTiles have made clear that this is an area they will continue to invest in heavily with native integration with different systems and an evolving language model. 

The second day of the workshop was spent rolling our sleeves up with LiveTiles Bots and getting into some rapid prototyping. We brainstormed some use cases that would be valuable internally at Engage Squared, and after settling on two examples started to build our custom bots. This was a great experience to put what we had learned into practice and get us practicing design thinking, solving problems and building our familiarity with the product and everything it can do. We’re looking forward to putting some finishing touches on our working bot prototypes and sharing them with the team back at Engage Squared! 

If you think your organisation could use a bot to help your staff, or it’s something you would like to know more about, we would love to chat to you about how bots can help to drive productivity and bridge gaps between systems and users, contact us to arrange an overview of LiveTiles bots. 

 

Written by Thomas Lalor & James Di Blasi

Engage Squared

New year, new name!

We’ve decided to start the new year with a new name! The consultancy formally known as WM360 will now be known as Engage Squared.

We were originally created as a child of our London, New York and Minsk office. We’re now setting up our own brand in Australia and New Zealand – called Engage Squared (or Engage2 for short) – and we are very excited about it!

For our valued clients, everything will be business as usual. We still have the same great team, and we’re still doing the same great work. We’ll also still be working closely with our European counterparts – they’re our partners now.

So how did we get our new name? Well, engagement is extremely important to us! We believe this can only be achieved by combining two things – Technology and People. Engage Squared will focus on bringing people and technology together to create great systems that people love to use!

Our new name reinforces the strong focus we apply to user experience, adoption, training, and user engagement. By taking this approach, we make sure that our clients find the right solutions for their business, and that their people are ready to adopt the solutions for better business outcomes and return on investment.

We loved working with you as WM360 and I’m sure you’ll love working with us as Engage Squared!

Happy 2017!