This article is more than 1 year old

Hey Cortana, how about you hide my app from the user?

Who needs CTRL-C, ALT-TAB, CTRL-V when Cortana can do it for you?

MS Ignite AU It's a way off yet, but one of the endpoints for Cortana, Microsoft's take on the digital assistant you can fall in love with, is to be a primary interface between multiple applications.

The idea is that instead of opening an Excel spreadsheet to find a datum you need, copying it, switching to e-mail, and sending it to the boss, you could tell Cortana what to do, without having any direct interaction with Excel or Outlook.

That idea popped up almost unbidden when The Register interviewed Nick Randolph of Built to Roam, who's talking about Windows Development this week at the Microsoft Ignite conference on the Gold Coast.

Randolph identified the direction Cortana is taking as one of the items the Windows development community is taking a shine to – particularly, in the local context, now that Windows 10 has Cortana support in Australia.

Developers, he told Vulture South, are just starting to realise the depth of integration that's already possible between Cortana and their own applications.

"Cortana responds to the voice commands, but rather than just launching the app and distracting the user, it can now surface a lot of the application and its interface back in Cortana – without launching the application."

That's not the "final destination" I led with, Randolph explained, but it offers up the odd picture of an application developer avoiding user interaction entirely.

"Some developers are saying 'we don't want the user to spend any time in our application'", he said. "They're looking at how they can communicate with the user directly through Cortana."

"The less time the user spends in the app, the more productive they're going to be."

Cortana provides an opportunity for a different way to make users more productive, he said, by minimising the number of times users have to change the context they're working in. The endpoint those developers have in mind is not to launch an application at all: let the user ask Cortana for something, get Cortana to launch the application and get the answer, and let Cortana return the information.

A simple and more immediately achievable example might be checking a retailer's phone app for opening hours or to find the nearest store. In that context, Randolph argued, there's no need to open the application at all, if the user just needs to "Hey Cortana" the question.

Limitations

The digital assistant is still a pretty new concept, though: nobody's really going to fall in love with the stilted syntax they have to use.

Think, for example, of the many ways you might wonder about a retailer's opening hours: "When is Colesworth's open?", "Are they open at six tonight?", "What are the hours for the St Leonards shop", and a thousand possible permutations thereof.

Even in simple examples, a dichotomy emerges: a simple step-by-step linear workflow (launch app, locate information, return information) is subject to the messy inexactness of language.

Randolph said working out how to define Cortana's "rather cryptic voice command file" is a skill in its own right: "not just how you ask for something, but how anyone might ask for it". Thankfully, simply because Cortana is new, that challenge is still manageable.

"Right now, the interface for Cortana is relatively restricted – there's a list of items in a template; or there's information with command buttons.

"There's no layout engine, which is a constraint, but on the other hand the back end doesn't have to be creative in terms of providing [results] to Cortana."

However, the background service model is there, and clearly having things run in the background behind Cortana is the "next evolution".

There will still be windows in Windows

We're not quite ready for that world, and Randolph said developers like him are happy that Microsoft has learned, once again, that "Windows" is plural.

The smartphone-app paradigm – just one window on the screen, swipe to switch apps – that was so unpopular in Windows 8 is now recognised as a mistake, he told Vulture South.

In Windows 10, Microsoft came full-circle to let apps be "one window, two windows, whatever, and you can resize them and have them behave more independently".

"Apps ... built for just one interface, that's stifling compared to a fully-fledged desktop client", he said.

The problem in that model is, of course, that at the same time, the developer needs to support platforms from the phone all the way up to the desktop with a 30-inch display, and maintaining performance all the way across the scale is "challenging", he said.

"You start with a Surface Pro and take it down to a phone, and the phone struggles just going from one screen to another," he said.

The key here is using tools – whether in Visual Studio or elsewhere – to monitor application performance, and to do it early. Do the instrumentation early in a project to get a baseline, Randolph advises, so that as developers add features, they can see where they're impacting performance.

Instrumentation is getting more sophisticated, he added, and it's an opportunity to see what real-world users are doing, and use that to make applications better (particularly in the world of the enterprise application). Randolph identified Microsoft's Application Insights, and the tools in Xamarin, as useful in this context.

Instead of merely fetching a bug report from a crash, developers can identify what needs work and what needs improvement.

For example (and don't do this in a way that's obtrusive or privacy-invasive, because someone's bound to notice), if users are taking the "long way around" to navigate to a straightforward and common activity, it's a hint that there's something wrong with the user interface.

Take a trivial example: if users search "help" to "paste unformatted text", and a lot of them are doing it, then you're clearly hiding the functionality away from plain sight.

Learning how to use those tools, he said, is something that an enterprise developer should be able to translate into money: making users happier and more productive can turn the trial download into a sale, this year's user into next year's paid upgrade, and so on. ®

Claimer: No disclaimer is needed. The Register didn't travel to Ignite, but we have lined up some hopefully interesting interviews while it's on. ®

More about

TIP US OFF

Send us news


Other stories you might like