Hardware is hard(er)
What designers need to know about the technology and business of IoT
As a designer working in IoT, you’ll be able to be a lot more effective if you understand some fundamentals about the technology and business aspects of connected hardware and services.
IoT is a team sport, requiring many types of expertise, and no one person can know all of it. So understanding the concerns of your technical and commercially focused colleagues will help you understand context, be a better collaborator, and create user experiences that are feasible and viable as well as nice to use.
This is a transcript, with slides, of a talk I gave at Future London Academy’s Product Design Week in 2021. I’ve split it into 3 sections:
Why IoT designers should care about technology and business, some key types of IoT product and service, and why hardware is hard(er).
The technology: what designers should know about distributed systems, hardware and data.
The business: what designers should know about value, costs and business models in IoT.
TL;DR: this is how the different domains intersect.
(With thanks to Simon Frost for expert input and reviewing).
IoT designers need to care about tech and business
Today, we’ll be covering technology and business. We’re going to be talking about design, but we’re also not going to be talking about design. We’re going to be talking about the other aspects of product development, which shape design. These are the things that designers don’t always know, but I think should know, in order to be most effective.
First, we’ll look at why doing IoT is hard, or certainly harder than quite a lot of software-only products and services. We’ll cover how to think about IoT UX, and why it's important for designers to understand a little bit about tech and business in this area.
And then we're going to dive into how the tech shapes the UX, and also how the UX should really be shaping the technology. This should be a two-way conversation with your engineers.
And then we’ll look a little bit at the business side. There are some particularly interesting challenges there with IoT. For example, how do you balance the cost of developing products and maintaining them with what the user perceives to be a fair value exchange?
Types of IoT product and service
I want to frame the kinds of products and services I'm going to be talking about, because people tend to think of different things when we talk about IoT.
Connected products
The first category is connected products. These are often what people first think of when they think of IoT, especially in consumer tech. The hardware is very front and centre in the user experience and value proposition. Connected light bulbs are a classic example. Or it might be something like the Elvie breast pump, which I worked on in the early stages of product development. With these products, the user can often interact with the hardware itself, and it might be possible to use the hardware without a separate digital interface at all. But at the very least, the hardware is very visible as part of the product-service system, and it’s where the user thinks the value lies.
Hardware-enabled services
The second category is what I call “hardware enabled services”. With these, the hardware itself is quite low profile in terms of the user experience, the user doesn't interact a great deal with the hardware. One example might be electric vehicle smart charging. You plug in the car of course, but the customer and business value is principally in the service that's enabled by the hardware: optimising charging for times when electricity is cheaper or greener. Very often these services, from a commercial perspective, are fundamentally data products, but ones that couldn't exist without connected hardware.
Digital identities/twins
And then there's a third category. And this is not really about connected hardware at all. This is around digital identities, or twins, for dumb things. A couple of years ago, I worked with a company called EVRYTHNG who have a digital identity platform. It’s used to provide unique digital identities to individual consumer goods. One example: many Ralph Lauren clothes, as they’re produced in the factory, are given a digital identity that can be linked to an QR code (as well as RFID tags). And that allows Ralph Lauren to track goods all the way through the supply chain. It also allows consumers to scan a code when they buy one of these products, and verify whether it’s authentic or not, because the scan will show whether it has a genuine digital identity.
For today, we're going to be focusing mostly on systems that have connected hardware, but the data issues for these digital identity services are really very similar.
Why is it hard?
Why is IoT hard for people who are used to working on pure software or hardware? Working with bringing those two things together is something of a mindset shift, you have all the best bits and also all the worst bits of both worlds. The product development processes for hardware and software look really, really different.
With hardware, there's a high upfront cost to developing something, and you can't change it once you've made it. You have to get it right. The further you go through that process, the more expensive it becomes to change your mind about what you're making, and then you've got standards that you have to adhere to, and they might be different from one country to another. So design and testing and manufacturing is complicated. And mistakes in manufacturing are difficult to resolve. And once you've got it out into the world, people buy it, but then you don't really have much interaction with them. You might have customer support, if someone has a problem or needs to return a device, but that’s it.
But with software, it’s really different. Very often modern software development is about getting something to market as fast as possible, learning from the way that people are using it, and evolving it over time. We use the phrase ‘always in beta’: we know that it's never going to be perfect. We know there will always be bugs. But there’s the expectation that you will continue to improve it, so if you discover that the service is not exactly what people want, or something isn’t right, you change it.
And when you put these two things together, you get an awful lot of moving parts and you get an awful lot of interdependencies. There are there are more things to go wrong, both with the product itself, but also with the development process. And so companies, like white goods manufacturers, for example, who come from hardware backgrounds, suddenly find themselves needing to think about information security, and privacy, and interoperability, and having to maintain cloud services. And that often comes as a bit of a shock. Likewise, when companies who have come from software backgrounds move into hardware, as lots of people tried to do a few years back, it often comes as a shock, just how hard it is to design, manufacture and test a piece of connected hardware. So it's really hard to do both of those things well.
Why does this matter to design?
And why should we as designers, worry about that? What you find when you work in this area is that everything about IoT product development is interconnected. Certainly when I first came into IoT, there were lots of things going on around me – conversations about how much memory to put in a device, or batteries, or routes to market - that I thought were nothing to do with me. But I soon began to realise that actually, they did have quite a radical effect on the experience that the products I was working on were offering. I realised that I had to start understanding some of the other conversations that were going on around me too, in order to participate.
One example, which I'll go into in a bit more detail later, is a conversation like “how frequently can we get a sensor to report data without running the battery down too fast?” or “can we justify putting a more expensive battery in?”. These don't immediately sound like things that are related to digital product or service design, but actually they will shape quite dramatically what data you have to work with and what you can do with it. That goes to the heart of both proposition and experience design.
So all of these things: technology, design, and business, need to be aligned to create good outcome for users. This talk is not about what tasks should fall under whose job description. I’m here to make the case for interdisciplinary working: all of the people with these different areas of concerns, also need to understand a little bit about each other's areas of concerns, and how to work with them. Making an IoT product successfully requires lots of different skill sets to collaborate. And there's just so much to know that no one person is going to be an expert in all of it.
One example: a few years back, I was working for a company who made connected sockets and switches. I was working with their designer who was trying to figure out how they could allow their users to name and describe the sockets and switches in the home. So this is a ceiling light, or this is the TV socket, or this is the socket in the corner of the living room. And he thought that this was something that you could decide just in the app, he thought that was his responsibility. But I noticed the firmware engineer in the corner looking over at us. So I called him over and said “This is what we're thinking about, what do you think might work?” And he said, “Well, I'm actually just coding a naming system into the firmware of the devices”. And so we ended up working on that naming scheme together. But had we left that conversation for another couple of weeks, he would have gone off and done his own thing. He didn’t think to talk to the designer, and the designer didn’t think to talk to him. And it would have then been harder to change that later on.
So you're not going to be an expert in everyone else's jobs, but it's really important that everyone in the team understands just enough about each other is doing, and their areas of concern, so that you know when to involve them. They will often have some quite pressing priority that’s very different to yours: for example, firmware engineers are often very concerned with power consumption. And it's important to understand where they're coming from.
A model for UX in IoT
In that respect, I want to talk about how I think about user experience for IoT. Designers often think in terms of touchpoints, some of which will be software user interfaces: voice, or smartphone apps, for example. And then there’s the industrial design and hardware design of the things themselves.
But there's a bigger picture here. Whenever there are multiple touch points that the user is interacting with - there's a software app, there's a device – then there's another layer to that, which is how those touch points are experienced together: system UX, or interusability. Now, that's something that I'm very interested in and have talked a lot about elsewhere, so it's not going to be the key focus for today. But I will mention a few key points later on.
The next thing underpinning that is what I will loosely call propositions and productisation. Who's this product for? What's the value to them? What's the business value to the organisation that's producing it? How do you build a service around it? How much are customers willing to pay for it? This is the intersection of big-picture experience design and commercial strategy. We'll talk later about what some of those conversations might look like.
And then finally, you have a bunch of operational and technical capabilities that shape your ability to build these experiences on top. This is the stuff that you may be thinking is really not design. And it's not usually the designer’s area of responsibility, but these are still things that shape the user experience. For example, on the operational side, it's things like maintaining service, customer support . On the technical side, which we'll go a little bit more into today, it's things like the capabilities of the devices themselves, how they connect, and how you get data out of them.
Now, this diagram is obviously a designer's eye view. All the design stuff is expanded out, and tech and businesses are oversimplified. If you asked engineers and commercial people to break down their view of this domain, it would look different: they’d have a simpler view of design, and much more to say about their own concerns! But this is a way to start thinking about the world beyond design, and how it nonetheless affects the user experience.
Tech: distributed systems, hardware, and data
Architecture and reliability
So let's start by going into the tech side. We’re going to be looking at distributed systems, systems, hardware and data. We’ll go through a number of things that engineers care about, in no particular order.
On the first one is architecture. The key questions are, what components does this system have? And what code runs where? And the reason that that matters to user experience is because it shapes how things can fail. Reliability will never be 100%. When I used to work just with software, I used to think of happy paths first and failure was an edge case. In IoT, failure is not an edge case: it will happen all the time. The trick is to mitigate the impact.
A key question with architecture is where the code is going to be running. One of the big questions that the engineers will always be concerned about will be whether to do computing functions in the cloud, or on what they call an edge device: i.e. the connected hardware itself. There are various pros and cons to each, and a system might have a mix of both.
To give you a couple of examples: on the left, we've got the LIFX light bulb. And on the right, we've got the Philips Hue bulb. Both of these things have smart rules, so you can set the system to turn on the light at eight o'clock, to make your house look occupied if you're not at home. But the difference between these two systems is that in the LIFX system, those rules are running on a cloud server somewhere. So if for any reason your home internet goes down just before eight o'clock, the server's not going to be able to get that instruction to your lights. So basically, they're just not going to turn on.
In the Hue system, the bulbs don't connect directly to the internet, they connect to a hub or bridge, which then connects out to the internet. So the rules run in the cloud, but they are also duplicated on the hub itself. So if the internet connection goes down, the hub can send out the instructions instead, and the lights will still turn on.
Putting intelligence in the cloud means you can have more processing power and the devices can be simpler and cheaper to make. But if you lose connectivity, it fails. It's a little bit more expensive to build hardware that has more smarts in it, but if you lose connectivity, it can still work.
Lightbulbs are pretty trivial: if the lights don't turn on, it's generally not the end of the world. But in other cases, failure can be really, really critical. An important example of this is vehicle autonomy. If you've got a car that's trying to make decisions about what it can see on the road in front of it and respond appropriately, you don't trust any of that processing to a remote computer over a network connection, because that's too slow and not reliable enough. The car has to be able to make critical decisions on its own in the absence of the network. That’s the other extreme of something that's critically important.
But there are lots of examples of things that are in between in terms of criticality, and engineers will look to find the appropriate balance for that particular type of system.
I want to show you an example of something I worked on a couple of years ago. This is a smart garden irrigation valve, something you can put in the garden to automatically water your plants (it was for a US audience). This system, when it was connected, would run off smart rules in the cloud. You would tell it about the kinds of plants you were growing, you would tell it about where you are, it would look at the weather, it would look at environmental conditions, and it would decide whether your plants needed watering that day and how much to deliver.
The middle screen here shows it's made some calculated predictions about how much it needs to water over the next few days. But putting things in gardens and expecting them to connect to a WiFi network is always a little bit risky. Very often the garden is on the edge of the home WiFi network, so devices lose connectivity quite a bit. So that was factored in to the system design. If this device loses connectivity, it doesn't just give up and kill your plants. It defaults back to a simpler, failsafe, schedule that's controlled on the device itself. So you've got a balance of both cloud-driven and locally controlled functionality.
Networking and connectivity patterns, and power
I've grouped the second and third things I wanted to talk about together. Because although they're quite different from a technical perspective, they're interrelated in their impact on design.
The first one of these is about networking and connectivity patterns. So you've got these devices and these different parts of your system. How are they connecting? And how often are they connecting, and how do the data and instructions move around in the system?
And then the other one is power. And that's technically quite a different thing. But the way that devices are powered, whether they run off batteries, whether they run off mains power, will often shape how often they connect. So if you've got something that runs off a battery, it uses up a lot of power to open up a network connection and maintain it. So typically, they won't be connected all the time, they'll go online and offline intermittently.
And there are two reasons why this matters to user experience.
The first one is latency. Things that happen over a network will not happen instantly. They might happen really quickly, or they might not happen very quickly. But the kinds of delays you get, even quite tiny ones, will often be noticeable to the user. So you've got to think about where those might occur and how you will handle them.
The second one is the qualities of the data that you get: how frequently are you going to get data readings? And when you are, how fine-grained is that data going to be? And that will shape the kinds of insights that you can offer the user and how you'll need to present that data.
This is my car. It's a Kia, and it’s got an app. You can press a button in the app to do things like start the climate control or lock and unlock it. When you do that, something interesting happens. You get this message that says, “I've sent your remote control command”. And then there's a little pause, and then it says “OK, I've locked the doors”.
If you think about interacting with a non-IoT app that's stored on your smartphone and doing everything locally, usually the action of pressing the button generates an immediate response. And the thing you wanted usually happens straightaway. But what Kia have done here is separate out the confirmation that the instruction’s been sent, from the confirmation that the action you wanted has actually happened. The instruction is going from the phone up to the Kia server, and then it gets forwarded to the car. And probably the car isn't sitting there with an open network connection waiting for instructions, because that might use up power, so it might check in periodically. Networking costs power. So a really common trade off is figuring out how often you can afford for the device to check into the network. But assuming that message gets through, the car then executes the command, and then it reports back to the cloud that it has, in fact, locked the doors. And then the cloud has to report back to you. And all this stuff takes time.
It's not too slow there, they've definitely improved things: on the old version of their app that process took about a minute. But still, it's not instant.
If you know basic human-computer interaction, you will know that there are perceptual time limits within which people need to see some kind of reaction or acknowledgement of a UI action from the system. And that's at the level of 0.1 seconds to feel like the response is instant. And 1 second not to feel that the task is interrupted. Very often, connected device interactions will be longer than those acceptable response times. So you've got to figure out ways to handle it. That might mean separating the parts of the interaction which acknowledge the user’s action and tell them you’ve sent the instruction to the car (which you can do quickly), from the part where you tell them that the action has been executed (which depends on latency). It’s not an elegant implementation of it, but that’s what Kia are trying to do here.
The other thing that's really interesting about how people perceive IoT is how our perceptions of computers and networks collide with our perceptions of physical things in the world. We’re used to video calls being laggy and dropping out, and downloads running slowly. But in the real world, we also have millions of years of evolutionary experience that tells us that physical objects respond to our actions immediately and reliably. If you push on a door, it will open. If you go to pick up an object, you know, it doesn't think about it for a couple of seconds before it lets you do that. So when you ask Alexa to turn a light on or something and it takes a little bit of time for those things to happen, it still feels a bit weird and laggy, and it introduces a moment of doubt. Is this thing actually going to happen? It feels a bit broken.
We call this the actuation delay. That moment (or many moments) between you saying “I want this thing to happen” and the networked thing actually doing it.
Sometimes, like the car, you need to be really explicit about the things that are going on. Kia probably thought it was really important that they didn’t give me the impression my car was locked until they could actually confirm that it was. That's understandable. But it makes for a slightly lumpy interaction.
Often, it's just enough to use some nice animations that give people a bit of a sense of progress and remote action. This video shows how my digital product designer colleague Steve Graham and I handled that with the watering device. This is somebody deciding they want to give their plants an extra bit of water, on top of the automated watering. There's this little animation that's covering for the delay while the instruction is being sent out to the device, before we receive confirmation that watering has actually started. And the other thing that we've used here is a bit of an animation in the background, just to show that there's water flowing. If you’re not at home you might not be able to see this, so we thought it was important to show. If you want water, you want to know it’s working, and you want to be reassured that you’ll know when water is flowing if it’s not what you’re expecting to happen. So that's another use for animation: giving people a sense of action at a remote location.
This is an in-home display for a smart energy meter. This shows how much energy is being used in a house, right now. (Some people think this thing is the smart meter, but it isn’t: the actual meter sits under your stairs, or wherever, and sends data to this display).
There are two different views here. You've got these cumulative views on the left, which tell you things like how much energy you've used today, or so far this today, so far this week, so far this month. And all of those show you both gas and electricity. And then you've got this view that shows you live usage right now. And that's only showing electricity: it doesn’t show live gas consumption. Why not?
This comes down to the power and networking issue I mentioned earlier. The electricity meter uses mains power, because it can: it has to be connected to mains power in order to do its job of measuring it! So there are no concerns there about conserving power and it can send a reading every seven seconds, which is basically live power, how much you're using right now. But you can’t run mains electricity next to a gas pipe, there's a risk of explosions, and nobody wants that. So the gas meter runs off a battery. So to conserve battery power, it limits how often it connects to the network and sends a reading only every 15-30 minutes. It isn’t live, you don’t know exactly how much gas you were using at 12.04 or 12.07pm, it’s a cumulative total of how much you used in the last 15 minutes or last 30 minutes. And so that shapes the frequency and the granularity of the data that you get, and the insights you can derive from that.
With live electricity data you can spot things like a sudden spike in usage. If there's a lot of power being used, and you’re not at home, maybe you left the oven on, or something else that's not supposed to be on. So live data gives you things that you can build immediate actions off. You can tell people something's wrong. You can also spot when something didn’t come on that you were expecting. Maybe it’s 10am and Granny hasn’t boiled her kettle for her morning cup of tea yet, perhaps she’s not OK.
A summary of how much gas you used in the last 30 minutes doesn't really give you any immediately actionable insights. But if you view that information over the course of a week or a month, or longer, you start to see patterns. Maybe you’ll see that your heating is coming on at times you don’t need it, when you're not there. Do you really need to be using gas on Saturday afternoon? The meaning you can extract from cumulative data for the user is different, so you need to represent that data to them in different ways than you would for live data.
And then there's another issue. This is about having vague or indeterminate data, and how you handle the fact that actually you don't really know what's going on right now. People often think that having data from a system means you always know what's going on, but sometimes it highlights to you that actually, you don’t.
This is an elder care system. There have been quite a lot of these in the consumer IoT market. This particular one was made by Stannah, who also make stairlifts in the UK, and Ross Atkin Associates. Ross is a brilliant design engineer, he does really great work designing products for older and disabled people. It uses sensors - in this case, it was smart plugs and motion sensors - to monitor the activity of older vulnerable people. So you can see whether granny’s boiling the kettle, or has the TV on, or whatever normal activities are for her. And if she’s not, maybe there's a problem. Sounds good, right?
The trouble is, the problem people expect these systems to solve for them is ”I want to check in on Granny, I want to know she's OK”. And to be alerted right away when there’s a problem. And this is exactly what you can't do with these systems, for a couple of reasons.
First of all, they can't guarantee to identify every time there's a problem. So first of all, you've got to know what's normal for granny. Is it OK if she hasn’t turned the bathroom light on by 8am, or is she just having a lie in? It’s pretty complicated to get the system to reliably identify ‘normal’.
Secondly, if you don’t have data that tells you that Granny is doing something, that might not mean that granny is not doing anything, it might just mean that your sensors are offline, or there's some kind of outage. So you just don't have any data, so you have no idea what’s going on. You can’t alert people that there might be a problem every time there's no data, because that is going to happen from time to time and the false positives will be really annoying. But also, you can’t lead them to believe that things are definitely OK because it might be that Granny boiled the kettle five minutes ago and was totally fine, but now she's had a fall and we just don't know about it yet. So everything looks OK from the point of view from the system, but it really isn’t.
I've worked on some of these products, and you go through a process of thinking “Well, what can we actually tell people? What can we promise that we can tell people?” And it comes down to this question of “Do we currently have any evidence that Granny is not OK”, which is not quite such a compelling value proposition.
So this is just one example to illustrate that the way that you're getting the data, and the reliability of the system, and the frequency with which you're going to get data often means that in an IoT system, you very often will not ever have perfect information about the state of the system. You might know what this device was doing 10 minutes ago, and this other one 5 minutes ago, and this other one two hours ago, but you never have perfect, live data about all of them at once. And you have to figure out how you're going to display that to people. And that bubbles up all the way to the value proposition, and whether you can you really offer them a product that's answering the question they want?
Coming back to the watering product that Steve and I worked on, there's a little example here that shows how we handled indeterminate data. We have these water usage graphs that showed how much you know how much water you’ve been using over various time periods. As I mentioned earlier, sometimes the valve would lose connectivity, which could last for days, but it was designed to continue watering on a failsafe schedule. We had to figure out how to handle the data that it reported when it went back online. And it couldn’t give us the same kind of data we would have got if it was connected, which was exactly how much it watered and when. Instead, it could only tell us how much water it had delivered in total over the entire period it was offline. So we had a graph that was designed to show gallons per day, but we had to accommodate an edge case of a total number of gallons over multiple days. So we had to average it out, and show it in grey to suggest that this was lower quality data, but at least it was there. And so these data issues will create these awkward edge cases that you’ll have to consider in information display.
API design
The final thing I want to cover in the context of tech is API design. You probably know what APIs or application programming interfaces are. They’re like UIs, for devices and parts of the system to talk to other devices and parts of the system.
Some of them will be really low level. You might have an operating system on a device, and one bit of the operating system will talk to other bits of the operating system through an API. Typically, as a designer, you don't need to worry about that stuff.
Others of them will be quite high level. So these will be data and functions that you can pull from a cloud service into a UI, and present to a user. And those are the ones that you, as a designer, will mostly need to care about.
10 years ago, I was the design manager for a connected home platform. This is the old design (it’s not my work). It had energy monitoring, it had a heating controller, and a security system. I was brought in to kind of think about how they could extend the platform design-wise to support a much wider range of future devices and applications. And I was told not to engage too closely with the engineers. My manager didn't want me to get bogged down in the way that things worked currently, because they had plans to rebuild the platform. She wanted me to think about what the ideal user experience should be.
Don't laugh. This isn’t the ideal experience. This was made by my brilliant interaction designer friend and collaborator, Alex von Feldmann. This was super early stage work so not a full representation of Alex's talents at all. These were early wireframes - essentially provocations: intended to use in research to prompt users who’d never experienced a smart home to think about what it would be like to use lots of different connected devices in concert.
Anyway, the engineers asked to see what we were thinking, and it went really badly.
My first mistake was assuming that engineers would know what a provocation was. They didn't get that at all, they just looked at this and went “That's a crap UI, isn't it?” That was the first challenge.
The second challenge was that we hadn't actually realised quite how far away our thinking was from the way that the system was currently built. As the engineers started to look in more detail, they went very quiet. Eventually I managed to get them to start to tell me what the problem was. And they said, “Well, that page is hundreds and hundreds of API calls, to put all of those things side by side, and it's going to take forever just to load, and put immense strain on the back end, and just be horrible, just be totally unresponsive”.
Now, they were well aware of the platform’s limitations and planning to re-engineer it, but that was a big project in its own right, and very complicated and messy as these things are. And they wanted to get a new UI out quickly.
So, that was the day that I learned that your API design is going to have to align with what you want to do with the user experience in order to deliver what you want to be able to do. To try and simplify what was going on, what we had was a bunch of devices, which were each reporting in a single piece of data to a server, where that piece of data could be requested directly by an app. There wasn’t really any processing of the data, or combining of devices, or analytics, going on in between the device and the user-facing app. So if you wanted to say “What’s the total power consumption of all my devices right now?” you'd have to call each device, ask for its power consumption, then do some coding gymnastics in the front end to add all these numbers together. It had made a lot of sense in the early days of the system, with a small number of devices, to build it like this, but it didn’t scale to more complex setups or use cases like the ones we were thinking about.
Whereas what we really wanted was, ways to be able to report data from different combinations of devices, and derive analytical insights: answer more meaningful questions to people, not just report device data. As a very simple example, summarising the status of all the connected things in .the house on the homescreen, so that you could see at a glance whether everything was OK. The moral here is that the structure and the granularity of the APIs need to be a good fit for what you're trying to do with user experience. If they're not, you either won't be able to do what you want, or it'll be really slow, it will be unresponsive, and your engineers will hate working on it. So this is something that needs to be a collaborative conversation.
Tech: summary
The key takeaway from all of this is that what you can do with user experience and IoT is very much shaped and constrained by lots of factors in system architecture.
Also, system architecture should be designed with an understanding of what the end result should be for the user because there will be decision points in that which could go one way or another, depending on what your priorities are.
And so it's it's really, it's really important that those two aspects of design are collaborative.
This is one approach I use. This is an anonymized fragment from a project I worked on recently, looking at creating tools for onboarding connected devices, embedded devices. So how does a user get something out of the box and get it onto a WiFi network, prove that it's your device, claim ownership of it, prove it's your device and you own it?
This was for an embedded operating system that would end up being used in all kinds of devices, so it was a general tool, intended to help device developers create a good onboarding user experience. So it had to work for all kinds of different devices, with different input and output capabilities and network technologies, whatever they might be. So I worked on this abstracted process, collaborating with engineers to figure out both what the user interactions needed to be, but also what needed to happen under the hood to support that, so that what we made was both good UX but also feasible.
We used service design style blueprints to work through those flows, not just from the perspective of the user, but also to represent a slightly simplified view of what was happening technically. So the top line in this diagram is user actions, then below that are things that the app needs to do that the user can see, then the line below is the things that the device does that the user can see. So you turn the device on, and it does something to show that it's powering up, like an LED flash, and then maybe it does something else, like a different flash, to show that it's trying to find a network. You need to separate out those two kinds of status information, because things might go wrong at either stage and if so, the user needs to be able to figure out what the issue is.
And then behind the scenes, you've got these three components to the system where code can run: you've got the app on the phone, you've got the device itself, and then you've got the cloud service. Various things have to happen between all of those. So for example, if the device has turned on for the first time and can't find a WiFi connection, it starts to advertise itself over Bluetooth. And then it has to generate an onboarding key so it can prove who it is to other parts of the system. And then devices have to authenticate to each other and the device phones home to the manufacturer cloud and proves that it belongs to that manufacturer. So there's all this stuff that happens behind the scenes, which you might not want to expose to the average user, but you've got to think about how to handle those things: those things will cause delays, they might sometimes go wrong. So, when you're designing the bits that the user sees, it really helps to have an understanding of what’s going on behind the scenes. If you just mock up what you think the user ought to see in an app without understanding what’s going on technically under the hood, you’ll miss a ton of important stuff.
Through using this approach, we managed to come up with not just the onboarding user experience, but also a network communication protocol for how for how the things should communicate with each other to support that.
Related to that, I also want to make a point about UX prototyping. I believe very strongly that prototyping is not just a process of making screens, you have to get the logic right. I'm old fashioned: I come from a world where we used to make lots of flows and wireframes. I use modern tools like Figma, and they’re good in lots of ways, but I do think that tools like that can encourage designers to dive straight in to fidelity screens, without always thinking the logic through. And this is massively important with IoT, because you miss the other parts of the system, and the logic of how all those interact. So when I do prototyping, I always think about how the different parts will be experienced side by side.
So in this example, we’re trying to get a microwave onto a WiFi network using Bluetooth. And you can see that the microwave LED is flashing because it’s doing something, and it’s able to tell the phone (over Bluetooth) that it’s scanning for WiFi networks, so both devices are synced up and providing good status information for the overall system. And then you go to the next stage where the microwave is telling you which networks it can see (which is not necessarily all the same networks that the phone can see, because WiFi is complicated). And now you use the phone UI to choose one.
So in each stage of that interaction there are different things happening on the two devices, but they’re working together to communicate with the user. Prototyping the app and device interactions together really helps to create a much more coherent experience.
Business: value, costs, and business models
Now, let’s get into some of the business concerns around IoT. This boils down to thinking about value and costs and business models. If you’re a designer, you might be involved in value but perhaps not the other parts. But they determine where you get to play, so it’s well worth understanding what’s going on.
Value
With any kind of paid product or service, there's always some sort of value exchange going on. There's a value to the organisation, generating revenue of some kind. But then there's also a cost to produce that product or service. And then on the user side, where product designers play, there’s the value of solving a particular problem for the user, and their willingness to pay for a product or service that solves that problem. That might be with money, but sometimes it’s also paying with their data, which the business can monetise in other ways.
On user value, the history of IoT, especially consumer IoT, is littered with products that didn't necessarily solve a particularly important problem for people, or didn’t solve it very well. I'm sure many of us can think of one or two. Now the IoT hype has mostly passed, I think the industry is now better at interrogating the value of putting connectivity into things before we actually do so. Mostly 😉
One interesting issue is that even for things that are apparently similar products, the value of putting connectivity into them can be quite different. I once worked on a project for a company who make high end power tools for construction. They were rolling out connectivity into their tools, and we were helping them explore what you could do with this, and how that should shape the web of services you could build around this. On the face of it, if you don’t work in construction, these things are all power tools, and you’d think that the use cases and value of connecting different tools might be quite similar. But we learned that the answer is that they were very much not the same from tool to tool.
The tool on the left is a nail gun. The value of putting connectivity into that seems to be easier invoicing, for people who fit drywall (partition walls, in the UK). They get paid according to the area of drywall that they have fitted, and the number of nails that they fired from the nail gun is really good proof of how much work they’ve done. So basically, you can simplify invoicing for them by putting connectivity into the gun.
The tool on the right is an impact wrench. This is used to tighten anchors on buildings, which are things like big bolts that hold steel building frames together. The value of putting connectivity into those is that you can track exactly how much torque was applied to tighten each anchor. That's really important in areas where there might be earthquakes, because there'll be really strict regulations about how much these things need to be tightened, and the construction company needs to prove they’ve been tightened with the right amount of torque. The old fashioned, non-connected way of doing that is that somebody has to go around and actually check a sample of them, but that’s time consuming and not as thorough, and it’s much more efficient if all your wrenches can report back exactly what was done to each anchor. So this is all about improving building safety. So two apparently similar things, using similar technologies, but two quite different value propositions for the customer.
In terms of business value, the obvious one is you sell things and people give you money for them, which is revenue. And there's often customer data you can capture, from which companies will look to extract further value. So if you're selling people energy monitors, for example, maybe you can mine the data you gather and learn about behavioural patterns, either try to sell them more services, or to sell on to other companies who have an interest in predicting energy usage. Or you use a service ecosystem to try and lock people into buying your hardware, like Apple.
This is the dashboard for a power tool manufacturer’s asset tracking system. So if you’re a construction company, you can keep track of all your tools, like the connected ones we saw. And this is all part of the ecosystem: they don’t just sell you tools, it's a service play, too. You can use this with Hilti’s connected tools. You can also use it with non-connected tools, via Bluetooth or NFC tracking tags which you can stick onto the tools, and you can use it with other people's tools. But ultimately the goal for them is that you buy more and more of their tools, because they work a bit better with this system.
Costs
And then we get to cost. The costs associated with making an IoT product are complex. You've got the hardware costs, and then you've got the software costs.
Take doorbells as an example. If you’re making a conventional doorbell, you've got industrial design, you've got the costs of components and manufacturing, you've got distribution, you've got packaging and manuals to pay for. You’ve got a bit of customer support in case somebody needs to return it.
If you want to make a connected doorbell, it's a lot more complicated and a lot more expensive. You’ve still got all the things associated with the conventional doorbell, and probably there's an expectation that you're going to provide live customer support as the queries will be more complex. Probably people expect a bit more from something like a Ring than they do from a five quid thing they bought from B&Q. And then you've got app design and build, and you've got the electronics, and you've got all your stuff associated with building and hosting and maintaining the cloud service. And you might have some ongoing liability. So even if you stop selling the product, people are still expecting you to run the cloud service. And people are still using it, you might have things like costs for video codecs - this is a real example - that you get charged for every time someone uses it. So as long as people are using that product, even if you've long ago stopped making it, if you want them to be able to keep using it, it keeps costing you money.
This is tripping up an awful lot of IoT product manufacturers, especially in consumer markets, and everyone’s wrangling with this issue of, “well, it costs more and who's going to pay for it?”. And ultimately, that will get shoved onto the customer to pay for in one way or another.
So there are a couple of ways that companies try to resolve this value/cost conundrum through business models.
The first one is you just charge people a bit more for the hardware, and then you provide the service and as far as the user is concerned, it's free. It's not really free, but you know, it looks free. Like my car I mentioned earlier, you pay for the car, but you don’t pay for the app or service. (Although Toyota are now asking users to pay for using their key fobs to start the car remotely, so perhaps even that is changing. I personally think that smarts a bit when the upfront cost of the hardware is as expensive as a car).
Alternatively, you try frame a service in a way that you can get the customer to pay for that on an ongoing basis. And so that's the kind of thing that something like a Ring doorbell will do: they'll try and sell you a coverage plan. There’s a very basic free tier, then there are higher tiers for more video storage and more devices. The big cost for them here will be hosting video, which is really expensive.
Business like services because they get recurring revenue, and retain customers through increasing the friction of switching to someone else’s hardware.
But the challenge comes when a company's products are not things that people think of as services. This is the June connected oven. Do you think of your oven as being a service? Do you think of your doorbell as being a service? Many people don’t, so when you try to sell them a subscription they’re like “What do you mean it wants five pounds a month from me, that's crazy talk!”.
So it can be hard to frame the service in a way that convinces people that they want to pay for it. These are the app membership tiers for the June oven. If you don't want to pay for it, you just get some push notifications. And if you want to go all the way up to the top and you want a live camera feed from your oven, then that's going to cost you $10 a month. I don’t have one of these but people I trust tell me these things are brilliant, and I'm prepared to believe it. But you have to really like using your oven to want to pay $120 a year for live video of your dinner cooking.
If you don’t balance costs and pricing properly, you can really screw up. If the business fails to estimate how much it's going to cost to maintain the service, they could go out of business. And if that means your product no longer works because the service doesn’t work, and then you've got a load of upset people because you sold them a brick. For example, Insteon, the home automation company, recently shut down all services, rendering their devices mostly useless. Or maybe it still works sort of, but it doesn’t do half the things you originally paid for.
And if you don’t want to shut things down or go out of business, then you have to figure out how to say “You know that thing we told you was free? Um, well… actually it’s not anymore.” Which is what Canary did with their security camera a couple of years back. Many people bought cameras at a time when you would get a certain amount of video storage for free. But then that started to get too expensive for Canary. So they said to their customers, “Oh, yeah, all these things we said were free, sorry, you're going to have to pay for them now”. And that caused an immense backlash and lots of very upset customers. So many people protested about it and made so much fuss about it that they had to give some of those things back for free anyway.
And so this whole balance of figuring out what can you offer, how long can you support it, how much is this going to cost you, and projecting all of this stuff is immensely complicated. It’s a somewhat different story in B2B where the idea of buying something as a service tends to be more palatable. But in consumer IoT it's acting as a brake on the market, because companies are struggling to find viable business models and consumers don’t trust that the products they buy will work for their intended lifespan.
IoT is interdisciplinary
So, I hope this has given you some thoughts about how tech, design and business come together around IoT.
With tech and design, it's about the architecture and the UX. So there are UX considerations in system architecture, and there are things in system architecture that will also shape UX, so there's a really important collaboration that needs to happen.
Between design and business, the cost will shape what you can afford to make. But also, the more value you can create, the more you can charge. And that's not necessarily just about kind of screwing your customers over for as much money as possible, sometimes that's about the mismatch between people's assumptions about where the value of the product is, and where the actual costs are in delivering that. So it's about trying to bring those things together so that it's attractive for both sides.
And then tech and business have their own concerns to align. Again, costs will shape what engineering can build and support, but again, the more value you can create from that, upfront and on an ongoing basis, the easier it is to cover those costs.
Understanding how all these parts fit together will help you be more effective as a designer, because you’ll be able to collaborate more effectively with your colleagues to create things that are both feasible and viable. And understanding your colleagues’ areas of concern goes a long way to building good working relationships.
Thank you!
As presented at Future London Academy Product Design Week 2021.