How do people interact with biofeedback data?
Problem Space
We live in an overly stimulated world. With hundreds of chimes, vibrations and notifications each day I know that I personally ignore the vast majority because I do not have the time or energy to engage with each one. This oversaturation of notifications is one of the many problems facing designers that seek to change human behavior through digital programs. This problem becomes even more intriguing when we consider how people react to notifications about events happening in their own bodies. This is the problem space that I am going to explore in the rest of this blog post.
Overview
In this blog post I am going to explore some existing products that use bio signalling embedded in wearable devices to either inform users or attempt to change their behavior directly. I will then dive into the literature and see how scientific studies have attempted to answer this question.
Exemplar Review
Apple Watch and FitBit Blood Oxygen Sensors
Fowler, G. (2020, September 23). Review | The new Apple Watch says my lungs may be sick. Or perfect. It can't decide. Retrieved September 25, 2020, from https://www.washingtonpost.com/technology/2020/09/23/apple-watch-oximeter/
Overview
The new Apple Watch 6 and the FitBit Sense are both offering blood oxygen readings as one of the newest features on their wearables. The two companies use similar technology to medical grade blood oximeters by shining light into the skin and measuring the color of the blood to determine how oxygenated the blood is. The main difference between the FDA approved devices that doctors use is that they are placed on the fingertip where there is a very large amount of blood vessels very close to the surface of the skin, whereas the smart watches are worn on the top of the wrist where it is much more difficult to get a blood oxygen reading. Now why would you even want to know your blood oxygen level? Well in case you didn’t know there is a bit of respiratory virus going around. So in the past 6 months the desire to know your blood oxygen levels has peaked during the pandemic because it is one of the key metrics doctors monitor in Covid19 patients in the hospital.
Of course I am sure that this is something the companies were going to put in their new devices, pandemic or not because with each new iteration of a device companies feel like they need to add some new feature for the marketing team to push.
Fowler, G. (2020, September 23). Review | The new Apple Watch says my lungs may be sick. Or perfect. It can't decide. Retrieved September 25, 2020, from https://www.washingtonpost.com/technology/2020/09/23/apple-watch-oximeter/
Now while these two companies use the same technology, they offer the information to the user in different ways. Apple offers daily averages as well as spot checks, while Fitbit only offers a night time average. I will discuss both information delivery experiences and why I think FitBit has gotten it at least slightly more right.
So we will start with Apple because Apple always gets to go first. They offer two ways to get your blood oxygen levels, as an average or as a real time reading. Initial testing done by Geoffrey A. Fowler, a tech columnist at the Washington Post showed that these real time readings from the Apple watch can differ wildly from reading to reading even when done back to back. This error rate remained high when compared to an FDA approved medical grade oximeter.
In line with the Apple watches daily averages the FitBit’s nightly averages had much less variability, but they still varied more than the FDA approved device.
Critique
While I do agree that letting people know more about the functioning of their own bodies is in general a good thing and can be used to make healthy decisions. I think that as designers we have to be wary about what information we give people and how.
First let's think about who is interested in their blood oxygen levels. There are three main groups; athletes, people with health problems and healthy people. Now giving athletes a way to track their training progress is allowed to have a decently high level of error because if the readings are wrong the user might eat a pizza when they should have eaten some quinoa. However, the other group is the one that concerns me with these devices. Those suffering from medical conditions, who might think that they can rely on readings given to them by their wearables to stay on top of a chronic lung condition.
Now imagine a person with emphysema and a comorbid heart condition seeing an add for the new apple watch. Deciding that it would be cheaper and easier to just monitor their blood oxygen levels at home on their watch then having to go into the doctor's office. They get the watch and measure their blood oxygen levels and get a deceptively low reading showing that they might be close to lung failure. This could cause a panic in the patient and with a pre existing heart condition they have a heart attack and die. One can imagine many other scenarios, including being lulled into a false sense of security by a false high reading.
Takeaways
Marketing should be straightforward and clear about the limitations and intended uses of health related technology to avoid people misusing the devices. Furthermore, once people have the devices they should be warned of potential misuse and informed of the potential error rate. Transparency is the only ethical option for pseudo medical tech devices.
Using long time interval averages to allow for outlier elimination can allow for a more accurate reading when using non ideal data, such as blood oxygen levels from the top of the wrist.
Fowler, G. (2020, September 23). Review | The new Apple Watch says my lungs may be sick. Or perfect. It can't decide. Retrieved September 25, 2020, from https://www.washingtonpost.com/technology/2020/09/23/apple-watch-oximeter/
Awake Labs
Understanding Stress Using Wearables. (n.d.). Retrieved September 25, 2020, from https://awakelabs.com/
Overview
Understanding Stress Using Wearables. (n.d.). Retrieved September 25, 2020, from https://awakelabs.com/
Awake Labs is a company using the heart rate monitor of a smartwatch to help monitor the emotional status of people with Autism Spectrum Disorder (ASD).
It works by creating a baseline average heart rate for participants over a training interval. Then with the help of the caretaker thresholds are set so that when a person with ASD’s heart rate goes above a certain BPM the application connected to the watch will send an alert to the caretaker. In an ideal situation the caretaker could then intervene and alleviate the source of the anxiety for the person with ASD. The overall goal of the project is to prevent people with ASD from being stuck in an anxiety inducing situation with no way to properly communicate their discomfort, while at the same time giving care givers an early alert system so they can head off potential panic attacks.
Critique
I do really like the idea behind this design, however after reading through the research I realized that the claim necessary for their product to work, that people with ASD have a predictable heart rate response to anxious situations. Might not be backed up by the studies that they say it does. In the paper Investigating the Autonomic Nervous System Response to Anxiety in Children with Autism Spectrum Disorders by Azadeh Kushki when they talk about their results they say, “ASD group showed elevated heart rate during both baseline and anxiety conditions.” and then conclude with “These results suggest that signals of the autonomic nervous system may be used as indicators of anxiety in children with ASD...”
This confusion of elevated heart rate to me means that the application could be sending notifications to the caretaker every time that the person with ASD watches a YouTube video that they find exciting or sees a cute animal that they want to play with. All normal and potentially non anxious situations that do not need intervention. Thus, after a number of false flags will the caretaker begin to ignore the notifications rending the application useless?
Lastly people with mental difficulties carry with them a stigma and when designing treatments for them we want to make sure that we as designers take this into account. I think it was smart that Awake Labs used commercial smart watches for this project so that there is nothing immediately identifying them as medical devices, but if all of the people with ASD are wearing the same device and no one else in the group wears one then that commercial watch becomes a dividing symbol of who is there to receive treatment and who is there to give treatment.
Takeaways
Make sure that if you are creating a product that is meant to be a treatment that the hardware you give people is either very discreet or ubiquitous so that it cannot become a symbol of disability or illness and therefore a target of stigma.
When you make claims about the efficacy of a product and then point to a scientific paper to back up your claims it would be good to have a testimonial from that researcher saying your product works, but at least explaining in simple language what the science says.
Emotiv MN8
Workplace Entreprise Solutions. (2020, July 29). Retrieved September 25, 2020, from https://www.emotiv.com/workplace-wellness-safety-and-productivity-mn8/
Overview
Emotiv is a brain computer interface company that I have been watching for quite a few years now and even tried applying to work at when I first graduated from college. They specialize in consumer grade EEG technology and their most recent product is an interesting one. It is a pair of Bluetooth earbuds with two EEG sensors built in designed to measure a person's level of anxiety and focus.
This seems like it could be a helpful thing for people like myself who struggle to stay focused on my work and could use a nudge every once in a while. However, that is not how it is being marketed by Emotiv. They are promoting it as an in office tool to increase worker productivity. As shown in the illustration below Emotiv is showing how an increase in stress can cause a decrease in worker productivity and with their EEG data companies can implement tools to make sure that workers remain calm and productive. From the wording of both images I provided here it is obvious that this is not meant for employees, but for managers.
Critique
Workplace Entreprise Solutions. (2020, July 29). Retrieved September 25, 2020, from https://www.emotiv.com/workplace-wellness-safety-and-productivity-mn8/
I think that having a wearable that can measure stress and focus could be a great consumer product even if it does lack precision because it could give users insights into their own emotional patterns.
What scares me is the way that this product seems to be designed for the enterprise level. While they try to lead with wellness and safety it still seems like the ultimate goal here is productivity.
As shown above that means office workers wearing headsets and then that data being sent to their managers for evaluation. That data could be used in productive ways to spot employees who are struggling and might need additional help to feel comfortable in the office environment. However, out of all of cubical farms that I have worked out over the years I can just imagine this data being used to even more heavily monitor and discipline employees for “slacking off”, taking away the last place that some office workers feel like they aren’t being watched, their minds.
It seems like either Emotiv built this for an idealized world where only companies who truly want the best for their employees will buy this product, or they knew what they were doing and wanted to use their EEG technology as a brain babysitting device to help managers squeeze every last drop of productivity out of workers.
Takeaways
When designing a product that offers new forms of insight into the human body, make sure that you think through how that new information can be used against the individual who is being monitored.
If you truly do not want your product to be used to control people make sure to build personal data encryption and protection so that only the user can see their data and decide what to do with that information.
Literature Review
Wearable Sensor/Device (Fitbit One) and SMS Text-Messaging Prompts to Increase Physical Activity in Overweight and Obese Adults: A Randomized Controlled Trial.
In this study the authors test if text messaging prompts based off of data from a wearable activity tracker can increase the physical activity of people who are overweight.
They hypothesized that since it had been proven that text messages are a good motivator for people attempting to quit smoking or to maintain a diabetes insulin regimen, that a similar approach could be used to help people who are overweight increase their physical activity and increase their overall health. So to test this they create a double blind study consisting of two groups with around 20 individuals each. They then tracked them over a 6 week period to see if the thrice a day text message reminders would increase the amount of physical activity they engaged in. To do this they set up automated text services to text participants at three times throughout the day and then had the participants upload their activity data at the end of the day each day.
Wang JB, Cadmus-Bertram LA, Natarajan L, et al. Wearable Sensor/Device (Fitbit One) and SMS Text-Messaging Prompts to Increase Physical Activity in Overweight and Obese Adults: A Randomized Controlled Trial. Telemed J E Health. 2015;21(10):782-792. doi:10.1089/tmj.2014.0176 (Fig2)
In the end the group that got text reminders did see an initial increase in physical activity during the first week of use, but returned to baseline with the comparison group by week 2 and stayed there until the end of the study. The authors theorized that this could be because after the first week spike the text message group grew tired of the texts, which they classified as too frequent and automated in a follow up survey, and stopped paying attention to them. Furthermore, the exit survey found that the group that did not receive the text reminders actually engaged more with their personal FitBit data and had a slightly higher average amount of physical activity.
To me these results fall in line with the basic tenets of stimulus reaction in the brain. There are two options when the brain is constantly presented with the same stimulus you either become hypersensitive to the stimulus, which typically means it terrifies or it annoys you, or you learn to ignore it. I think that many of us can think of an instance in our own lives when we have walked into a room full of people with a horrible smell and wondered how anyone could be in there and minutes later sit down having completely forgotten the smell.
Back to the study and its relevance to my topic, it confirms what I already feared, that translating digital notifications into real world behavior change is going to be a very difficult task. However, it has also given me a decent purview of what does not work and how sometimes control groups can offer just as much insight as the study group.
Wang JB, Cadmus-Bertram LA, Natarajan L, et al. Wearable Sensor/Device (Fitbit One) and SMS Text-Messaging Prompts to Increase Physical Activity in Overweight and Obese Adults: A Randomized Controlled Trial. Telemed J E Health. 2015;21(10):782-792. doi:10.1089/tmj.2014.0176
Literature Updates
10/2/2020
A meta-analytic study of self-help interventions for anxiety problems
This paper sought to determine the effectiveness of self administered anxiety treatment, through a meta analysis of available literature on the subject in 2005 when the paper was written.
The method of meta-analysis is one that I think is employed at the beginning of almost every one of my design projects, granted not in such a scientific way as this paper. However, a good competitive analysis is something that I need to do shortly to see what the field looks like since last I checked.
The results of the paper reaffirm my hypothesis that self help methods are significant in their efficacy, but not as effective as in person therapist delivered interventions.
This analysis helps add data to back up my assertion that in home self administered anxiety interventions can have a significant impact on overall anxiety level. I will continue to build on this as I progress throughout my project.
A meta-analytic study of self-help interventions for anxiety problems. (n.d.). ScienceDirect.com | Science, health and medical journals, full text articles and books. https://www.sciencedirect.com/science/article/pii/S0005789406000128?casa_token=8kNmvIbBhBUAAAAA:Oa7WFKqqRZ4lcKC_o56pP3uZhzxVd9KoA7rse4hCGAt7C9b-ikMfnRSg1RMt-Iyn6NtGmHaPnhE
10/9/2020
Toward Preventing Post-Traumatic Stress Disorder: Development and Testing of a Pilot Predeployment Stress Inoculation Training Program
In this paper the authors created a set of breathing techniques that when coupled with biofeedback showed promise for preventing PTSD in military personnel.
This is useful for my capstone becuase it shows how the coupling of biofeedback and breathing techniques can not only help people who are axious become less anxious in the moment, but that these techniques can be preventative in nature as well.
Lastly, Gregory Lewis is my professor for wearable sensors and he was a part of this paper so he can give me the inside scoop on how they went about designing the study and the breathing techniques for biofeedback.
Laurel Hourani, PhD, MPH, Stephen Tueller, PhD, Paul Kizakevich, MS, Gregory Lewis, PhD, Laura Strange, ANC GaARNG (Ret.), Belinda Weimer, MA, Stephanie Bryant, MS, Ellen Bishop, MS, Robert Hubal, PhD, James Spira, PhD, Toward Preventing Post-Traumatic Stress Disorder: Development and Testing of a Pilot Predeployment Stress Inoculation Training Program, Military Medicine, Volume 181, Issue 9, September 2016, Pages 1151–1160, https://doi.org/10.7205/MILMED-D-15-00192
10/16/2020
Towards a ubiquitous care model for patients with anxiety disorders
Just a disclaimer, this paper was written in Portuguese so I used Google translate to read it and as with any translator that creates significant room for error.
The authors of this study decided to test using a biodata wearable wrist band in conjunction with a gamified anxiety reducing prompt to help patients with anxiety disorders. Throughout the study they found that patients who used the “iAware” gamified treatment did in fact have fewer anxious episodes. While the authors did include their game architecture models, they did not explain what biodata they were using from the wrist band to determine when the patient was experiencing an anxious episode. So I sent the author an email in the hopes that he would be able to elucidate that point for me.
This study gave me a good working model for what a successful game architecture could look like for helping patients with anxiety.
Lucas Pfeiffer Salomão Dias and Jorge Luis Victória Barbosa. 2019. Towards a ubiquitous care model for patients with anxiety disorders. In Proceedings of the 25th Brazillian Symposium on Multimedia and the Web (WebMedia '19). Association for Computing Machinery, New York, NY, USA, 141–144. DOI:https://doi-org.proxyiub.uits.iu.edu/10.1145/3323503.336064210/23/2020
Somaesthetic Appreciation Design
Patricia actually sent me this paper so shout out to her.
It showcases a smart home system for deep breathing and meditation that uses lights to communicate to people what their breathing looks like and helps talk them through slowing it down and focusing on parts of their body that are holding tension.
I was actually already thinking about using the concentric circle design to pulsate with the persons breathing, but I still need to figure out a way to present people with their breathing rates in a way that do not cause them to become stressed by the rapidity of their breathing and potentially cause them to hyperventilate.
Kristina Höök, Martin P. Jonsson, Anna Ståhl, and Johanna Mercurio. 2016. Somaesthetic Appreciation Design. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). Association for Computing Machinery, New York, NY, USA, 3131–3142. DOI:https://doi-org.proxyiub.uits.iu.edu/10.1145/2858036.2858583
10/30/20
Designing Interactive Clothing to Raise Awareness of and Comfort The Wearer Suffering from Anxiety
The authors of this paper talked about a line of clothing created to help people with anxiety cope. They focused on two aspects of the the persons experience. The comfort of the garment and how they help the person remain calm and grounded, as well as what the garments can communicate to others. They did this by creating pleasant experiences for the person wearing the garments such as having a personalized pleasent sent pouch embedded into the collar of the turtleneck sweater. While in order to communicate with others they created subtle color changing icons on the jacket to help friends and ally’s know how the person was feeling at any given time, “The neutral meanings of the symbol ensure that only people who know the wearer well enough understand what it means.”
I will use these ideas of creating comfort for the person with anxiety while also creating some sort of non obvious way to communicate the feelings of the user to their friends and ally’s. I think the use of color changing like a mood ring is a well understood concept and could be utilized in many different ways. Also creating materials and other physical sensations that help strengthen the digital interventions I think could be incorporated into any type of wearable sensor.
Tyana Hendriksma, Joanne Maartense, Rosanne de Feyter, Rong-Hao Liang, and Loe Feijs. 2020. Designing Interactive Clothing to Raise Awareness of and Comfort the Wearer Suffering from Anxiety. In Companion Publication of the 2020 ACM Designing Interactive Systems Conference (DIS' 20 Companion). Association for Computing Machinery, New York, NY, USA, 25–29. DOI:https://doi-org.proxyiub.uits.iu.edu/10.1145/3393914.3395884
11/6/20
BioFidget: Biofeedback for Respiration Training Using an Augmented Fidget Spinner
The BioFidget is a really interesting design concept that solves many of the issues faces by Biofeedback devices. It does not require the user to wear a new sensor, but rather gathers data during the play process through one of the best heart rate measuring zones on the body the tips of the fingers. It also can be used to measure and guide breathing in a physical way by having the user blow on the wings of the spinner. I wonder if there is a way that you could pair a peripheral device like this to a smart watch or a more long term sensor device to create a more accurate system.
The authors also described the framework that they used to create their biofeedback system, “To guide the users to perform right direction of interaction at the right time, the visualization should clearly indicate the current state (i.e., mode), what to do next (i.e., feedforward), and the effects caused by the performed action (i.e., feedback).” This model can be easily adapted to almost any form of biofeedback device. Making sure that there is a clear and easy path forward for users is vital to ensure that the feedback that users get does not overwhelm or underwhelm them. You want to make sure that they stay engaged yet not overstimulate them and I think that this narrative is key.
Capstone Reflections
10/2/2020
This week I decided to exclusively focus on people with anxiety disorders as my user group. I decided on this group because of two things; A) anxiety disorders are the most prevalent mental health issue in the world and B) anxiety is often directly linked to physical responses such as an increase in heart rate, breathing and perspiration all relatively easily measured biosignals.
I then put myself into the shoes of a person with anxiety using a self help biofeedback device to see what parts of the process could use the most attention. I found that the area with the most room for improvement was the actual alerts themselves, because how the alerts and prompts attempt to engage with the user will greatly affect how the user reacts. Whether they decide to engage and stay engaged or whether they ignore the messaging.
Lastly, this week I created a group in my wearable sensors class in which I am going to be testing and hopefully creating the anxiety alert wearable device. I teamed up with an electric engineer so he can help me with the technical side and I recruited my professor to help because his own research interests marry so well with my project that we can work in conjunction so with that extra help I am excited to see what we can get done.
10/9/2020
This week I was able to discuss how a breathing game could be created without further increasing the anxiety levels of the person playing the game. I was also able to figure out what exactly how biofeedback works.
In my group we talked about what we want to get out of our capstones. Both me and Vignesh have decided that we want to pursue projects that have real world marketability because we want to be able to help people with our designs. However, we have also conceeded that if we fail to produce something real we can at least turn the research and development that we will and have done into something of an academic paper so that people in the future can know at least one way that something doesn’t work.
10/16/2020
This week I has my desk crit session with Dana and she gave me some suggestions about the direction that I should take my capstone. I am exploring these options and will be pivoting my research as needed.
In preparation for the desk crit I also did some research into the commercial competition in the wearable anxiety reduction space. There seem to be two companies (Feel and Healium) that are doing a good job in this space right now using purely heart rate levels to determine if someone is having an anxious episode. Thus, I believe if I can focus on two things to differentiate myself.
1) Create a sensor system that is more reliable, specifically using heart rate variability and ECG instead of just simple heart rate. This way there are fewer false flags because often times simple heart rate can spike for reasons other than anxiety.
2) Create a therapeutic interaction that is both non intrusive and engaging at the same time. Something that people can do quickly, but also something that people want to keep doing.
If I can manage to do these two things well I think that I can create a truly helpful and value laden product and experience for people.
10/23/2020
This week we had our pin up presentations. It was nice to see what everyone else was working on and seeing how their ideas have changed throughout the semester. It is also always nice to see how people present their ideas because presenting design ideas can be a tricky task.
I received some good critique about my design:
1) That the color contrast of my design not be enough for accessibility standards.
2) That the circle within a circle was great, but might need visuals other than color to represent change for those that are green colorblind for example.
3) That there should be more of a conversational back and forth, as if a friend trying to calm you down from an anxious state.
I am definitely going to take the conversational idea and integrate that into my design. I like the idea of having a friend on your wrist. One that will coach you through your breathing exercises.
10/30/2020
This week I spent a good deal of time playing with the ECG electrodes and the PyBadge that we were given to work with for my wearable sensors class and just thinking about how something so technological and medical in nature can be packaged in such a way that it is comforting. In relation to the article that I mentioned this week there is also a company https://reflectinnovation.com/ that has creating a meditation ball wrapped in fabric with hand shaped divots for meditating. These different example have gotten me thinking about what material properties can influence the way that people interact with and experience digital products created to help them calm down. It reminds me of a cruel study where monkeys who were separated from their mother bonded better with a monkey mannequin wrapped in cloth than a bare metal one. Showing that at a primal level every sense matters when it comes to our emotions.
11/6/2020
This Tuesday was the beginning of the presidential election and the entire day Wednesday I was on edge and the only thing that I could compare it to was, I felt like I had just drank a cup of coffee the entire day. It was not until Wednesday night when I went on a bike ride that I realized that feeling that I had was anxiety. I know it seems strange that I have spent so much time studying anxiety while I myself am not an anxious person, but getting an insight into how distracting and unpleasant being in a constant state of anxiety can be really helped reinvigorate my passion for this project. I now can directly understand why a prolonged anxious state can cause so many physical and mental health issues because I felt my body and mind being strained that entire day on Wednesday while I wrestled with the possibility that Trump could have another four years to plunge us into an even darker future.
Overall, I think this experience was a good one because I can now empathize with the people I want to design for and I can think back to that day when I ask myself the questions of will this help or just make someone more anxious?
