Why the Oxford Study on Teen Screen Use and Well-Being Should be Discarded

Recently the media is buzzing about a new University of Oxford study that seems to indicate there is no significant association between teen digital technology use and well-being. The study by Amy Orben and Andrew K. Przybylski published in Nature Human Behavior, claims that teens using computers and smartphones are no worse regarding their well-being than teens eating potatoes. We can relax now about teens being on their phones all the time thanks to the findings of what some are calling the most rigorous study to date regarding this subject matter. The study says it has a sample size of over 355,000 teens and used progressive statistical techniques.

Well I seemed a little surprised by the study’s findings, so I decided to take a closer look. Here are the highlights of what I learned about this study.

1. All data came from three existing datasets. They did not collect the data themselves.
2. All three datasets collected data using self-reporting surveys. This included questions about technology use and how well they thought they were doing.
3. The first dataset referred to as YRBS collected data starting from 2007, well before smartphone use became commonplace with teens.
4. The YRBS dataset simply asked two questions about technology use: did they use electronics, and did they watch TV.
5. The second dataset referred to as MTF collected data starting from 2008.
6. The MTF dataset asked a few more questions about technology use. It asked about using social media, weekend TV viewing, and using the internet for news.
7. The third dataset referred as the MCS study is regarded as the best quality data by the researchers. This dataset though has a sample size of less than 8,000.
8. The MCS dataset surveyed teens from 13 to 15 years old born September 2000 to January 2001.
9. The MCS study asked only five questions about technology use. These involved questions about watching TV, playing electronic games, using the internet at home, using social media, and whether they owned a computer. Four of the five questions did try to measure time usage by self-reporting time on a 9-point scale.
10. There were also other questions for comparison ranging from sleep, eating breakfast, and eating potatoes.

The media is claiming this study to be exhaustive and rigorous. The study did use a slick way of processing data using a technique called Specification Curve Analysis (SCA). This form of analysis crunches the associations in as many pathways as possible and meaningful given computational limits. So that sounds great, but I consider the datasets to be very high level and low quality since the questions are general and only self-reporting surveys were used. In an era where computer use is somewhat ubiquitous, the general nature of the questions (e.g. do you own a computer) leads to noisy datasets. So, a lot of computer time was used for analysis, but as the saying goes, garbage in, garbage out. You cannot squeeze out nuanced truth from very high-level questions, no matter how much you crunch the data.

Aside from the low-quality datasets, this study has a fundamental flaw in its design. The surveys only measure technology use and well-being at a point in time for an individual. This method cannot understand the impact of technology use exposure over the duration of time. The real question to ask is how much technology use over how much time impacts the development and well-being of adolescents. Surveys at a single point in time cannot answer this question. The MCS dataset, which is considered the best dataset of the three, only looks at 13 to 15 years old in the UK. It would have been better to follow these children through middle school, high school, and college, continuing to monitor technology use in much more detail with real measurement on types of content, and then measure real mental health events (not self-assessment from survey).

If you look close at the results of the associations discovered, the nature of the noisy datasets starts to be revealed. As mentioned previously, other questions unrelated to technology use were asked for baseline comparisons. These other associations were compared to technology use. For example, from the MTF dataset, binge-drinking was found to be 8.1 times more a negative impact on well-being than technology use. Bullying is 4.92 times from the MCS dataset. Eating potatoes is 0.86 times which is close to 1 and thus the statement that technology use is as safe as eating potatoes. But looking close at the data, you see that being arrested is 0.96 times. So, you could say that technology use has the same impact on well-being as being arrested. Music has 32.68 times positive impact on well-being. Going to the movies has 11.51 times positive impact. Those numbers for music and movies seem completely off. That is what happens when you have noisy datasets. These association numbers can be all over the place. Getting enough sleep can have as much as a 44.23 times positive impact on well-being. Getting enough sleep was asked by all three datasets. What is interesting is that the associations vary from 1.65 times to 44.23 times for sleep across the three datasets. This again is noisy datasets rearing their ugly head. And if you are a parent of a teen, you know that technology use can impact sleep of adolescents. How then does sleep have such a huge positive impact, but technology use has virtually no impact? Bullying in today’s context is largely taking place on-line. How then does bullying have a 4.92 negative impact, but technology use has very little?

Unfortunately, we need to wait for better research to answer the question of technology use and well-being. In the meantime, if you are a parent of a teen, please use your common sense. Here are some points to consider.

1. If your teen is playing video games excessively where it impacts his or her sleep, eating habits, socialization, and getting school work done, get professional help for your child. Video game addiction is now recognized by the World Health Organization as a real disorder.
2. Talk to your child about on-line bullying. If you think your child is being bullied on-line, then you need to help your child deal with that.
3. Make sure your child does not go to bed with his or her smartphone. Sleep is very important. Teens will communicate via social media all hours of the night if you let them.
4. Consider using technology that blocks access to porn sites. Boys are viewing a lot of porn and we have no idea the impact this has on their view of women and expectations of normalcy in intimate relationships when they grow up to be men.
5. Talk to your teen about distracted driving. Put some tangible rules in place about phone use and driving. And as a parental role model, put your phone away when you drive.
6. Consider eating dinner together as a family and not allowing TV or phones during this time. Your kids will complain at first, but the whole family will come to value this most basic of human bonding experience.