ClearFactAll topics

Should AI systems be prohibited from inferring latent psychological traits (personality, cognitive abilities, mental health status) from behavioral data without explicit consent?

5 peer-reviewed studies for, 5 against

Key takeaways

Stronger evidence: For

Verdict

This is partly a values question, but the evidence favors requiring explicit consent. On the "for" side, Kosinski, Stillwell and Graepel (PNAS, 2013) showed that Facebook Likes alone predict sexual orientation, political views, personality and intelligence, and Stachl et al. (PNAS, 2020) found that routine smartphone logs predict personality about as well. Matz et al. (PNAS, 2017) showed such inferred traits can be used to tailor ads to millions of people, and reviews by Burr and Cristianini (Minds and Machines, 2019) and Martinez-Martin et al. (npj Digital Medicine, 2018) conclude that existing rules do not adequately cover consent, privacy and accountability for these inferences. On the "against" side, Eichstaedt et al. (PNAS, 2018) and Coppersmith et al. (Biomedical Informatics Insights, 2018) show real health benefits, detecting depression and suicide risk from social media. Eckles, Gordon and Johnson (PNAS, 2018) argue the persuasion effects of trait targeting may be overstated, Azucar et al. (2018) found prediction accuracy only moderate, and Barocas and Nissenbaum (2014) argue consent itself is a weak safeguard because inferences learned from consenting people apply to everyone. On balance the "for" side is stronger: sensitive inference from ordinary data is demonstrably feasible, and the main benefits shown so far came from people who consented. The against side's best points argue for exceptions, such as clinical or emergency uses, and for protections beyond consent, rather than for allowing unconsented inference.

Strongest study on each side

Ranked by citation count.

Strongest study for

Private traits and attributes are predictable from digital records of human behavior (PNAS, 2013)

Using Facebook Likes from over 58,000 volunteers, a simple model predicted highly sensitive attributes including sexual orientation, ethnicity, religious and political views, personality, intelligence, happiness and substance use. It distinguished gay from straight men in 88% of cases and Democrats from Republicans in 85%, and predicted Openness nearly as well as a standard personality test retested on the same person. The authors discuss the privacy implications of such inferences from ordinary online behavior.

Strongest study against

Facebook language predicts depression in medical records (PNAS, 2018)

Among 683 emergency department patients, the language of their Facebook posts identified those later diagnosed with depression with accuracy (AUC 0.69 to 0.72) roughly matching standard screening questionnaires, and up to 3 months before the diagnosis was recorded. Depression is widely underdiagnosed, and the authors see such inference as a scalable complement to existing screening.

Open the interactive page →

The evidence

For (5)

Private traits and attributes are predictable from digital records of human behavior (PNAS, 2013)

Using Facebook Likes from over 58,000 volunteers, a simple model predicted highly sensitive attributes including sexual orientation, ethnicity, religious and political views, personality, intelligence, happiness and substance use. It distinguished gay from straight men in 88% of cases and Democrats from Republicans in 85%, and predicted Openness nearly as well as a standard personality test retested on the same person. The authors discuss the privacy implications of such inferences from ordinary online behavior.

Psychological targeting as an effective approach to digital mass persuasion (PNAS, 2017)

Three field experiments reaching over 3.5 million people used personality traits inferred from digital footprints to tailor ads. Ads matched to a person's extraversion or openness produced up to 40% more clicks and up to 50% more purchases than mismatched or generic ads. The authors warn that inferring psychological traits at scale makes it possible to influence large groups, raising risks of manipulation and privacy violations.

Predicting personality from patterns of behavior collected with smartphones (PNAS, 2020)

Machine learning on 25 million passively logged smartphone events from 624 volunteers (calls and messaging, music, app use, mobility, phone activity and day-night patterns) predicted Big Five personality traits with accuracy similar to social-media-based predictions. The authors conclude that private traits can be extracted from data phones collect routinely, posing serious threats to privacy and enabling psychological targeting.

Data mining for health: staking out the ethical territory of digital phenotyping (npj Digital Medicine, 2018)

Digital phenotyping uses smartphone and wearable signals to measure cognition, mood and behavior. It currently runs with informed consent in research but is expected to spread into healthcare and consumer apps, creating new categories of mental health risk data. The authors find that existing ethical and regulatory frameworks do not clearly cover it and call for safeguards on transparency, informed consent, privacy and accountability.

Can Machines Read our Minds? (Minds and Machines, 2019)

Reviews a large body of studies in which machines infer emotions, aptitudes and skills, values, sexual orientation, personality, and conditions such as depression and addiction from samples of online behavior. Concludes that machines can now access this kind of psychological information and that this raises urgent ethical issues of consent, privacy, and the use of persuasive technology to steer behavior.

Against (5)

Facebook language predicts depression in medical records (PNAS, 2018)

Among 683 emergency department patients, the language of their Facebook posts identified those later diagnosed with depression with accuracy (AUC 0.69 to 0.72) roughly matching standard screening questionnaires, and up to 3 months before the diagnosis was recorded. Depression is widely underdiagnosed, and the authors see such inference as a scalable complement to existing screening.

Natural Language Processing of Social Media as Screening for Suicide Risk (Biomedical Informatics Insights, 2018)

Shows deep learning on social media posts can detect signals of suicide risk and describes a screening system usable by non-specialists such as primary care doctors. Because it currently runs only for people who opt in, the authors note that potentially life-saving technology reaches only a fraction of those at risk, and ask whether the current trade-off between privacy and prevention is the right one.

Field studies of psychologically targeted ads face threats to internal validity (PNAS, 2018)

A critique of the 2017 psychological targeting experiments. Because the ad platform decided who saw each ad, people shown different ads were not randomly assigned, so the reported gains in clicks and purchases cannot be cleanly attributed to personality matching. The letter cautions that the persuasive power of trait-based targeting, a key harm cited for restricting such inference, may be overstated.

Predicting the Big 5 personality traits from digital footprints on social media: A meta-analysis (Personality and Individual Differences, 2018)

Pooling studies that predict personality from social media data, this meta-analysis found moderate accuracy, with correlations of roughly 0.29 to 0.40 depending on the trait. Such predictions are far from mind-reading: they are informative at the group level but error-prone for any individual, which tempers claims that AI can reliably expose a specific person's psychology.

Big Data's End Run around Anonymity and Consent (Privacy, Big Data, and the Public Good, Cambridge University Press, 2014)

Argues that informed consent cannot protect privacy in big data. People cannot meaningfully understand what may be inferred from their data, and once a few people consent, models trained on them can draw the same inferences about others who never agreed. The authors argue protection should instead depend on whether a given flow and use of information fits its context, implying a consent requirement alone would be a weak safeguard.

The debate

Maya: Machines can now read sensitive traits from data people never meant to share. Kosinski and colleagues showed in PNAS that Facebook Likes alone predicted sexual orientation correctly 88 percent of the time, and political party 85 percent. Nobody consented to revealing that.

Leo: They predicted it at the group level. Look at the meta-analysis by Azucar and colleagues: personality predictions from social media correlate only about 0.3 to 0.4 with real test scores. That's useful for statistics, but it's not mind-reading.

Maya: It doesn't have to be perfect to be used. Matz et al. ran field experiments on 3.5 million people and found ads matched to inferred personality got up to 50 percent more purchases. Imperfect inference was still good enough to change behavior at scale.

Leo: Eckles, Gordon and Johnson challenged that in PNAS. The ad platform chose who saw which ad, so people weren't randomly assigned. The headline effect may be much smaller than claimed.

Maya: Even so, the data keeps getting richer. Stachl et al. predicted personality from ordinary smartphone logs: calls, music, app use, location. People carry that sensor all day without knowing what it reveals.

Leo: And that same capability saves lives. Eichstaedt and colleagues found Facebook language flagged depression as accurately as screening surveys, up to three months before diagnosis. Coppersmith's team showed suicide-risk detection is feasible, and they ask whether requiring opt-in is costing lives.

Maya: Notice who those studies used: people who consented. The benefit doesn't require secret inference. Martinez-Martin, Insel and colleagues, who want digital phenotyping to succeed, still call for informed consent, transparency and accountability before it spreads to consumer apps.

Leo: But consent is a weak shield. Barocas and Nissenbaum point out that once a few people agree, the model can make the same inferences about everyone else. A consent rule can give a false sense of protection.

Maya: That's an argument for stronger rules, not none. Burr and Cristianini conclude these inferences raise urgent issues of consent and manipulation. Requiring explicit consent is the minimum.

Leo: Fair, as long as there are clear exceptions for clinical care and emergencies, and limits on how inferences can be used even with consent.

Maya: Agreed. Consent first, plus limits on use.

Frequently asked questions

Can AI tell your personality from your online activity?

Research shows models can predict personality traits from Facebook Likes and from routine smartphone logs. A meta-analysis found the accuracy is moderate, so predictions are more reliable for groups than for any single person.

Can AI detect depression or suicide risk from social media?

One study found the language of Facebook posts identified patients later diagnosed with depression about as well as standard screening questionnaires, sometimes months before diagnosis. Other work shows social media posts can signal suicide risk, though such screening currently runs only for people who opt in.

Do ads targeted to your personality actually work better?

Field experiments reported that ads matched to people's inferred extraversion or openness produced more clicks and purchases. A published critique argues that because people were not randomly assigned to ads, those gains cannot be cleanly credited to personality matching.

Is consent enough to protect people from AI inferences about their psychology?

Some scholars argue it is not, because people cannot fully understand what may be inferred, and models trained on those who consent can be applied to those who did not. Reviews also find existing rules do not clearly address consent, privacy and accountability for these inferences.

Related questions