A little while ago, I wrote a post about an article in Science about the relationship between “objective” measurements of “quality of life” and subjective measurements of “life-satisfaction”. The article found a very high correlation between these measurements leading the authors to claim that there was now “objective verification” of the subjective measurements often used in research.
As I pointed out in the previous post, lying just behind the claim of verification was the claim that money can buy you happiness. If quality of life and life satisfaction are highly correlated, then perhaps since quality of life can be bought, so too can life-satisfaction.
At the time I pointed out that there were some problems with the research. I also wrote a letter to Science which I have just been informed they are not publishing. I did, however, get a thoughtful response from the authors who redid some of their calculations to take account of my criticisms. I’m copying below the letter and the authors’ reply for your interest.
Here’s my original letter to Science.
In their report “Objective Confirmation of Subjective Measures of Human Well-being: Evidence from the U.SA.”, (17 December) A. Oswald and S. Wu compared subjective life-satisfaction data of geographical areas across the United States with objective measurements of quality of life in those areas. Since they found a significant correlation between the two measurements, they conclude that subjective life-satisfaction data “contain genuine information about the quality of human lives.”
It is widely known that life-satisfaction judgments are subject to arbitrary contextual factors (1 – 3) and focusing illusions (4). It is therefore crucial that life-satisfaction data are collected in such a way that contextual priming of subjects is either avoided or controlled for.
The life-satisfaction data used by Oswald and Wu come from a the United States Behavioral Risk Factor Surveillance System (BRFSS) – a survey that collects data mostly about personal health. After roughly 100 questions, the survey asks for a life-satisfaction judgment.
Although Oswald and Wu make no mention of the nature of the other questions asked in the survey, their regression analysis does control for socioeconomic factors. They do not, however, control for health factors which make up the majority of the questions asked of the respondent prior to the life-satisfaction question.
Given the literature on the context-dependence of life-satisfaction judgments and the focus of the survey on the respondents’ health, it is to be expected that the their judgments of life-satisfaction would be strongly correlated with their responses about the status of their health. The life-satisfaction judgments are therefore better interpreted as judgments of satisfaction with personal health outcomes.
Thus, Oswald and Wu’s conclusion that subjective judgments of life-satisfaction are strongly correlated objective measurements of quality of life is not warranted. Rather, given the contextual priming of the respondents, the conclusion should be that subjective judgments of personal health satisfaction are strongly correlated with objective measurements of quality of life.
This alternative conclusion, although not uninteresting, is not as surprising as the conclusion drawn by Oswald and Wu.
1. N. Schwarz, F. Strack, European Review of Social Psychology 2, 31-50, (1991)
2. N. Schwarz, F. Strack, H. P. Mai, Public Opinion Quaterly 55(1), 3-23, (1991)
3. S. Oishi, U. Schimmack, S. Colcombe, Journal of Experimental Social Psychology, 39(3), 232-247, (2003)
4. D. A. Schkade, D. Kahneman, Psychological Science 9(5), (1998)
Here’s the authors’ reply.
Response to Slezak’s letter
Andrew Oswald and Steve Wu
We are grateful to the author for his letter. Michael Slezak makes a valuable, interesting point. He is concerned about ‘priming’, namely, about the possibility that people’s life-satisfaction answers are influenced by the exact questions they answered before they reached the life-satisfaction question. He would like us to control for people’s health. It should be said that it is intrinsically unlikely that to do so would affect the nature of any cross-state correlation; this is because Americans in each state answer the same questions before they respond to the life-satisfaction question; hence, put loosely, any ‘bias’ is likely to wash out across states.
But to check this, we have re-done our calculations in the way Slezak suggests. We get the same answer as before.
We re-ran the life-satisfaction regression equation including now (i) an extra independent variable for general health on a self-reported 1-5 scale, as well (ii) one for the number of days in the last 30 days where physical health was not good. Both health-measure variables are statistically significant, as would be expected. However, the two sets of state-dummy coefficients (with and without the health controls) turn out to have a high correlation coefficient of 0.93. Moreover, the new state coefficients with health controls continue to be strongly correlated with the Gabriel state ranking: r = -0.53 (vs. -0.6 that we got originally).
Thus our article’s finding — there is a match between objective and subjective well-being — is unaffected. Nevertheless, we are grateful to Michael Slezak for suggesting this check.
Andrew J Oswald
Professor of Economics
University of Warwick UK
Oswald, A., & Wu, S. (2009). Objective Confirmation of Subjective Measures of Human Well-Being: Evidence from the U.S.A. Science DOI: 10.1126/science.1180606
Schwarz, N., Strack, F., & Mai, H. (1991). Assimilation and Contrast Effects in Part-Whole Question Sequences: A Conversational Logic Analysis Public Opinion Quarterly, 55 (1) DOI: 10.1086/269239
Norbert Schwarz, & Fritz Strack (1991). Context Effects in Attitude Surveys: Applying Cognitive Theory to Social Research European Review of Social Psychology, 2
Oishi, S. (2003). The contextual and systematic nature of life satisfaction judgments Journal of Experimental Social Psychology, 39 (3), 232-247 DOI: 10.1016/S0022-1031(03)00016-7
Schkade, D., & Kahneman, D. (1998). Does Living in California Make People Happy? A Focusing Illusion in Judgments of Life Satisfaction Psychological Science, 9 (5), 340-346 DOI: 10.1111/1467-9280.00066