social psychology’s crisis of confidence

A recent NYT Magazine article has prompted colleagues and friends alike to ask me, what’s going on in your discipline? Perhaps you’ve heard that there’s a “crisis” in social psychology. It’s been covered prominently–e.g.,  NYT, AtlanticSlateWikipedia. This essay is my attempt at explaining.


The present crisis in social psychology can be traced to two highly publicized events in 2010 and 2011—publication of impossible findings using accepted methods of rigorous psychological science (Bem, 2011; Simmons, Nelson, & Simonsohn, 2011), and cases of fraud, notably Diederick Stapel (Finkel, Eastwick, & Reis, 2015; Yong, 2012). These events prompted numerous special issues on methodological rigor, replication, and transparency (e.g., Ledgerwood, 2016; Stangor & Lemay, 2016), large-scale efforts to replicate findings in flagship journals (Open Science Collaboration, 2015), and ominous commentaries from leaders of the field (e.g., Kahneman (2012), “I see a train wreck looming”). The current crisis echoes that of prior decades (Elms, 1975; Gergen, 1973; McGuire, 1973), but has notable differences (Hales, 2016; Spellman, 2014). First, I discuss how common research practices undermine our ability to make valid inferences. Second, I elaborate on why the field is grappling with these issues, and how the current crisis differs from those of the past. I conclude with recommendations for moving forward.

Common (and “Questionable”) Practices

Many research practices in social psychology (e.g., selectively reporting a subset of measures used) have long been recognized as “questionable” because they increase false inferences (e.g., Greenwald, 1975; Rosenthal, 1979). Yet, these practices remain surprisingly common (John, Loewenstein, & Prelec, 2012), due to perverse incentives, norms, or lack of awareness (Nosek, Spies, & Motyl, 2012). Many questionable practices are justifiable sometimes (particularly when reported transparently), though all of them increase the likelihood of false inferences (Nosek et al., 2012 for review). Here, I focus on the practice I see as most central to the current crisis.

The principle common research practice to the present crisis is opaque and misleading reporting of researcher degrees of freedom (Simmons et al., 2011). Researcher degrees of freedom are the set of possible methodological and statistical decisions in the research process. For example, should outliers be excluded? Which items should be used? It is rare, and sometimes impractical, to have a priori predictions about how to make all, or even most, of these decisions. Thus, it is common practice to explore alternatives after seeing data. In a given dataset, slightly different alternatives can lead to vastly different conclusions, and there may be no objective justification for taking one alternative over another (Gelman & Loken, 2013). For example, imagine a test that is non-significant when data are log-transformed, and significant when they are truncated. These two approaches may be equally justified for skewed data. However, we often rationalize in favor of alternatives that meet our expectations, in this case, statistical confirmation of our hypothesis (John et al., 2012). There are many other biases that lead us to favor positive alternatives (e.g., motivated reasoning or hindsight bias). Recall Richard Feynman’s advice to Caltech’s class of 1974, in science “the first principle is that you must not fool yourself – and you are the easiest person to fool.”

Furthermore, bias-prone decisions compound to exacerbate false inferences, even when decisions are seemly bias-free. By way of analogy, imagine the research process is a garden of forking paths. Each fork in the path represents a decision (e.g., truncating data), which eventually leads to an outlet (representing the conclusion). The long and winding path taken through this labyrinth may be justified by scientific logic at each juncture. However, because there are so many junctures, it is improbable that any two scientists (or even the same scientist a year from now) would take the same path through the garden. Deviation at a single fork can lead to disparate outlets, because new decisions are informed by data that were altered by previous decisions (Gelman & Loken, 2013). This is how 29 research teams can examine the same dataset with the same hypothesis, and come to 29 different conclusions (Silberzahn et al., 2017). When decisions are not determined a priori, they are inevitably guided by data and biases that influence the validity of inferences.

Research degrees of freedom increase the likelihood of false inferences, however they do not intrinsically undermine scientific progress. Nonetheless, it is not only common practice to maintain flexibility in design and analysis (Gardner, Lidz, & Hartwig, 2005; Ioannidis, 2005), it is also common to publish results as if only a single path was explored, or even as if a single path was predetermined (Begley & Ellis, 2012; Bem, 2003; Giner-Sorolla, 2012). Such presentation makes it challenging to distinguish between confirmatory (more reliable) and exploratory (more tentative) research. Without reliable representation of the current evidence, it is difficult to determine the degree to which an effect is understood and valid, as well as where to place future research efforts. The regularity of many researcher degrees of freedom accompanied by opaque or misleading reporting is central to the current crisis.

Why we are Reeling

Social psychology is grappling with a crisis (again), because formerly theoretical concerns about replicability (Elms, 1975; Gergen, 1973; McGuire, 1973), have been made tangible by empirical findings (Bem, 2011; Simmons et al., 2011) and fraud (e.g., Stapel)—both of which received considerable attention beyond ivory towers. A Google News search of “replication crisis and social psychology” reveals over 7,000 articles in the last few years including prominent outlets such as NYT, BBC, and WSJ. Scholars agree that outright fraud is a problem, but a rare one, and thus, not a primary concern. In contrast, questionable research practices are concerning because they are so common (John et al., 2012) and can result in impossible findings (Simmons et al., 2011). Many point to Daryl Bem’s (2011) paper on “precognition” as the catalyst of the present crisis. The paper, published in JPSP, appears to show that people have extrasensory perception. The distinguished Lee Ross, who served as peer reviewer, said of it, “clearly by the normal rules that we [used] in evaluating research, we would accept this paper… The level of proof here was ordinary. I mean that positively as well as negatively. I mean it was exactly the kind of conventional psychology analysis that [one often sees], with the same failings and concerns that most research has” (Engber, May 2017). Bem empirically arrived at an improbable conclusion (ESP exists) using common practices for entry into our flagship journal. This prompted Simmons and colleagues (2011) to use the same common practices to conduct an experiment that came to an impossible conclusion (that listening to certain songs can change the listeners’ age). These events led many social psychologists to question common practices, and revisit theoretical concerns of the past.

This Time is Different

The current crisis echoes that of prior decades (Gergen, 1973; McGuire, 1973), even centuries (Allport, 1968; Schlenker, 1974), in that it is concerned with replicability (Stangor & Lemay, 2016)—and rightfully so. The transparent communication of methods that enables scientific knowledge to be reproduced is the defining principle of the scientific method, and perhaps the only quality separating scientific belief from other beliefs (Nosek et al., 2012; Kuhn, 1962; Lakatos, 1978; Popper, 1934). Just as replicability is a sign of a functioning science, so too may be the perpetual self-conscious grappling with claims for scientific status. Psychologists and philosophers of science have long debated the scientific status of social psychology (Schlenker, 1974). In fact, such self-critical angst can be traced to the historical origin of the discipline when we differentiated ourselves from philosophy (Danziger, 1990). Yet, there are notable differences between the “crisis of confidence” in the 1970s (Elms, 1975), and that of today.

First, the former crisis was largely characterized by concerns about external validity, whereas today’s crisis in primarily concerned with threats to statistical conclusion validity (Hales, 2016). For example, McGuire (1967, 1973) worried that our focus on the “ingenious stage manager” of the laboratory produces conditions that render null results meaningless and positive result banal, while at the same time being unlikely to replicate outside the laboratory. Another example is found in Gergen (1973), who argued that social psychological effects are hopelessly dependent on the historical and cultural context in which they are tested, and thus impossible to generalize to principles in a traditional scientific sense.

In contrast, today’s crisis is concerned with the validity of statistical conclusions drawn from an experiment (Hales, 2016). Instead of asking, “does the effect generalize?” We are now asking, “does the effect exist at all?” In the previous crisis, Mook (1983) famously argued in defense of external validity. Laboratory experimentation need only concern itself with “what can happen” (as opposed to “what does happen”). It is the theory tested by a particular experiment that generalizes, not the experiment itself. A compelling defense, however, the assertion rests on the validity of statistical conclusions. The contemporary crisis is grappling with the assertion that common practices not only demonstrate “what can happen,” but that they can be used to show that “anything can happen.” If anything can happen in our laboratories, what differentiates our science from science fiction?

A second way in which the current crisis is different is related to changes in technology and demographics (Spellman, 2014). Technological changes are eliminating space concerns, and increasing speed and transparency of communication. One consequence of which is that people who fail to replicate research can more readily share that information, and see that they are not alone. Thus, it is easier to be critical of the finding itself rather than assume a methodological mistake was made (McGuire, 1973). Similarly, increases in diversity of the field have precipitated more critical questioning of the status quo. In brief, today’s crisis has elements of a social revolution that were missing from prior crises (Spellman, 2014). These factors will fuel a more persistent push for change this time around.


I conclude with recommended changes to improve confidence in our science. In fear of presumption, I follow McGuire (1973) in submitting my suggestions as koans—full of paradox and caveat; they are intended to be at once provocative and banal.

Koan 1:“Does a person who practices with great devotion still fall into cause and effect?…No, such a person doesn’t.”


In 2000, the National Heart Lung and Blood Institute (NHLBI) initiated a policy requiring all funded pharmaceutical trials to prospectively register outcomes in an uneditable database, After the policy went into effect, the prevalence of positive results reported in NHLBI-funded trials dropped from 57% to 8% (Kaplan & Irvin, 2015). Preregistration improves confidence in published findings because it reduced selective reporting. More broadly, preregistration makes researcher degrees of freedom more apparent, reduces opaque and misleading reporting (Nosek, Ebersole, DeHaven, & Mellor, 2017), and allows us to better distinguish between confirmatory and exploratory research (Nosek et al., 2012).

Koan 2: “Having our cake and eating it too.”

Explore Small, Confirm Big

There is growing recognition that “small sample sizes hurt the field in many ways” (Stangor & Lemay, 2016), because it undermines both statistical confidence and the perception of rigor (Button et al., 2013). However, there is a trade-off to reckon with—it is resource expensive and unreasonable to test all hypotheses with large samples (Baumeister, 2016). We can have our cake and eat it too if we instead explore new questions with small samples to determine which are worth putting to larger confirmatory tests (Sakaluk, 2016). True, so long as we call a spade a spade. Small-N studies should leave the reader with the impression that the effect is tentative and exploratory, and then attempt to confirm “big” (Baumeister, 2016; Dovidio, 2016). Though, there is disagreement over implementation. Should there be separate journals for small-exploratory and large-confirmatory studies (Baumeister, 2016)? Should those studies appear in sequence in the same paper (Stangor & Lemay, 2016), or in different sections of the same journal (Dovidio, 2016)? My contention is that any of these approaches will be better than the status quo, so long as “truth in advertising” is maintained.

Koan 3:“He who pays the piper calls the tune.”

Gatekeepers and Replicators

Editors and reviewers tacitly agree that replicability is foundational to confidence and scientific progress, yet few journals incentivize replication. A recent study found that, of 1151 psychology journals reviewed, only 3% explicitly stated that they accept replications (4.3% of 93 social psychology journals; Martin & Clarke, 2017). If researchers could be assured that replications get published, more would be conducted. However, what makes for a constructive replication is widely debated. A promising approach is to test hypotheses as exactly as possible, while simultaneously testing new conditions that refine and generalize (Hüffmeier, 2016). Publishers must provide carrots to replicate, preregister, increase sample size, etcetera, or, as Nosek and colleagues suggest (2012), let us do away with them. Make publishing trivial and engage in post-publication peer review, they say. This allows researchers to decide when content is worth publishing and shifts the priority of evaluators to methodological, theoretical, and practical significance, and away from apparent statistical significance. Registered reports prompt a similar shift by enabling results-blind peer review (Munafò et al., 2017). Publishers could act as managers of peer review, focusing solely on bolstering confidence and rigor in the process, instead of also engaging in dissemination, marketing, and archiving. This is a worthy and feasible objective in the internet age (Nosek et al., 2012).

Koan 4: “What is the way? …An open-eyed man falling into the well.”


The ultimate solution to our confidence dilemma is openness (Nosek et al., 2012). Make more information from our studies available. Preregistration helps make the research plan transparent, but the field would also benefit from changing norms around sharing and archiving data, materials, and workflows (Simonsohn, 2013; Wicherts, Bakker, & Molenaar, 2011; Wicherts, Borsboom, Kats, & Molenaar, 2006). More transparency not only addresses fabrication, it also enables verification, correction, and aggregation of knowledge—all of which bolster confidence in (and progress of) science. There is concern that greater transparency unveils the messy complexity and conflicting evidence of our science. That it enables science deniers and other malevolent critics in their efforts to mislead the public. To this I say, “fools believe and liars lie,” regardless of truth or access. In my admittedly optimistic view, earnestly open presentation wins confidence in the long run. For example, scientists who concede failures, explore reasons for failure, or are transparent in their publication of failures (as opposed to denying their validity, hiding them, or not acting) are perceived as more able and ethical (Ebersole, Axt, & Nosek, 2016). Scientists overestimate the negative consequences of a failed replications and transparent reporting (Fetterman & Sassenberg, 2015).


The present crisis is not entirely new, but it has critical difference. If we can use common research practice to find the impossible, where does that leave our science? I venture that these koan may move us to embrace our science not as history entirely (Gergen, 1973) but perhaps as evidence-based history. So too, in the style of Rozin (2001), may we start to embrace the exploratory and narrative nature of our present science. Perhaps then, we will again find our confidence.

References (click here)



Can Theory Change What it is a Theory About?

In Beyond Freedom and Dignity B.F. Skinner writes, “no theory changes what it is a theory about; man remains what he has always been.” By this Skinner means that the underlying rules or processes that guide human behavior are constant, and that knowledge of these processes does not change their nature. However, throughout the social psychological literature we see suggestions of just the opposite—knowledge of a psychological process can change the psychological process. For example, Schmader (2010) provides evidence that simply teaching people about stereotype threat may “inoculate them against its effects.” The theory of social identity threat postulates that people are sensitive to contexts that threaten their identity, and when such a situation is detected people engage in ruminative conflict that can distract them enough to undermine their performance in that setting. Schmader is claiming that giving people knowledge of psychological processes predicted by theory changes the processes that unfold. This point raises several important questions: what is a psychological theory? Does psychological theory describe stable processes in the Skinnerian sense? Can we think of psychological theory in the same way that we think about theories of say physics or biology? If we believe theory must have some element of stability (e.g., if we believe light traveled at the same speed in the middle ages as it does today), and that theories exist out side of and are independent from our knowledge of their existence (e.g. the theory of special and general relativity existed before Einstein identified them, and his discovery did not change their quality), then can we classify social psychological theories as theories? My sense is no. Or maybe we need to modify our definition of what qualifies as a theory. Or perhaps our definition of stability in the processes that underlie phenomena and our belief that observation is independent from underlying processes needs modification.


Schmader, T. (2010). Stereotype Threat Deconstructed. Current Directions in Psychological Science, 19, 14–18. doi:10.1177/0963721409359292



Temporal Self-Regulation Theory: Why we keep trying (and failing) to go for that early morning run.

keep-the-dream-alive Last night in a burst of optimism I set my alarm for 5:30 AM. I thought I would sneak in an early morning run around the neighborhood before work. But as bells rang at that un-godly hour, I cracked an eye to a dark, cold room and groped for the snooze button. Ten minutes later, with a slight increase in clarity, I delayed once more “today, sleep is more important”…snooze again. As you might have guessed, I didn’t wake up in time to run.

We’ve all had a similar experience. Our preconceived intentions to engage in healthy behavior too often fail to come to fruition when it’s time to act. But we also intuit that our intentions are somehow linked to our behavior.

Most of the prevailing theoretical models of health behavior such as the Theory of Planned Behavior (Ajzen & Madden, 1986, request pdf), posit that intentions in combination with a number of other factors, such as behavioral beliefs, can predict likelihood of behavior. And these theories do predict behavior reasonably well (see Godin and Kok, 1996), but they fail to explain why large increases in intention only lead to small changes in behavior (see review by Webb and Sheeran, 2006). In this way these theories fail to fully explain health behavior.

Hall and Fong (2007), developed Temporal Self-Regulation Theory to help explain why, when it comes to health-related actions, the intention–behavior link may break down. They postulate that perhaps our intentions sometimes fail to lead to behavior because,

[many health behaviors are] associated with a characteristic set of contingencies whose valence changes dramatically depending on the temporal frame.

I’ve added emphasis to the quote to help break it down. In generally when psychologists talk about behavioral “contingencies” they are referring to if-then conditions that create potential for the occurrence of certain behavior and its consequences. Using the running example above, one behavioral contingency could be stated, “if I run in the morning, then I might be healthier when I’m older”. The “valence” of this contingency is positive—who doesn’t want to live a long and healthful life? “Temporal frame” refers to the very human capacity to think not only in the present moment or short-term, but also to weigh long-term consequences of our actions. Our example contingency has a long-term orientation. The authors contend that valence of the contingency changes with temporal frame, so let’s say I am thinking in the short-term, the behavioral contingency could then be stated, “if I run in the morning, then I might be tired for the rest of the day”. This is, of course, negative in valence. So the theory predicts that I will be more likely to create an intention to run in the morning if I’m focussed on the long-term as opposed to the short-term. This helps explain why it’s so hard to engage in health protective behaviors (such as running) and dis-engage in health risk behaviors (such as smoking). It is hard to delay gratification and most health risk behaviors are satisfying in the short-term and unsatisfying in the long-term, while most protective behaviors are predominately unsatisfying in the short-term and satisfying in the longer-term .

So, back to why my intention to run in the morning failed to lead to running after the alarm went off.

Last night when I set my alarm for 5:30 AM I was thinking about my long-term health, “I’ll look and feel so good in my summer swimsuit after working out” or “I’ll be less prone to disease when I’m older”.  Further the immediate costs of setting the alarm were low—I only had to click few buttons. In contrast, while reaching for the snooze, the costs of running were more immediate and the short-term consequence were salient, “I’m tired now, and I’ll be too sleepy to be productive today if I run”.

These tables and figures from Hall and Fong (2007) demonstrates how protective and risky health behavior have the opposite contingency valence with respect to time orientation. As depicted in the table 1, participants in this study estimated the point in time at which they would notice the benefit/cost of health protective behaviors (e.g. exercise and dieting), and health risk behaviors (e.g. smoking and drinking).

temporal proximity measure Hall and Fong (2007) Sticking with our morning run example, Figure 1 below demonstrates that people don’t notice the cost of running when thinking about rising at the crack of dawn for a run (question #1) or when deciding to run by setting the alarm an hour early (question #2). We start to feel the cost when the alarm goes off and we have to get out of bed and dress (questions #3). The perceived cost continues to grow as we run and after we’ve successfully run once (questions #4 and #5). We start to feel the cost less once we’ve made this morning run a regular routine for a week (question #6). As we continue to engage in our morning run routine the perceived cost continues to decrease, completely disappearing after a several years (question #9).

Now, what about the benefit of running early in the morning? Figure 1 indicates that we don’t feel the benefit of our run until we’ve done it regularly for a week (question #6), at which point the benefits grow exponentially for a year (question #8) and then decreases toward zero as we approach a decade (question #10).

These results provide evidence that the perceived benefit of running occurs well after the initial behavior occurs, while the perceived cost is felt just before, during  and a short while after the behavior initiates.

So when we are making the decision to set the alarm early for tomorrow’s run costs are low and abstract, so we are focusing on the long-term. When the alarm goes off and we are engaging in the behavior the costs are high and concrete, so we are focusing on the short-term. 

Before looking at Figure 1 below, notice that numbers 0 through 9 on the x-axis correspond to questions 1 through 10 in Table 1 pictured above. This is because academics like to make things more complicated than they need to be :). Screen Shot 2014-03-20 at 9.00.55 AM Figure 2 shows that the same trend holds for another health protective behavior (dieting). Screen Shot 2014-03-20 at 9.01.08 AMAs expected, the authors found the opposition result for health risk behavior—costs come after engaging in behavior and benefits occur before/during, see Figures 3 and 4 below. Screen Shot 2014-03-20 at 9.01.19 AM Screen Shot 2014-03-20 at 9.01.31 AM

So how does Temporal Self-Regulation Theory help me running in the morning? It suggests that on thing that might help is to try to minimize the short-term costs and maximize the short-term benefit. This can be hard, but may be as simple as rewarding yourself with a favorite breakfast if you complete the morning run.

Obviously, perceived temporal proximity with regard to behavior is only part of the picture. The authors introduce a working model (below) to illustrate Temporal Self-Regulation Theory more fully, which I’ve enhanced with definitions of each component. The model introduces two factors, behavioral prepotency and self-regulatory capacity that (1) influence (or moderate) the link between intentions and behavior; and (2) directly influence behavior in the absence of intentions. Health behaviors are complex and theories require continuous testing and refinement but Temporal Self-Regulation Theory adds an interesting new component to existing theories that is surely worth further consideration and testing.

Enhanced schematic representation of Temporal Self-Regulation Theory


Ajzen, I., & Madden, T. J. (1986). Prediction of goal-directed behavior: Attitudes, intentions, and perceived behavioral control. Journal of Experimental Social Psychology, 22, 453-474.

Godin, G., & Kok, G. (1996). The theory of planned behavior: A review of its applications to health-related behaviors. American Journal of Health Promotion, 11, 87-98.

Hall, P. a., & Fong, G. T. (2007). Temporal self-regulation theory: A model for individual health behavior. Health Psychology Review, 1(1), 6–52. doi:10.1080/17437190701492437

Webb, T. L., & Sheeran, P. (2006). Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence. Psychological Bulletin, 132, 249-268. doi: 10.1037/0033-2909.132.2.249


Links That Tickled Me

  1. Nate Silver’s revamped FiveThirtyEight blog made it’s debut this week. His post, What the Fox Knows, explaining the impetus behind their mission and the value of “data journalism” is worth a read.
  2. @econtalker had a heated, but interesting conversation with Jeffery Sachs on the Millennium Villages project. Jeff responds to episode with Nina Munk on her book critical of the project’s impact, The Idealist: Jeffery Sachs and the Quest to End Poverty. Healthy skepticism of foreign aid is a good thing, but my take from these episodes is that the Millennium Villages project has measured, positive impact. This said it is clear that further evaluation from impartial party is required.
  3. I’m drudging through a couple thick and juicy papers on self-regulation:
  • Hall, P. a., & Fong, G. T. (2007). Temporal self-regulation theory: A model for individual health behavior. Health Psychology Review, 1(1), 6–52. doi:10.1080/17437190701492437
  • Mann, T., de Ridder, D., & Fujita, K. (2013). Self-regulation of health behavior: social psychological approaches to goal setting and goal striving. Health Psychology : Official Journal of the Division of Health Psychology, American Psychological Association, 32(5), 487–98. doi:10.1037/a0028533

What I’m looking for in a graduate advisor. And why it’s good for science.

Applying to doctoral graduate programs is an arduous, time-consuming, and often ambiguous process. Testing, essay writing, and networking aside, it’s hard to identify the right program/person to spend the next 5-7 years with. I spent countless hours reading up on promising schools. And once I selected the faculty members conducting research that peaked my interest and matched my background, I needed to find out if they were even considering students. University websites are often out of date, and some faculty are unclear with their future plans and expectations. So I, and many other bright-eyed students, send thoughtful emails and cross our fingers on a reply:

“Dear Dr. So-And-So, I LOVE your work on [blank], and it is related to research I have done on [blank] in Dr. [Blank’s] lab. Are you taking students next year?”

A little more transparency would not only save everyone a lot of time and anxiety, it could determine the future of a science. Professors must receive dozens of these emails every Fall. And I imagine many students ask about out-dated research agendas based on the out-dated websites. It’s understandable that professors often fail to post updates on their activities and respond to these emails. Their time is being pulled in many directions, and expectations are growing. But, as Ben A. Barres (2013) argues, strong student-advisor relationships are integral to the continued success and innovation of a scientific field. Too often students never receive a clear answer and so spend a lot of time and money applying to programs completely in the dark. Prospective students pray their faculty member of choice is actually considering students for the next year, still working on topics listed online, and not going on sabbatical or reducing the size of their lab.

A breathe of fresh air. I came across several professors and programs that posted clear information on what they expected out of doctoral applicants. Here is the best example-A short blog post saving prospective students the time and money of applying to a non-match, and professors/administrators the time sifting through emails and applications not directed toward their current and future research agenda. The post made it immediately apparent whether or not this faculty member was a good match for me.

So in the interest of transparency, time-saving, and sustained success of the field, here’s what I am looking for in a graduate advisor. 

Note: I used Barres (2013) as a guide and focus on my field of interest, psychology.

Are they a good scientist?

  1. How many publications do they have? How many are recent? How many are in my area of interest?
  2. What is the impact of their publications on the field (h-index)?
  3. Are they publishing research (not just reviews) in top journals (i.e. are they innovators in their field?)
  4. Has their lab or center recently secured major grant funding, such as NIH, NSF, NIMH, etc.

Are they a good mentor?

  1. Get in touch with your prospective mentor’s current students and ask them questions about their mentor. Make sure you are in a space where they can answer honestly.
  2. Do they spend time with students discussing science? Good mentors spend time with their students designing good experiments, interpreting/analyzing data, writing research papers and grants, reviewing papers for journals, and practicing talks for conferences.
  3. Do they encourage students to engage in activities (that may be outside of their research interests) that are good for the student’s training? Activities such as TAing, attending conferences, and taking summer courses or workshops.
  4. Is there room to develop your own ideas or are you a slave to faculty research?
  5. Are they aloof, a micro-manager or somewhere in between?
  6. Is there a team spirit in the lab/center, where people collaborate effectively and are not pit against each other in a fight for attention, resources or scholarly success?
  7. Are lab meetings group discussions in which everyone contributes their thoughts and ideas, or is it primarily a time where the faculty member lectures or dictates to presenters what they should do next?
  8. What is the Postdoc to PhD student ratio in the lab/center? A high ratio might be an indication that your prospective mentor doesn’t see mentorship as a priority.
  9. How big is the lab/center? If it is relatively large it might indicate that your prospective mentor doesn’t have the time to give you individual attention.
  10. How many joint-publications and first-authors do their current students have?
  11. Ask for their CV if it is not available online.
  12. Ask for a list of the faculty’s former students. Find out what these students are doing today. Are they still in research? How successful are they? Are their achievements something you aspire toward?

Are their research interests similar to mine?

A point about this final question worth noting before diving in. I will quote Barres (2013) directly because he just puts it so well:

“An advisor should not be selected solely because he or she is the one researcher at your university that happens to work on the precise focused topic that you think you are most interested in. […] In my experience, this is exactly what nearly every graduate student does! Keep in mind that if you like solving puzzles, as all scientists do, there will be many different puzzles that you will find equally rewarding to work on. […] Begin your search for an advisor by casting as broad of a net as possible.”

Ok, now I’ll throw my broad net:

  1. Do they conduct experiments or studies that explore the etiology of health behavior, disease, or illness?
  2. Are they interested in development or evaluation of real-world health interventions or programs?
  3. Do they use or have an interest in developing research or interventions that use mobile or internet-based technologies?
  4. Do they employ diverse methodologies? Do they collaborate across the disciplines of psychology, public health, sociology, or economics?
  5. Are they interested in one or more of the following topics?: Health Behavior Change, Theory-Driven Psychology Interventions, Health Promotion, Disease Prevention, Emotion Regulation, Health Message Framing, Obesity, Exercise, Nutrition, Built Environment, Decision-Making, Mindfulness, Adverse Events or Trauma, Stress, Psychophysiology, Methodology, Technology for Health Research, Vulnerable or High-Need Populations.

I hope this post provides some useful suggestions for students applying to graduate programs. Please feel free to add ideas in the comments. I also hope this post underlines the importance of transparency and openness in science. With the advent of internet-based technologies, a move toward clarity, free-flow of information, and open communication will help science continue to flourish in the 21st century. And it might ease the migraine-inducing match-making process for students and faculty alike.


Barres, B.A. (2013). How to pick a graduate advisor. Neuron, 80 (2), 275-279. doi:10.1016/j.neuron.2013.10.005

Is “Emerging Adulthood” a new developmental stage?

Jeffery Arnett (2000) proposed a new development stage, “emerging adulthood” to better describe individuals aged 18+ who are not yet independent and don’t think of themselves as adults. Arnett proposed this term in place of “early adulthood” which is traditionally used to describe people 18-25 years old. Arnett argues that early adulthood is often an inappropriate classification because it implies that these individuals have achieve adulthood despite showing dramatically different characteristics (e.g. not ready for children, live with parents, etc.).

The primary argument against emerging adulthood as a stage of development claims that it is not universal. In defense of Arnett’s theory, Nelson et al. (2004) attempt to show that emerging adulthood may be present in other cultures, but conceptualized in different way. They examined emerging adulthood as a cultural construct by exploring Chinese student conceptualization of adulthood. The authors attempt to compare these conceptualizations to a mainstream United States student population in an attempt to show that emerging adulthood is a construct influenced by culture.

The major weakness of Nelson et al. (2004) lies in the method implemented. The authors compared US to Chinese students of the same age, however failed to gather evidence that the younger and older Chinese populations differ. It would have been more fruitful to compare discrepancies between two age groups within each cultural group. For example, the article suggests that some items suggest that Chinese students share attributes of U.S. emerging adults, the question “How certain are you about your religious/spiritual beliefs?” revealed that only 6.3% of students were certain in their belief system. The authors suggest that this may be an indication that Chinese students of this age are still exploring in a similar way to U.S. students. This claim is unwarranted, as a direct comparison between Chinese students and the older population (to examine the uniqueness of this phenomenon within the culture) was not conducted.

Future research should compare more within-culture differences to determine the presences of emerging adulthood in other cultures. Arnett makes the argument for emerging adulthood in the United States by performing  within-culture comparisons across ages and examining how they have changed over time. It is reasonable to believe that similar changes are occurring in other cultures; however, the indicators may be distinct as they are culturally sensitive. Cross-cultural methodology is challenging; the study of developmental constructs will require creative experimental design to establish meaningful and/or causal relationships.

Arnett, J.J. (2000). Emerging adulthood: A theory of development from the late teens through the twenties. American Psychologist, 55(5), 469-480. doi: 10.1037/0003-066X.55.5.469

Nelson, J.L, Badger, S., & Wu, B. (2004). The influence of culture in emerging adulthood: Perspectives of Chinese college students. International Journal of Behavioral Development, 28, 26-3.

How Does Adoption Affect the Health of Same-Sex Couples?

With increasing acceptance of same-sex marriages in the public sphere I thought I would give treatment to some psychological research in the area. There is often rhetoric on the adoption rights of same-sex couples and how these adoptions affect the health of the child. However, little air time is given to how adoption affects the health of same-sex couples. After all adoption is a complex and often stressful process for any couple. Throughout development, homosexual individuals go through many of the same transitions as heterosexuals. However, in some cases, these transitions may be more stressful due to issues such as heteronormativity, stigmatization, institutionalized heterosexism and social prejudice (Goldberg & Smith, 2011). Given these factors the transition into parenthood, while representing a time of heightened stress among heterosexual couples, may uniquely influence the well-being of homosexual couples. In this more formal blog entry I analyze factors affecting depression and anxiety of homosexual couples as they transition into adoptive parenthood.

The number of homosexual couples choosing adoption is increasing (Goldberg & Smith, 2011), but there is little research on how the transition to adoptive parenthood affects homosexual couples. Further, adopted children are at greater risk for emotional and behavior problems, many of which are associated with relationship quality and psychological well-being of the parents (Goldber, Smith, & Kashy, 2010). A better understanding of factor influencing mental health during this transition will inform effective therapies and support for this population that will result in relevant and important outcomes (Killian, 2010).

Social support, self-concept and depression/anxiety

Goldberg & Smith (2011) was the only study found that directly examined how social support influences changes in anxiety and depression after adoption for same-sex couples. The study measured state legal climate and relationship quality as well as support from the neighborhood, workplace, family and friends. State legal climate was classified as either positive or negative using the Human Rights Campaign’s “Family Equality Index”. All other measures were self-reported questionnaires that gathered the participants’ perception of support (e.g. “I rely on family for social support”). The authors also measured “internalized homophobia” which examined whether the individual thinks positively (or negatively) about their sexual orientation (what this paper refers to as self-concept).

The results indicated that social support (as measured by state laws for adoption) moderated the relationship between individuals’ self-concept (internalized homophobia) and changes in depression/anxiety through the adoption period. Participants high in internalized homophobia before adopting showed an increase in depression and anxiety after adopting in a negative legal context but a decrease in depression and anxiety when the adoption was conducted in a positive legal context.

The authors also found a number of main effects social support has on depression and anxiety over the adoption period. Lower perceived gay-friendliness in the neighborhood and higher internalized homophobia were related to higher depression (but not anxiety). Perceived support from friends was related to lower anxiety (but not depression). Higher levels of perceived workplace support, family support and relationship quality were associated with lower levels of depression and anxiety.

Similarly, Ross, Epstein, Anderson, & Eady  (2009) found that social support (at various levels) plays an important role in psychological outcomes after adoption. They conducted structured interviews with seventy-four individuals who legally adopted a child into a same-sex relationship under new laws in Ontario, Canada. After transcription and coding three major themes were identified: supportive experiences, unsupportive experiences, and identity-base experiences. Results indicated that participants living in small neighborhoods (where they were well known and well supported) had positive feelings with regard to the adoption experience. However, if the small community was unsupportive, participants characterized the experience as more difficult. Further, findings recognize the interaction between external social support, self-concept and psychological well-being. In other words, the most positive experiences around adopting were expressed by couples working with agencies that identified their sexual identity as a potential strength in parenting an adopted child. Agencies that legitimize same-sex parenting through recognition of the strengths inherent in it may have bolstered the prospective parents’ self-concept (the belief in their ability to parent as a sexual-minority). This change in confidence may, in turn, have influenced positive changes in anxiety and depression surrounding adoption (though these conclusions are beyond the scope of the authors’ analysis).

Another relevant study by Goldberg, Smith, & Kashy (2010) had similar findings. In their analysis of relationship quality before and after adoption the authors examined similarities and differences between gay male, lesbian and heterosexual couples. The results suggest that, on average, all parents, regardless of sexual orientation and type of relationship, experience a decline in the quality of relationship. However, higher adoption agency satisfaction is related to lower relationship conflict post adoption (particular among same-sex couples). The study also found that higher depression before adoption led to larger increases in relationship conflict among all couples, however it is likely that this relationship is bidirectional (relationship conflict leads to higher depression as well). These findings highlight the influence positive social support (as provided by agencies) has on many psychological outcomes that include (and are related) depression and anxiety.


Overall, the current research suggests that social support influences the development of depression and anxiety as same-sex parents transition into adoptive parenthood. However, the mechanism behind this relationship remains unclear. While the influence of social support may show the strongest impact on depression/anxiety when acting through the individuals’ self-concept, it is important to note that self-concept may in fact be derived from various areas of social support. Most studies, however, measure social support solely through the participants’ perception of support. In order to understand the relationship between social support and self-concept more objective measures of social support should be utilized. Also of note, social support moderated self-concept and depression/anxiety when it was related to institutional processes involved in adoption (i.e. state laws or adoption agencies). Perhaps some areas of social support (e.g. family support) contribute more directly to certain psychological outcomes while other, more peripheral, areas of social support (e.g. societal laws) are buffered through self-concept. More research is needed to ascertain the process by which social support influences depression and anxiety in this context.

This area of research is relatively young, but holds promise for developing strategies to assist same-sex couples through the adoption process. Encouraging organizations and agencies to help same-sex couples develop positive self-concept in the context of adoption may improve psychological well-being throughout the process. This may be particularly important among homosexual males who appear to be more prone to self-doubt, due to their sexual orientation, in their ability to parent (Downing, Richardson, Kinkler, & Goldberg, 2009; Ross et al., 2009). Promulgating knowledge within LGBTQ communities about the strengths in adoptive parenting as a sexual minority may also reduce negative psychological outcomes after adoption while encourage more couples to adopt (Ross et al, 2009). Future work should consider the process by which social support, self-concept and depression/anxiety are related in an effort inform intervention strategies.


Downing, J., Richardson, H., Kinkler, L., Goldberg, A. (2009). Making the decision: Factors influencing gay men’s choice of an adoption path. Adoption Quarterly, 12, 247-271.

Goldberg, A.E., Smith, J.Z. (2011). Stigma, social context, and mental health: Lesbian and gay couples across the transition to adoptive parenthood. Journal of Counseling Psychology, 58,139-150.

Goldberg, A.E., Smith, J.Z., & Kashy, D.A. (2010). Preadoptive factors predicting lesbian, gay, and heterosexual couples’ relationship quality across the transition to adoptive parenthood. Journal of Family Psychology, 24, 221-232.

Killian, M. L. (2010). The political is personal: Relationship recognition policies in the United States and their impact on services for LGBT people. Journal of Gay & Lesbian Social Services, 22, 9-21.

Ross, L.E., Epstein, R., Anderson, S., & Eady, A. (2009). Policy, practice, and personal narratives: Experiences of LGBYQ people with adoption in Ontario, Canada. Adoption Quarterly, 12, 272-293.