• written by Hans IJzerman & Olivier Dujols

This is a cross post from the Inquisitive Mind blog

One of the things we will miss possibly the most this pandemic winter in the Northern Hemisphere is gezelligheid [ɣəˈzɛləxɛit]. No real English equivalent of gezelligheid exists; the closest word in the English vernacular – coziness – still doesn’t capture the same feeling of intimacy and belonging. What does communicate a similar sentiment and is more familiar to US ears is the Danish concept hygge [hʊɡə] and the Swedish concept of lagom. Wikipedia describes gezelligheid as “’conviviality’, ‘coziness’, ‘fun’” or “just the general togetherness that gives people a warm feeling”. Perhaps the best description of gezelligheid is the sense of belonging when sitting around a warm fire during a cold Christmas. But gezelligheid undeniably extends beyond the nuclear family; when you are outside in the shops where strangers gather and you experience that conviviality so typical of the current time of the year it is also “gezellig”. None of this will be something we will encounter much this winter. 

Gezelligheid, as it may be, is one way to socially cope with a cold and dark winter. Regulation of temperature is a driving force behind feeling close, being intimate, and feeling loved. Across the animal kingdom, the regulation of temperature is crucial to survival. Not being able to regulate one’s temperature leads to certain death. When temperatures drop, animals (including humans) use more energy to warm themselves up. The cost of temperature regulation is often countered by distributing it across kin, a phenomenon called social thermoregulation. Take penguins, for instance. When dealing with the harsh winters of Antarctica, they get together and “huddle” in a circle. Even if the ambient temperature is -40 degrees the temperature at the center of  the huddle can rise to a whopping 99.5 degrees. 

But penguins are not humans and humans are not penguins. Humans deal with fluctuating temperatures in modern times in many more sophisticated ways than penguins (i.e., relying on central heaters or warm clothes). But this has not been true for most of human history, so the imprint of what we call social thermoregulation has definitely left its mark on human culture:  Different languages, for instance, have a different number of temperature terms. Some languages distinguish between cold, lukewarm, and warm; others only have two temperature terms, like cold and warm. And languages differ whether they use metaphors where they combine warmth and affection. In Dutch, for example, you can refer to someone that you are not particularly fond of as “Zij laat me koud” (literally “she leaves me cold”), while in English one can say that you have a fond and warm memory of someone. Out of 84 languages the linguist Masha Koptjesvkaja-Tamm sampled in her research, 52 use metaphors combining warmth and affection – the others do not. 

The imprint of social thermoregulation is not limited to language. A team of researchers for example found that people who live in more “clement” climates (climates with an average temperature close to 71.6 °F) score higher on personality factors related to socialization and stability as well as on personality traits related to personal growth and plasticity. In one set of our own studies, we observed to what extent people would like to call or email a loved one and relate this to outside temperatures. We repeatedly find that if temperatures are lower, people have a higher desire to call or email other people. In other studies, when we experimentally manipulated temperature to be lower or higher, in lower temperature conditions people tend to think more about people they feel close to. Perhaps one of the effects that represents the Dutch concept of gezelligheid we detected in a sample of over 1500 people in 12 countries, where we found that one of the best predictors of people’s core body temperature was the diversity of their social network, meaning the more types of social groups they participated in (e.g., a volunteer group, a group at work, a sports team, a family group) the better protected their core body temperatures were if they lived in a colder climate. Gezelligheid, after all, is not just sitting around the warm fire during Christmas, it is also being in a shopping mall and sharing stories with your sports team or your local faith group. 

It won’t come as a surprise that as psychologists, we worry. We worry about the lack of gezelligheid this pandemic winter. And we worry that people miss out on gezelligheid because we suspect that social thermoregulation is mostly achieved successfully when being physically proximate, which is why Zoom calls likely won’t cut it. And yet, humans have been flexible in adapting to nearly every physical environment in history, which is why there must be ways we can cope, even if it is temporarily. A startup out of MIT has developed a bracelet that allows users to send each other warmth, years back, Sony patented a controller for its playstation for temperature feedback, and Japanese engineers are developing a robot that transmits warmth while hand-holding. We had imagined developing a “relationship simulator” to reduce the lack of intimacy intimate partners may have had while apart during the pandemic.  

Yet the research we are engaged in cannot (nor should) rival the research currently being conducted on vaccines, where BioNTech received a 375 EUR Million grant from the German government to solve one the worst crises in recent human history to develop a vaccine against Covid-19. Compared to that, our research is on a shoestring budget. That means that the work we have done does not achieve what researchers in other disciplines would call a high Technology Readiness Level, or, better, we don’t have comparable confidence in our work as one can have for a Phase III trial for the Moderna or Pfizer/BioNTech vaccines. Providing concrete advice on how to resolve the lack of gezelligheid is not psychology’s forte, but having an insight into the mechanisms behind social thermoregulation will make you all the wiser on how to make educated guesses to help you deal with the lack of gezelligheid.   

Hans Rocha IJzerman is the author of Heartwarming: How Our Inner Thermostat Made Us Human.

• written by Patrick Forscher & Hans IJzerman

This is a cross post from the Inquisitive Mind blog

During this pandemic winter, many of us will be away from the people we love most. The absence of the physical presence of loved ones deprives us of hugs, physical touch, and feelings of physical and psychological warmth that no amount of Skype or Zoom ever seems to fully replace.

In his forthcoming book, Heartwarming, one of the authors of this editorial (Rocha IJzerman) explores the science of why this is, a field called social thermoregulation. This science points to promising technologies like the Embr Wave, smartphone-controlled warmth-producing bracelet, that seem to provide ways to compensate for the physical and psychological warmth that Skype and Zoom lack. Yet the book stops short of recommending these technologies, or indeed offering any concrete advice at all. Why is this?

One reason stems from the sordid tale of Brian Wansink. A former marketing researcher at Cornell, Wansink excelled at conducting clever-sounding studies on the psychology of eating and packaging them into bit-size, actionable pieces of advice tailor-made for public consumption. He would then pitch this advice in outlets ranging from O Magazine to the Today Show. His work even served as the basis for federal policy in the form of the Smarter Lunchroom Movement. Yet, proper scrutiny revealed a shocking level of sloppiness in Wansink’s research. Although Wansink had spun his research into advice for millions of people and a federal program funded to the tune of $22 million dollars, all these products were built on foundations of sand.

Nor is Wansink’s work the only example. Take the example of research on implicit bias. Defined as a set of fast, relatively automatic mental pairings between social groups and other concepts, the concept of implicit bias was popularized largely due to the efforts of Anthony Greenwald and Mahzarin Banaji. These two researchers developed a test, the Implicit Association Test, and demonstrated it in a press conference in 1998 in which they claimed to have data demonstrating that 90 to 95 percent of people harbor “the unconscious roots of prejudice. The test became an immediate sensation, receiving favorable coverage by the wildly popular science journalist Malcolm Gladwell, NPR correspondent Shankar Vedantam, and, more recently, even then-presidential candidate Hillary Clinton in the rarified halls of a US presidential debate.

Yet a closer look at the evidence on implicit bias reveals a field that is woefully unready for application. The Implicit Association Test is plagued with measurement problems (as are, for that matter, other alternative measures of implicit bias). For example, too often the same person who takes the test multiple times gets a different result, probably because the test is contaminated by large amounts of measurement error. The relationship between people’s score on the IAT and their actual behavior is also quite weak. Finally, a closer look at studies that ought to be most relevant to policy — those that try to change implicit bias — reveals that this field relies very heavily on student samples, rarely assesses whether the changes persist, and contains little to no evidence that changes in implicit bias lead to changes in behavior.

In Heartwarming, Rocha IJzerman wanted to avoid the mistakes of Brian Wansink, Tony Greenwald, and Mahzarin Banaji. We believe psychology can learn from other research fields, like rocket science and drug development, that excel at translating research findings into safe and effective applications. Rocket science uses a framework called the Technology Readiness Levels to evaluate the state of the evidence behind an application and to guide the research program. Drug development uses a series of phased trials to evaluate both the effectiveness and the safety of a novel drug. As we have seen from the accomplishments of these fields — from putting people on the moon to creating a safe and effective Covid-19 vaccine in record-breaking time — this systematic process works.

Heartwarming judges the science of social thermoregulation using standards closer to those of rocket science and drug development rather than those of Brian Wansink. According to those standards, social thermoregulation, though promising, is not yet ready to serve as the basis of advice and other applications. Our unpublished quantitative review suggests the available evidence on social thermoregulation can support the broad conclusion of Heartwarming: there are intriguing links between the regulation of temperature and social relationships that promise to provide unique insights into the roots of human sociality. Yet social thermoregulation research still largely relies on samples of participants that are very peculiar — college students from Europe and the United States. We also know little about the effectiveness of different dosages of warmth for changing psychology. Nor, as would be routine in drug development, have the technologies based on social thermoregulation research been evaluated in household settings for things like unintended side effects.

Thus, while Heartwarming does present science that is promising enough that it could in the future serve as the basis for advice and application, it does so with caveats. The book therefore notes where social thermoregulation research relies too heavily on unusual and non-representative samples, and it also notes where its findings have not been subjected to the type of big-team, large-scale testing that you might get in drug development.

Psychology may yet initiate the reforms necessary to make it robust enough for application. Psychology researchers (ourselves included) have developed our own frameworks to help evaluate just when psychology evidence is ready for application. Moreover, psychology researchers have created organizations and proposed broad-based reforms to make the big, team-style research more commonplace.

In the meantime, beware psychologists bearing overconfident advice. You are bound to be disappointed.

Hans Rocha IJzerman is the author of Heartwarming: How Our Inner Thermostat Made Us Human.

Patrick S. Forscher

“There’s nothing as practical as a good theory.”

This quote is often attributed to the person hailed as the “father of social psychology”, Kurt Lewin. In social psychology textbooks, the quote is used to justify a mode of social psychology whereby theories that are developed and tested in the lab “naturally” and “inherently” lead to useful social applications (Billig, 2015).

Here I will argue that theory is not always helpful for solving practical social problems. This is because fixating overmuch on theory can lead to what I call theory blindness, a singular focus on the aspects of the problem that are relevant to the theory at the expense of other aspects that are just as important when the problem is considered in its entirety. I review a case study that illustrates the dangers of theory blindness and close with an argument for a focus on the practical problem as it exists in its original context rather than a focus on any particular theory.

Theory application as a model for practical impact

The dominant model of how social psychology comes to have a useful impact on society, at least as portrayed in social psychology textbooks (Billig, 2015), is what I will call the theory application model. In this model, theory development may be somewhat informed by a particular social problem, but theory testing happens largely in tightly controlled environments, such as the laboratory or an online survey platform. The process proceeds something like this:

1. The social psychologist develops a theory. The theory can come from many sources: introspection, personal experience, the literature, or perhaps as a response to current events. Although this is rare in social psychology, the theory may even follow a process of formal theory development. These theories may be somewhat informed by a social problem, but not necessarily the problem in its full, original context.

2. The social psychologist tests hypotheses derived from the theory in tightly controlled environments. The theory is then used to derive hypotheses that, in social psychology at least, are usually tested using experimental methods, most often in the laboratory or a tightly controlled online platform like Qualtrics. The results of these experiments are used to support the theory, refine it, and add boundary conditions. The usual justification for the high degrees of control in this step is that this control is necessary to isolate and manipulate the psychological processes that are relevant to the theory (Billig, 2015).

3. The social psychologist applies the theory to a practical problem as a way to test its generalizability. Once theory-derived hypotheses have been tested in tightly controlled environments, the social psychologist may choose to conduct a study (for example, a field experiment) specifically designed to address some practical social problem. Usually, this exercise is framed as an application of theoretical principles already established in the previous two steps; the implication is that the study cannot directly refute the theory but merely demonstrate how its principles generalize to new, somewhat uncontrolled and messy settings.

Theory highlights some things and de-emphasizes others

The theory application model of practical impact illustrates how theory might, at least in principle, help solve social problems. Psychological theories provide working models of how psychological processes produce behavior. Insofar as individual behaviors “add up” to produce a social problem, psychological theories can highlight the psychological processes that produce those individual behaviors. Theories therefore also provide ready explanations for social problems and, due to their usefulness in identifying causal mechanisms behind individual behaviors, a guidebook for intervention via disrupting those mechanisms.

For example, the appraisal theory of emotion argues that emotions are produced by a person’s interpretations of events – in other words, their appraisals. This theory highlights these appraisals as a cause of emotion. The implication is that, to solve problems related to widespread negative emotions, such as the emotional fallout of a natural disaster, one should look to identify and change people’s interpretations of the natural disaster. This can lead scientists to develop and deploy specialized measures of people’s appraisals of the natural disaster and apply interventions that target those appraisals. Under the spotlight of appraisal theory, aspects of a social problem related to appraisals become valid observations while other aspects of the problem become de-emphasized.

Herein lies the danger of the theory application model of practical impact: if the theory does not fully capture the problem’s original context, it can leave important aspects of the problem outside its theoretical spotlight. For example, the lens of appraisal theory might cause well-meaning scientists who wish to provide mental health assistance in the wake of a natural disaster to neglect the real mental burdens caused by the economic and social fallout of the disaster. Focusing too much on people’s appraisals of a natural disaster as a cause of poor mental health is a demeaning way to help people who have just lost their homes and loved ones. In addition to leading to misguided forms of help, theory blindness can have real material costs if lives depend on the help’s effectiveness. In the most extreme situations, these costs can include lost lives and livelihoods.

The curious blindness caused by theory is part of a phenomenon that philosophers of science call the theory-dependence of observation. Theories define what counts as a valid observation. Theories can lead people to develop entire instruments dedicated to the measurement of the processes posited by theory (see Greenwald, 2012, for examples). In the context of a social problem, everything outside the measurement paradigm becomes “noise” that is outside the theory’s scope. When the theory-derived measures are deployed to solve a social problem, these “extraneous” factors can therefore be interpreted as irrelevant to the target problem.

I am not the first to note the peculiar dangers of theory for application. Daniel Kahneman called this phenomenon “theory-induced blindness” and linked it to the mental heuristics and biases that dominated his decision-making research. Curiously, Kurt Lewin himself may not have endorsed theory as a route to application (see Lewin, 1931) – at least in the way social psychologists currently go about it. (Even curiouser, Lewin was also not the originator of the quote that leads this blog.) 

Despite this history of similar critique, I believe the dangers of theory for application are not widely recognized. I therefore illustrate these dangers with a concrete case study.

Case study on Implicit bias

The concept of implicit bias was developed to explain a particular contradiction in the United States: in large, national surveys racial attitudes appeared to consistently improve from the 1960s through the early 1990s (Schuman, Steeh, Bobo, & Krysan, 1997). Yet, despite these improvements, racial disparities remained more stable than many scholars would like (Lee, 2002). These stable patterns in disparities were reflected in the laboratory, where even participants who claimed to value equality seemed to act unfairly when assessed on “unobtrusive” measures of bias (i.e., measures where, from the participant’s perspective, no particular response could be clearly labeled as “prejudice” or “discrimination”; Crosby, Bromley, & Saxe, 1980). Solving the puzzle of why people’s self-reports contradict their laboratory behavior provided an enticing means of making a practical impact.

Dual process models, such as the “prejudice habit model” (Devine, 1989), arose as an explanation for this contradiction. In these models, people’s beliefs had changed throughout the 1960s up until the 1990s, but a fast, relatively automatic mental process had not. When national surveys asked people how they felt about Black people, those surveys measured beliefs. However, the studies of “unobtrusive” bias measured behaviors that were more influenced by the automatic process.

At first, research in the dual process tradition was stymied by the fact that there was no independent measure of the automatic process. This changed with the creation of “implicit measures”, especially the Implicit Association Test. These measures were important for two reasons. First, they gave researchers a tool that purported to quickly, easily, and directly assess the automatic process (which came to be known as “implicit bias”). Second, they created a “palpable experience” (Monteith, Voils, & Ashburn-Nardo, 2005) that, when made widely available through websites like Project Implicit, greatly enhanced the standing of implicit bias in the public imagination.

The result was an explosion of research on implicit bias and a steady increase in the reach of the concepts into the public sphere. This research included observational studies designed to validate the new class of implicit measures and demonstrate the potential consequences of implicit bias (Cunningham, Preacher, & Banaji, 2001). The research also included manipulations designed to investigate the procedures that could change implicit bias (Dasgupta & Greenwald, 2001).

A funny thing happened over the course of this research on implicit bias: the implicit measures, but especially the IAT, became a target of change in their own right. In effect, the IAT became a stand-in for the problems related to social disparities. Thus, the existence of bias on the IAT became a convenient way of representing those disparities, and demonstrating the presence or absence of change on the IAT became a way of demonstrating which procedures might have promise for changing social disparities. 

The result was a cottage industry of interventions and trainings all developed around the concept of implicit bias, especially as represented by the IAT. This has come to a head in the public sphere, where large companies like Starbucks and Delta Airlines have rolled out company-wide training programs focused on implicit or unconscious bias. Governmental agencies have also taken an interest; the New York Police Department has instituted its own training, as has the UK civil service (this program was recently scrapped).

Meanwhile, the evidence is muddled at best that implicit bias does indeed cause real-world disparities, and in some settings there is strong evidence of the importance of non-psychological factors. Take racial disparities in who is subject to police misconduct. In many US jurisdictions, firing problematic officers is extremely difficult because weaker disciplinary oversight is one of the main concessions that police unions have extracted through collective bargaining. This concession makes it difficult to remove officers who are acting out of line, with a disproportionate impact on communities of color. The implication is that reforming systems of collective bargaining might be an effective means of reducing police misconduct, especially in communities of color.

This insight comes not from a particular theory, but from a careful examination of the historical and policy environments of US policing. However, an observer who approaches problems in US policing wearing the blinders of dual process theories might miss this important insight, pursuing instead reforms such as mandatory implicit bias training that do not effectively alleviate the suffering caused by police misconduct.

The problem-solving model as an alternative to the theory application model

The police union example illustrates an alternative means of achieving practical impact that is different from theory application model: a focus on a particular, substantive problem as it exists in its original context. The process that I suggest bears some similarities to action research – a research method initially developed by, ironically, none other than Kurt Lewin (Lewin, 1946). My suggested process goes like so:

1. Analyze the target setting to develop a definition of the target problem. The first step is developing a careful analysis of the problem setting. This analysis should, at a minimum, draw on a consultation with stakeholders, but it can also draw on historical and policy analysis and pre-existing administrative data (if available). Any and every tool that provides insight into the problem is on the table; this stage may therefore require tools from many disciplines and levels of analysis. At this stage, the problem-to-be-solved may not yet be specifically defined. A major goal of this stage is to integrate the various sources of information to come to a more firm definition.

2. Select measures of the problem and define success and failure using those measures. A definition of the problem is little help if there are no systems for assessing progress in solving the problem. This second stage involves defining the problem in terms of a measure that can be readily deployed in the problem setting. In some cases, no pre-existing monitoring system exists; in those cases, this second stage may involve devising and implementing such a monitoring system.

3. Define the universe of interventions you can implement. Not all interventions are within the realm of possibility in all target settings; what is feasible will be constrained by pre-existing policy, law, and available resources. This third stage involves surveying the range of possible interventions you can deploy given these practical constraints. The interventions themselves can come from anywhere, whether from psychological theory or from the careful analysis of the target setting developed in the first step.

4. Implement one or more interventions with a plan for progress monitoring. The final step is to implement one or more of the interventions identified in the third step with a plan for progress monitoring using the measures identified in the third step.

As in action research, these four steps can form a feedback loop: if the intervention tested in the fourth step is unsuccessful, that can form the basis for a new analysis of the target problem in the first step.

Theories may indeed have a role to play at different stages of the problem-solving model. However, and unlike the case with the theory application model, the orientation of the problem-solving model is on a particular problem in its original setting rather than on a theory and its degree of empirical support. The problem setting therefore serves not just as a venue for demonstrating a particular theory’s generality but as the entire focus of the research process. Moreover, the model embraces the “messiness” of the problem setting’s history, politics, and resource constraints rather than attempting to control the messiness away, as understanding this messiness is critical to developing interventions that work.

Conclusion: Theories are not always helpful for generating practical impact

Theory has been enjoying something of a renaissance in psychology research, and a large list of scholars have put forward persuasive arguments that theory development is critical for advancing knowledge about psychology (Muthukrishna & Henrich, 2019; Fried, 2020; Robinaugh et al., 2020; Navarro, 2021; van Rooij & Baggio, 2021 1; 2). As a response to this resurgence, a small but growing list of scholars have started to identify limits to the usefulness of theory in knowledge advancement (Barrett, 2020; Eronnen & Bringmann, 2021).

Personally, I do not dispute the usefulness of theory – if the primary goal is indeed knowledge advancement. If the primary goal is pragmatic, I worry that processes like theory blindness may interfere with a clear-eyed view of the target social problem. For this reason, I believe that a problem-focused approach is a more effective means of achieving pragmatic goals.

Notes:

These ideas were conceived at the Relationship Preconference at the 2021 meeting of the Society for Personality and Social Psychology. Thanks to Farid Anvari, Hans IJzerman, Daniël Lakens, Duygu Uygun-Tunç, Peder Isager and Leonid Tiokhin for their helpful comments on previous drafts of this blog.

This is a cross post from the PSA blog

Patrick S. Forscher and Hans IJzerman

Note: This post accompanies three surveys, which are embedded as links in the post’s body. For convenience, you can find direct links to them here.

• Translator survey
• Demand for translation survey
• Membership dues survey

—

In our previous post, we argued that the PSA has a grand vision and a budget that cannot easily support it. If the PSA is to fulfill its aspirations, the PSA must increase its funding so that it can support an administrative staff.

In this post, we will assume that the PSA wants to fulfill its grand vision. We will therefore explore ways that the PSA could create the funding streams that are necessary to achieve that vision. We will review four options: 

1. Grant funding
2. Charitable giving
3. Fees-for-service
4. Membership dues

Most of our energy to date has focused on grant funding and charitable giving. A sustainable funding model will require a mix of all the options so that we can balance their strengths and weaknesses. We can think of the PSA’s funding as a sort of investment portfolio where we devote a percentage of effort to each potential income stream based on the PSA’s financial strategy and risk tolerance.

In the remainder of this post, we will evaluate the strengths and weaknesses of all four funding streams. For each funding stream, we will also review the activities that we have already attempted and what else we can do in that category. 

The post is thorough and therefore long. Here are some big-picture takeaways:

• We have already tried, quite vigorously, to secure grant funding and charitable giving, with modest success
• Large-dollar grant funding is high-risk, high-reward and likely cannot be counted on to secure the PSA’s financial future; small-dollar grant funding cannot easily support staff
• Charitable giving mostly yields small amounts of money, though donated research assistants could help provide labor for the PSA
• Fees-for-service offer some potentially interesting funding streams, including conference organization (continuing the success of PSACON2020) and translation
• Membership dues arguably fit the structure of the PSA best out of all funding streams, though we should think carefully about how to implement these without compromising our value of diversity and inclusion

We will argue that the PSA should pursue fees-for-service and membership dues, small-dollar grants, and charitable giving. Large-dollar grants should only be pursued in periods when the PSA can tolerate risky funding strategies. Finally, the PSA will need a bookkeeper and Chief Financial Officer to formalize its finances.

Grant-Writing

Grant-writing involves writing a specific proposal to a formal agency. The agency judges the proposal on its merits and its fit with the agency’s mission and, if the proposal is sufficiently meritorious, issues an award. This is the conventional way that scientists try to fund their work. 

We argue that grant writing is high effort, high risk, potentially high yield, but with high volatility. Grant-writing thus makes for bad planning long-term. 

Strengths

• Grants can be very big. The big granting institutions have a lot of money. For example, one of the grants we wrote in the past three years was for €10 million. This dictum is not always true — many grants are for amounts closer to €1,000 — but when grants pay off big they pay off big.
• Many scientists have grant-writing expertise. Because grants are the traditional means through which science is funded, grant-writing expertise is easy to find among the scientific community. This makes grant proposals easier to get off the ground than many other funding activities.

Weaknesses

• Grants usually fund projects, not infrastructures. Funding an infrastructure like the PSA requires an ongoing investment, whereas projects complete within a few years. For this reason, granting agencies view projects as lower-risk funding priorities. Most grant mechanisms are therefore project-focused rather than infrastructure-focused.
• Grant-writing is high risk.At the US National Institutes for Health, the US’s largest funder, the overall success rate for its most common grant is 23%. This means that, if the PSA relies exclusively on grant funding, we ought to expect 3 out of every 4 grant proposals to not yield any money. It is hard to build a staffing plan for an organization under such risky conditions.
• Grant-writing is inefficient. Grant-writing requires so much time and energy with such a low probability of paying off that the expected return may not justify the expected time investment. One computational model suggests that these inefficiencies are inherent to any funding scheme structured as a competition between proposals because scientists waste as much or more effort on the proposals as they would gain from the science that the proposals fund.
• The money is not flexible. The PSA wants to use its money for many things: grants to under-resourced labs, paying staff, funding competitions to make scientific discoveries, and many other things. Grants have strict accounting rules that rule out many or most of these activities.

What have we done already?

• One proposal to the European Research Council’s Synergy mechanism. This proposal was project-based and very large (€10 million). Some of the money would have gone directly to the PSA, but making the PSA funding work within the constraints of ERC accounting rules was challenging. Writing this grant took years of preparation, including one month of very intense effort by five PSA members. The proposal was rejected.
• Two proposals to the US National Science Foundation. Both were somewhat large (~$200K) and may have been able to support PSA staff. Both were rejected.
• Two proposals to the John T Templeton Foundation. Both proposals are project-based and somewhat large (~$200K). One proposal was rejected and one is in limbo due to Covid-19.
• Two proposals to the European Research Council’s Marie Curie mechanism. Both proposals were project-based and would have supported postdocs (~€120K). Although the postdocs would not have worked directly for the PSA, some of the proposed work would have indirectly benefited the PSA. Both proposals were rejected.
• One proposal to Université Grenoble Alpes. This proposal was designed to fund grant-writing for both the PSA and UGA. This proposal was funded and has supported a postdoc at a cost of ~€120K. Although the postdoc officially works for UGA, around half their time is donated to the PSA.
• One proposal to the Association for Psychological Science. This proposal was for $10,000 USD. All the money had to go toward participant recruitment for the PSACR project. This proposal was funded.
• One proposal to Amazon. This proposal was small ($1,352 USD) and strictly dedicated to paying for server hosting for PSACR. This proposal was funded.
• One proposal to the Society for Personality and Social Psychology. This proposal was small ($1,000 USD) and could only be used to support participant recruitment for PSA005. This proposal was funded.
• One proposal to the Society for Improvement of Psychological Science. This proposal was small ($900 USD) and devoted to offsetting the costs of applying for nonprofit status. This proposal was funded. 

The above grants translate to around $12,862,252 in funds sought and $163,252 received. This demonstrates the risk and volatility in grant funding.

In addition to the above activities, PSA leadership has submitted a large number of letters of inquiry and less formal communications seeking funding opportunities. These did not reach the stage of formal proposals.

What else can we do?

The PSA has explored most of the available options related to grant funding. Thus far, it has had the most success writing small, focused grants. These can support specific activities, such as participant recruitment, but cannot easily support staff.

Summary

Grant-writing for large grants is a high-risk, high-reward activity. Grant-writing for small grants usually requires investment of resources, but also cannot easily fund PSA staff. Grant-writing is also the activity into which the PSA has spent most of its funding energy. The PSA may want to focus its grant-writing activities into smaller, project-focused grants.

Charitable giving

Charitable giving involves contacting individuals to solicit money and other resources. Charitable giving is similar to grant-writing in that some entity gives money to the PSA of their own free will. The method differs from grant-writing in that the giver lacks a structured organization that announces funding guidelines and priorities. 

We argue that charitable giving is moderate effort, low risk, low to mediumyield, and medium volatility. Charitable giving allows for somewhat better planning long-term than does grant-writing.  

Strengths

• Allows direct outreach to supportive members of the community. The PSA has generated a lot of support and goodwill. Direct outreach allows members of the community to express this goodwill in the form of direct financial support. 
• Unconstrained by deadlines. Many other financial streams, such as grants, come on tight deadlines. Solicitations of charitable giving can happen at any time.
• The money is (usually) flexible. Grant funding comes with strict accounting rules. Charitable giving sometimes comes with strings attached, but less often.

Weaknesses

• Donation amounts are often small. The cost for staff salary runs in the tens of thousands of dollars. Most gifts run in the tens of dollars.
• The overhead for these gifts is often large. This limitation is linked to the first. In contrast to grants, where a payment from a granting agency only needs to be processed once, in a donation drive, payments need to be processed as many times as you have donors. Sometimes these payment processes need to go through a third party, such as PayPal. This can create substantial overhead in both labor and finances.

What have we done already?

• Set up a Patreon. At the time of writing, this Patreon collects €200 per month. We have used this money to help fund special projects, such as the PSA001 Secondary Analysis Challenge, data collection for PSA001, PSA006, and PSACR, and other miscellaneous expenses.
• Funding drives. We have conducted at least four funding drives in the past three years. These are usually structured around funding a specific project or activity, such as PSACR, and are often announced via the PSA blog. However, we also conducted a funding drive during the first PSA conference, PSACON2020. It is difficult to estimate precisely how much money these drives have generated, but the sum is likely less than $1000.
• Direct outreach. We have also done direct outreach to specific people who we have thought might be willing and able to donate to the PSA. Usually we have relied on personal networks for these direct solicitations. This process is smoother now that the PSA is an official nonprofit and can therefore accept direct tax-deductible donations.
• Speaking fee donations. A few members of PSA leadership have donated fees they’ve received to speak about multi-site research. A typical speaking fee is around $200. This source of income generates money, but is hard to scale to the point where it would support full-time staff.
• Received donations of research assistant time. Once special form of charitable giving is staffing time that is donated from an organization to the PSA. This system allowed the CO-RE Lab, for example, to donate the time of one of its interns to the PSA. This donation enabled, among other things, the creation of the first Study Capacity Report. This system works best when the research assistant or intern can be directly embedded within a PSA supervisory structure.

What else can we do?

The PSA’s new status as a pending nonprofit gives it more options that fit under charitable giving. The option to accept donated time from research assistants is especially attractive — especially if the donation can be formalized in an official agreement between the PSA and the donating organization. 

The PSA can also do more direct outreach to members, both through the PSA mailing list and at the time of registration with the member website (an option that bears some similarities to “membership fees”, discussed below).

Summary

Charitable giving is highly flexible and has generated some money for the PSA, though often the specific monetary amounts are small. One attractive option for the PSA now that it is a nonprofit is to accept more donations of time from member lab research assistants. This option works best when the agreement is formalized through an official agreement and when the research assistant can be embedded within a PSA supervisory structure.

Fees-for-service

A fee-for-service venture involves directly selling something, rather as a for-profit company would. Some of the proceeds from sales go toward maintaining the fee-for-service venture; the rest go toward supporting other activities within the organization.

One particularly interesting service the PSA can provide is translation. The PSA has extensive experience translating psychological tasks and measures into multiple languages. Moreover, the PSA has experience implementing the translations in software. Finally, if the PSA can help researchers recruit participants from more countries and cultures, that furthers its mission of ensuring that psychological science is more representative of humanity writ large. We will have more to say on this subject below.

Fees-for-service ventures are difficult to classify in terms of their effort, risk, yield, and volatility; much depends on the specific venture under consideration. Nevertheless, we believe they are underexplored and could usefully complement the PSA’s other efforts.

Strengths

• Can leverage already-existing strengths of the organization. If an organization must already be skilled at a specific task, developing a business around that skill allows the organization to leverage its existing strengths to generate a sustainable income. For example, the PSA must accomplish many things to conduct a multi-site study, including develop a large, global community of scientists and researchers around a common mission, coordinate agreements across many institutions, develop materials that are appropriate across many cultures, translate those materials into many languages, create and maintain large databases about its members, and disseminate knowledge about multi-site studies. Many of these skills can be leveraged into an income-generating enterprise. 
• Builds a sustainable income stream. Once established, a fee-for-service income stream can sustain itself over long periods of time, buffering against the risks associated with other income streams (for example, a grant running out after three years).
• The money is flexible. Because the money that comes from a fee-for-service venture is earned rather than donated by a third party, there is no donor who can attach strings to how the money is used. Thus, this money is very flexible and can be used to support staff salaries and any other activities the PSA desires.

Weaknesses

• Startup risks. Startups require up-front investments and there is no guarantee that a market will materialize. Thus, any investment made into a fee-for-service venture may, in the worst case, be lost.
• Mission capture. On the other hand, if a fee-for-service venture is very successful, that venture may capture the mission of the non-profit as a whole. Thus, fee-for-service ventures work best when they are well in line with the primary aims of the non-profit.
• Conflicts of interest. Some fees-for-service ventures may create conflicts of interest for PSA members who work on them, as these ventures may create incentives that conflict with the PSA’s core mission.

What have we done already?

• A PSA conference. This year, the PSA hosted its first conference, PSACON2020. The PSA charged a $60 registration fee with a generous waiver policy — the conference was planned on the assumption that ⅔ of the participants would receive waivers. The conference had 62 paying attendees (plus many more who received waivers) and brought in $3,720. This money went towards supporting the salary of the PSA’s junior administrator (and the main organizer of the conference), Savannah Lewis.

What else can we do?

Fees-for-service are a largely untapped opportunity for the PSA. Conducting multi-site studies is difficult and the PSA has developed a broad array of institutional knowledge to complete these studies. Much of this institutional knowledge can be leveraged into marketable services.

The success of PSACON2020 suggests that we could continue this conference in the future. If attendance remains stable or grows, this could become a small but consistent funding stream for the PSA.

A second interesting option is starting a dedicated PSA journal. This option entails some risk, as projects published there cannot be published in journals that are currently very prestigious (such as Nature Human Behaviour); authors on projects therefore take on some personal risk in the decision to instead publish at the PSA journal. It is also unclear whether university libraries would be willing to pay subscription fees for a PSA journal. On the other hand, the success of society journals suggests that once this revenue source is established, it could be a large and reliable one.

One last interesting option is translation. Translation of psychological measures and tasks is difficult, yet the PSA has lots of experience pulling off this task. PSA member Adeyemi Adetula has proposed a PSA-affiliated translation service with a first focus on Africa. This blog contains two surveys, one for people interested in being paid translators and one for people who want measures and tasks translated. If you are interested in this idea as a means of funding the PSA, head over to the blog, read the proposal, and complete one or both of the surveys.

Summary

Fees-for-service involves leveraging the PSA’s strengths to sell a service, generating a new income stream. The PSA has started some limited versions of fee-for-service funding, but this is still a largely untapped opportunity. 

In addition to continuing the success of PSACON, the PSA could explore starting a service to provide translations of psychological tasks and measures. Both these activities are in line with the PSA’s core mission — the conference because it builds community and supports members, translation because it provides the translated measures and tasks necessary to recruit participants whose native language is not English.

Membership dues

Membership dues are a special case of fee-for-service funding, but the model fits the structure of the PSA so well that it is worth discussing in its own section. The membership dues model involves charging some amount of money to gain access to some or all of the benefits of PSA membership. This is the model followed by scientific societies, which often gate grant funding and conference participation in exchange for annual dues of, say, $247 (APA) or $237 (APS).

Membership dues also have some overlap with some forms of charitable giving. This is because one way to ask for a donation is at the time of registration through the PSA member website. The major difference between this sort of solicitation and a membership fee is whether the default membership is free or paid. Membership dues do have one major, somewhat hidden, advantage over a solicitation at the time of member registration: due to grant accounting rules, many PSA members who have their own grants cannot spend this money on donations. On the other hand, grant money can typically be spent on a membership fee. Instituting membership dues may therefore unlock grant money that is currently inaccessible to the PSA.

Some implementations of membership dues may conflict with a core value of the PSA, namely diversity and inclusion. If dues are required of all members, they risk pushing out prospective PSA members with less resources. There may be implementations of membership dues (such as a dues structure that allows for fee waivers) that can help mitigate this risk. 

We believe that membership dues are low effort, moderate risk, medium yield, and low volatility. They therefore allow for better long-term planning than many other funding streams. However, we also believe that the specific kinds of risks that membership dues incur require feedback from PSA members before we rush into any decisions. We have created a survey to help seek this feedback; we lay out more context for why we see such promise in dues below.

Strengths

• Many people have pots of money that cannot be given as gifts. Under the Grant-writing section, we noted that grant funds are inflexible. One consequence of this lack of flexibility is that PSA boosters who have grant funds are often not allowed to use them in charitable gifts because the expense cannot be justified to the granting agency. Membership dues provide a way for these people to justify giving money to the PSA to granting agencies, thus unlocking a resource for the PSA that would otherwise go untapped.
• Leverages the PSA’s best resource, its community. The great strength of the PSA is that it has created a large international community (1400 members and counting) that believes in its mission. Membership dues allow the PSA to leverage this strength to better secure its long-term sustainability.
• Dues are linked to PSA member benefits. The PSA provides many benefits to members, including a connection with a large international community of colleagues and collaborators, exposure to cutting edge methods, and access to high-impact that can be added to one’s CV. Membership dues would emphasize that the PSA is only able to provide these things by maintaining a large infrastructure.
• Precedent in scientific societies. The dues funding model is the same one used by scientific societies, so dues-granted membership should be a concept that is familiar to PSA members.
• Wide variety of potential implementations. Dues can be charged on a yearly basis or could be tied to joining a study (the cost of joining would be an administrative fee). The PSA could also consider a variety of waiver policies.
• The money is flexible. The money is earned, so it could go toward supporting everything from staff salaries to ad hoc projects.
• Builds a stable income stream. Once a membership dues program is in place, the income stream from the program should be relatively stable year-to-year.

Weaknesses

• Risk of lowering membership. Gating membership by adding a joining fee could lower PSA membership, draining the PSA of its most valuable resource. Some policies could help manage this risk — for example, the membership dues could be suggested rather than required.
• Risk of driving away members with fewer resources. The PSA has a particular interest in increasing its membership in countries that provide less support for the sciences, such as those in the African continent. Imposing membership dues could put joining the PSA out of reach for these prospective members. Once again, policy could help manage this risk — for example, the PSA could provide generous waivers, just as it did for PSACON.

What have we done already?

Nothing.

What else can we do?

Membership dues come with some real risks, so we want to carefully investigate this option before we take any hasty action. We also want to ensure that whatever action we take has the broad support of the PSA membership. 

We have prepared a survey to investigate what PSA members think about different models of membership dues. By taking this survey you can help us evaluate the feasibility of using this as an income stream to promote the sustainability of the PSA.

Summary

Membership dues is arguably the funding model that best fits the structure of the PSA. However, some implementations of dues may conflict with the PSA value of diversity and inclusion. Membership dues have great potential as a funding stream, but we should think carefully about whether and how to implement them.

Conclusion

We strongly believe that the PSA has the potential to do enormous good for psychological science, and perhaps the social sciences at large. However, fulfilling this potential will require dedicated funding, and to obtain this funding the PSA may have to look beyond its current focus on grant-writing and charitable giving. Of course, the PSA need not abandon these income streams; it can simply balance these activities with efforts in developing funding streams through, say, fees-for-service ventures and membership dues.

If the PSA does develop a more balanced financial strategy, it will need staff to manage this strategy. This probably means appointing a Chief Financial Officer and a bookkeeper. The Center for Open Science, for example, has staff to manage its financial strategy and its books.

The PSA is at a crossroads. We think it has a real opportunity to grow into a more formal and mature organization that carries the big team science torch for psychology at large.

In one of my projects of my PhD, I – Olivier – am working on developing and validating a scale (the Social Thermoregulation, Risk Avoidance, and Eating Questionnaire – 2, or the STRAEQ-2). In the first phase of this project, we involved people from different countries and asked them to generate items. We did so to better represent behaviors from people across the globe and less so just from the EU/US. This item-generation stage involved 152 authors from 115 universities. In this blog post we present this first phase and share the code we used for this first phase.

Is attachment for coping with environmental threats?

Based on the premise that environmental threats shape personality (Buss, 2010; Wei et al., 2017), the goal of this project is to measure individual differences in the way people cope with the environment and to discover if these individual differences are linked to attachment. Attachment theory postulates that people seek proximity with reliable others to meet their needs (Bowlby, 1969), and the psychological literature suggests that distributing threats on others is metabolically more efficient (Beckes & Coan, 2011). In a previous project findings (STRAQ-1) Vergara et al. (2019) showed something consistent with this: individual differences in the way people cope with environmental threats (temperature regulation and risk avoidance) were linked to individual differences in attachment (Vergara et al., 2019). But the reliability of the scale created and validated in this project was somewhat inconsistent across the countries in which they collected data. We therefore decided to extend the STRAQ-1 findings to make it more reliable across countries, now focusing on:

1. Fluctuation in temperatures: generating a thermoregulation need (one’s need to maintain internal temperature within a comfortable range in order to survive), 
2. Physical threats: inducing risks avoidance (one’s need to avoid predators or people who want to do harm in order to survive), 
3. Lack of food: requiring food intake (one’s need to prevent from starvation).

We are also dividing each dimension into 4 subdimensions to make the scale more consistent with the attachment literature: sensitivity to the need, solitary regulation of the need, social regulation of the need, and confidence that others will help to cope with the need.

What are the behaviors that account for coping with environmental threats?

We first took the step to generate the items and we wanted to avoid the mistakes of the past.  Do people in Peru, China, Nigeria, or Sweden deal with temperature the same way? Probably not. So to have scale items that reflect a diverse range of coping behaviors we asked our collaborators; we designed a Qualtrics survey in which they read a description of each construct and we showed them example items.

In total, 737 items were generated by 53 laboratories from 32 countries. To automatize the procedure, and to avoid copy-and-paste errors, we created an R script to import all the generated items from Qualtrics into a Google Document. We created a text document from the Qualtrics file, including the subscale name followed by every text entry (item) for each sub dimension juxtaposing the name of the country that generated the items. Here is the piece of code that we used:

# this code select the ‘temperature sensibility subscale’ (plus their respective countries) in the data frame ‘items’ that is the qualtrics files containing all the items generated, and write the items in a .txt document called ‘Items.txt’:

library(tidyverse)
items %>%
  select(thermo_sens, country) %>%
  write_csv("Items.txt", col_names = TRUE, append = TRUE)

# we repeat this for all the subscales, adding them in the same file (Items.txt). For example here we add the solitary thermoregulation subscale:

items %>%
  select(thermo_soli, country) %>%
  write_csv("Items.txt", col_names = TRUE, append = TRUE)

The lead team imported the list of items (.txt) in an easily shareable document that track changes (google doc) to correct misspelling, reformulate and remove doubles in the list of items (all modifications are available here, and a clean version is here). 

How to select relevant and diverse behaviors (items) from a large list?

Because it would be too fatiguing for our participants to answer all of the 737 previously generated items in one sitting (on top of other questionnaires we offer them), we reduced the amount of items to be included in the main survey. We created a diverse advisory committee (including 9 researchers from Chile, Brasil, Morocco, Nigeria, China, the Netherlands, and France); via an online survey this advisory committee rated to what degree they thought the items were representative of their respective constructs.
Then, we computed the mean and standard deviation for each item. We selected the 10 highest means and lowest standard deviation items per subscales, and we replaced closely related items (~5 per subscales) to get a wider range of behaviors to be included in the scale. To facilitate the procedure, we created dynamic tables per subscale in an Rmarkdown document. These tables allow you to arrange per score (mean or sd), to select per country, or to search for specific words in the items. Here is the code that we used to generate the tables:

# from a data frame call ‘df’ that contains all the items and all the subscales we compute the mean and standard deviation of the expert ratings: 

df <- df %>%
  mutate (mean = rowMeans(cbind(expert_1, expert_9, na.rm=TRUE),
        sd = rowSds(cbind(expert_1, expert_9), na.rm=TRUE),
        mean = round(mean, digits = 2),
        sd =  round(sd, digits = 2)
   )

# then we tidy the data (that is we switch from a short format to a long format) in order to have one items per rows:

df_tidy <- df %>%  
  gather("expert", "rating", -item, -subscale,
          - mean, -sd, -country_list, -country)

# we create a new dataframe including only one subscale (here the sensitivity to temperature) and arrange the data frame having first the 10 highest means and lowest standard deviation:

temp_sens <- df %>%
  filter(grepl('temp_sens', subscale)) %>% #select rows "temp_sens" in the subscale column
  arrange(sd) %>%
  arrange(desc(mean)) %>%
  select(item, mean, sd, country)

# and finally we print an interactive table, that display automatically the 10 first rows (this can be change):

library(DT)
datatable(temp_sens)

We also plotted the distribution of the ratings (general, per experts, and per subscales) of the items to detect if there were some issues with specific subscales. Here is the ggplot code that we used to do that:

# first we create a color palette that will be used for the plot:

palette <- c("#85D4E3", "#F4B5BD", "#9C964A", "#C94A7D", "#CDC08C",
                  "#FAD77B", "#7294D4", "#DC863B", "#972D15")

# and here we plot the rating of the experts per subscales:

expert_plot <- df_tidy %>%
  ggplot(aes(rating, fill = expert, show.legend = FALSE)) +
  geom_bar(show.legend = FALSE) +
  facet_grid(expert ~ subscale, labeller = labeller(subscale = subscale.labs)) +
  scale_fill_manual(values = palette)

expert_plot

We then created a world map of the countries that generated the final list of the 120 STRAEQ-2 items, to observe how diverse our list of items was (we ended up doing some replacements for the final project, to increase the diversity of the scale).

# first import the world.cities database from the ‘package maps' in order to have latitude and longitude of countries around the globe:

library(maps)
data(world.cities)

# then you may want to rename some country in your data frame (here my data frame contains  the 120 selected items amd is called ‘items_120’) to match the name of the cities included in the world.cities data frame, in our case we needed to rename three cities:

items_120$country[items_120$country == "United Kingdom"] <- "UK"
items_120$country[items_120$country == "United States"] <- "USA"
items_120$country[items_120$country == "Serbia"] <- "Serbia and Montenegro"
 
# the next step is to merge the desired columns from the world.cities data frame with your data frame by country in order to get the latitude and longitude next to your cities’ names:

items_120 <- world.cities %>%
    filter(capital == 1) %>%
    select(country = country.etc, lat, lng = long) %>%
    left_join(items_120, ., by = "country")

# we compute the number of items that we have per country, this is needed to vary the size of the dots in the final map:

df_country_count <- items_120 %>%
  group_by(country) %>%
  summarise(n()) %>%
  rename(n = "n()")

# we create the data frame to generate the map:

items_120 <- left_join(items_120, df_country_count)

# we create the map (depending on your data you may want to change the radius argument fonction in order to adapt the differences in dot size, and also the value of the fillOpacity argument):

m <- leaflet(items_120) %>% 
   addTiles() %>%
   addCircles(
       lng = ~lng,
       lat = ~lat, 
       weight = 1,
       radius = ~ log(n + 5) * 100000, 
       popup = ~paste(country, ":", item, "(", subscale, ")"),
       stroke = T, 
       opacity = 1, 
       fill = T, 
       color = "#a500a5", 
       fillOpacity = 0.09
    )

# finally we can print the map:
m

This is where we arrived in the project so far. We are now at the stage to translate the scales for the project into various languages to collect data for the final project. 

Interested in participating in the project?

It is still possible for you to join the project. We will ask you to: 

1. Submit an IRB application at your site (if necessary). In order to make this step easier for you, we have written an IRB submission pack, which is available on the OSF page of the project, 
2. Translate the scales that are included in the main project via a forward-translation and back-translation method.
3. Administer an online questionnaire to at least 100 participants at your site (more is always possible). 

Please note that to account for authorship we are using a tiers author list combined to the CRediT taxonomy. We also expect to publish a data paper from the project that will help for future reuse of the data. We project to collect data from approximately 11 000 participants across the globe (we currently have 118 sites in 48 countries), to validate the STRAEQ-2 scale across countries, to measure individual differences in the way people cope with environmental threats, and to explore the links these differences maintain with individuals differences in attachment.

This blog post was written by Olivier Dujols and Hans IJzerman.

This is a cross post from the PSA blog

Patrick S. Forscher and Hans IJzerman

In 2017, Chris Chartier shared a blog post that revealed a grand vision for psychology research: psychologists could build a “CERN for psychology” that does for psychology what particle accelerators have done for physics. This “CERN for psychology” would be an organization that harnessed, organized, and coordinated the joint efforts of psychology labs throughout the world to take on large, nationally diverse big team science projects.

The months after Chris’s blog went live revealed that enough people believed in this vision to start building this “CERN for psychology”. These early efforts would evolve into the Psychological Science Accelerator, a network that, according to our recent study capacity report, now spans 1400 members from 71 countries. In these early months, the PSA also collaboratively developed a set of five guiding principles, namely diversity and inclusion, decentralized authority, transparency, rigor, and openness to criticism, that form a coherent vision for the type of science we want psychology to be. We want to help transform psychology to become more rigorous, transparent, collaborative, and more representative of humanity writ large.

Now, three years after its founding, the PSA stands at a crossroads. This crossroads relates to our broad vision of what the PSA is and should be and the means through which we achieve that broad vision. This post will cover the first issue. As we will describe, we believe our early documents point to a vision of the PSA as active, rather than passive, but that a lack of funding streams constrains our ability to achieve that mission.

Minimal and maximal visions of the PSA

Although the PSA was established to coordinate the activities of research labs, there are a wide range of options as to how this coordination is implemented. The specific implementations anchor two radically different visions of the PSA: a minimal vision and a maximal vision.

Imagine a PSA that is radically different from the one we have now: the PSA as a mailing list. 

This mailing list contains the contact information of people who are willing, in principle, to participate in team science projects. To use the mailing list, people design a team science study and email a description of the study to the list. People who receive a study invitation through the mailing list and freely reply to the study proposer. The mailing list itself is unregulated, so there is no vetting process for any of the emails people send over it, nor is there any support for the people who send invitations through the mailing list. This vision of the PSA is highly minimal in the sense that the PSA plays very little role in coordinating or implementing team science projects. However, this vision is also very low cost, as mailing lists are cheap to set up and almost free to maintain. In a sense, this minimalist version of the PSA already exists in the form of StudySwap — a useful tool, but not a transformative one.

Now imagine a PSA that is a bit more similar to the one we have now: the PSA as an active implementer of big team science projects.

In this vision, the PSA completes all stages of the team science research process. This means that the PSA takes partial ownership over its projects and participates in project decisions. This includes selecting projects to undertake through a rigorous review process, assisting with the theoretical development of studies, and improving on the design of studies by soliciting multiple rounds of feedback from relevant experts. In this vision, the PSA also actively coordinates and manages study teams to ensure that relevant administrative procedures (such as ethics review) are followed and to ensure that the various stages of the project occur on a reasonable schedule. The PSA also takes an active role in communicating completed projects to the world, perhaps by managing its own journal (the Proceedings of the Psychological Science Accelerator) and through its own dedicated press team. Finally, in this vision, the PSA has a variety of procedures to proactively improve its processes, including novel methods research, team science best practice research, project retrospectives, and exit interviews of PSA staffers who decide to leave the organization. This is the deluxe vision of the PSA, a PSA that is active but that requires lots of money and staff to maintain.

These two visions of the PSA — the minimal and maximal — also anchor an entire universe of in-between visions that are not so extreme. However, what is arguably true is that the vision of the PSA laid out in Chris’s blog post, as well as the one implied the PSA’s five guiding principles, are both much closer to the maximal vision of the PSA than the minimal one.

Money is necessary to implement a maximal vision of the PSA

If we accept that fulfilling the PSA’s mission requires something closer to a maximal vision of the PSA, we need to find ways to build the PSA into this more maximal vision. At a minimum, building and maintaining a maximal vision of the PSA requires people who can do the activities involved in this maximal vision. These people need to be recruited, managed, and retained, otherwise they will work at cross purposes, get into interpersonal conflicts, and burn out. In short, in addition to the people who carry out the PSA’s vision, the PSA needs an administrative structure to help these people carry out their work.

The PSA does, in fact, have a defined administrative structure. We have, for example, defined a set of committees to govern its activities and a set of roles that should be filled for each project. These roles outline an aspiration for the PSA to proactively conduct team science research — further suggesting that the PSA has a maximalist vision for itself.

These roles are many. Our recent evaluation of the PSA’s administrative capacity identifies fully 115 of them. If we assume that each position requires 5 hours per week of work to complete the associated responsibilities, the 115 roles require 29,900 hours per year to staff. Unfortunately, maintaining this level of labor has, at times, been challenging because of our reliance on volunteers who have other daily commitments (such as jobs that pay them). 

We can rely on volunteers to carry this load for a time, but doing so carries some real risks. For example, the heavy load can risk burning out the most active volunteers who take on the most labor and lead to large, costly mistakes if the labor requirements for large projects are not met. This load can also provoke interpersonal conflict if active volunteers feel that their labor is not properly recognized or credited. Finally, there are risks associated with who is able to be a volunteer: in a volunteer model, only those who can afford to will donate their time and efforts to the PSA. This squeezes out the voices and talents of some of the very people the PSA wants to elevate, such as those from Africa, Middle America, and South America. Our 2020-2021 study capacity report estimates that 90% of all people involved in administrative roles are from North America and Western Europe. The majority of these, 63% of all administrative roles, are located in North America.

Other growing organizations have managed the transition from an exclusively volunteer organization to one that is funded by some sort of income stream. We can learn from their history, which shows that the organizations that became sustainable did so by leveraging their already-existing strengths.

The first step down the path of creating a sustainably funded organization involves acknowledging that many of the positions outlined in PSA policies are best served, not as volunteer positions, but as paid positions. We must also acknowledge that the costs of paying for this labor may be high. If we assume that all 29,900 hours are paid, and paid at even a very low wage ($7.25/hour, or US minimum wage), we still get a labor cost of at least $216,775 per year. This does not consider taxes, vacation, or other overhead. 

Of course, it may not be necessary to pay for all 29,900 hours, either because our staffing estimates are inaccurate, or because our labor becomes more efficient when we switch to a paid model. Yet the mere act of thinking through these considerations requires recognizing that the PSA’s maximalist vision has a financial cost.

What we can be is constrained by our ability to obtain resources

The vision of the PSA outlined in its founding documents is grand. If implemented successfully, this vision could have an impact on psychology that is transformative, creating a science that is more inclusive, more collaborative, more transparent, and more robust.

Yet the PSA cannot realize this vision of itself for free. Currently, the PSA attempts to be a maximalist institution on a minimalist budget. That has worked during the PSA’s early years, but such a model may not be sustainable long-term. If we wish to implement a maximal vision for the PSA, we will need to focus dedicated energy into obtaining the funding needed for this implementation. As we will describe in a follow-up post, this will likely require developing funding streams outside of the traditional grant mechanisms to which scientists are accustomed.

—
Funding Note: Patrick Forscher is paid via a French National Research Agency “Investissements d’avenir” program grant (ANR-15-IDEX-02) at Université Grenoble Alpes awarded to Hans IJzerman.

This is a cross post from the PSA blog.

Patrick S. Forscher, Bastien Paris, and Hans IJzerman

How many resources does the PSA possess? This is a question that affects many activities within the PSA — prime among them the annual decision of how many studies the PSA is able to accept. Here are a few other examples:

• People involved in the selection of studies must decide whether the PSA can feasibly support studies with special requirements, such as a proposal to conduct a multi-site EEG project or a project involving a low-prevalence (and therefore difficult to recruit) population. 
• People involved in writing PSA-focused grants must be able to accurately describe the size and scale of the network to make their grant arguments and planning concrete. 
• People involved in managing the PSA’s finances need to know the people and projects that have the highest financial need. 
• People involved in regional recruitment need to know how many members are currently located in a specific world region and the number of participants those members can muster for a typical PSA study. 

In its first three years, we have had to rely on ad hoc sources to answer questions about PSA resources. Today, with the release of the PSA’s first study capacity report, we now have a source that is more systematic. This blog describes the logic that underlies the report, gives some of its top-level findings, and outlines what we plan to do with the report now that we have it.

How to think about and report on PSA resources

The PSA’s most basic activity is running multi-site studies, and one of the most fundamental resource-dependent decisions PSA leadership must make is how many proposals for these multi-site studies the PSA will accept. Thus, a single multi-site study provides a useful yardstick for measuring and thinking about PSA resources.

The PSA’s newly-ratified resource capacity policy takes just such an approach. It considers PSA resources from the perspective of helping PSA leadership decide how many studies they should accept in a given year. From this perspective, the most basic unit of analysis is the study submission slot, a promise by the PSA to take on a new multi-site study. Study submission slots are limited by at least two types of resources:

1. Data collection capacity. This is the PSA’s ability to recruit participants for multi-site studies. Data collection capacity is mainly governed by the number of PSA members located in psychology labs throughout the world. However, money can also expand the PSA’s data collection capacity; the PSA has occasionally contracted with panel-provider firms to recruit participants on its behalf.
2. Administrative capacity. This is the PSA’s ability to perform the administrative tasks required to support multi-site studies. Administrative capacity is mainly governed by the availability of labor, whether that labor be paid or volunteer.

The resource capacity policy also allows for the possibility of study slots that add on special requirements or evaluation criteria. These special submission slots might require, for example, that any studies submitted for consideration to that slot involve EEG equipment. Alternatively, the slots might require that the submitted studies involve investigating the psychological aspects of the Covid-19 pandemic. We will go into more detail about how we think about these special submission slots in a later post. For the time being, we simply note that assessing our resource capacity will allow us to understand the sorts of special submission slots we can accommodate.

The PSA’s data collection and administrative capacities are both in flux. The PSA’s ability to accommodate more specialized types of studies also fluctuates on a yearly basis. Moreover, the PSA is committed to the cultural and national diversity of psychology research — activities that are dependent on its reach in under-resourced countries. Accurate assessment of all these capacities therefore requires ongoing documentation of its members, member characteristics (including country of origin), and its yearly activities. Currently, our documentation happens in a shared Google Drive, Slack, the PSA’s OSF project and the subprojects for each of its studies, and the recently-created PSA member website.

According to policy, these various sources of documentation are consulted in a comprehensive way to form a complete picture of the PSA’s resources. This consultation results in an annual  study capacity report, which can inform decisions and activities involving the PSA’s resources.

Findings from the first study capacity report

The first PSA study capacity report is large and comprehensive. Here are some big-picture findings:

• The PSA currently has 1,400+ members from 71 countries.
• Out of seven studies, six are still underway collecting data.
• Based on our past data collection capacity, we have the ability to recruit a minimum of 20,000 participants over the upcoming scholarly year for new PSA projects.
• Two out of three PSA members come from North America (24%) and Western Europe (41%). 
• We do not have sufficient information to accurately estimate the number of administrative hours available for each PSA role.

However, these big-picture findings hide a lot of important detail that may be important for PSA decision-making. For example, here are a few additional tidbits that come out of the report:

• The number of PSA member registrations almost tripled as a result of the COVID-Rapid project.
• At the time of the report’s writing, and excluding PSA007, the PSA will need to recruit 30,000 participants to complete its active roster of projects.
• About 20% of the PSA’s membership have a social psychology focus area.
• About 90% of people in active PSA administrative roles are located in North America (63%) and Western Europe (21%).

If you’re interested in digging into more of these details, you can find the full report here.

What’s next?

As outlined by policy, the main purpose of the report is to inform decisions about how many studies the PSA can accept in the next wave of study submissions. Thus, an important next step for this report is for the upper-level leadership to use the report to come to a decision about study submission slots.

However, the study capacity report has already catalyzed a number of ongoing conversations about what the PSA is, what it should be in the future, and how the PSA should go about meeting its aspirations for itself. Some of these conversations have resulted in their own dedicated blog posts, which will be posted to the PSA blog in the next few days.

In the meantime, we welcome your thoughts about the PSA’s study capacity and issues related to it. We believe that compiling this report has been a useful exercise precisely because the process of compiling the report has inspired so many useful conversations about the PSA’s direction and goals. This reinforces our commitment to maintaining this useful reporting structure in future years.

—
Funding Note: The study capacity report was made possible via the work of Bastien Paris; his internship at Université Grenoble Alpes is funded by a grant provided by the Psychological Science Accelerator. Patrick S. Forscher is paid via a French National Research Agency “Investissements d’avenir” program grant (ANR-15-IDEX-02) awarded to Hans IJzerman.