What research methodology do most social psychologists use?

Perspect Psychol Sci. 2018 May; 13(3): 359–372.

Abstract

We propose a SAGE model for social psychological research. Encapsulated in our acronym is a proposal to have a synthetic approach to social psychological research, in which qualitative methods are augmentative to quantitative ones, qualitative methods can be generative of new experimental hypotheses, and qualitative methods can capture experiences that evade experimental reductionism. We remind social psychological researchers that psychology was founded in multiple methods of investigation at multiple levels of analysis. We discuss historical examples and our own research as contemporary examples of how a SAGE model can operate in part or as an integrated whole. The implications of our model are discussed.

Keywords: SAGE model, qualitative methods, quantitative methods, history of psychology, social psychology

We believe there is, but should not be, a separation between quantitative and qualitative social psychological research. Both methodological approaches should work more harmoniously. Yet there is a persistent tension between the two (Shweder, 1996). These tensions can be understood on two levels: ontological and practical. The SAGE (synthetic, augmentative, generative, experiential) model of social psychological research is a novel framework that can potentially overcome these methodological divides on a practical level. The synthetic model provides a novel integrated mixed-methods approach to guide how social psychological research can be conducted.

To motivate our SAGE model, we begin by delineating the specific way we invoke quantitative and qualitative methods. This usage is based in the practical research methods that fall within each designation, although we also acknowledge the ontological differences. Next, we draw on the historical roots of psychology to demonstrate the potential of psychology as it has been conceptualized historically. We advance these foundational roots by articulating a synthetic mixed-methods model for social psychological research. Next, we detail our SAGE model for social psychological research. After describing the entire model, we discuss each segment more closely. We end by discussing some implications that this framework could have on alleviating salient concerns in contemporary social psychological research and for advancing psychological science.

Qualia and Quanta: Mixed Methods in Practice

Quantitative methods place emphasis on sampling, comparing, counting, calculating, and then abstracting. In contrast, qualitative methods tend to privilege understanding experiences and meaning-making processes through contextualization, interpretation, narration, and exemplification. There is no practical reason that both forms of methodology should not inform one another to increase holistic understanding of social psychological phenomena. We think that many social psychology researchers would agree with this idea. However, on a practical level, qualitative and mixed-methods research is rarely featured in premier social psychology journals.

To illustrate this point, we followed the example set by the Open Science Collaboration (2015). These researchers aimed to replicate findings from a sample of articles in three premier psychology journals. We examined the same three journals for the methodologies used in all published empirical articles in these journals in 2016. The three leading journals were Psychological Science, Journal of Personality and Social Psychology, and Journal of Experimental Psychology: Learning, Memory, and Cognition. Although there was variance in the types of sampling, data sources, design, analysis, contextualization, and interpretation, there was one common theme in all the empirical articles. No purely qualitative articles were published. Although some empirical papers published in 2016 in these journals used mixed methods, none explicitly drew on ethnographic observations or qualitative interviews to generate experimental hypotheses or to augment statistically reasoned conclusions.

The premier journals in psychology privilege quantified data and statistical analyses. There are good reasons for quantification: Controlled experimentation, hypothesis testing, and rigorous analyzing are essential in conducting good science and amassing knowledge. Moreover, both qualitative and quantitative researchers have a variety of journals in which to publish their findings. However, an ethos of complementary and mixed-methods approaches is often absent, certainly from the leading social psychological journals. On a practical level, social psychological research privileges quantified data and statistical analyses over qualitative data.

The persistent tension between the two forms of psychological inquiry might best be explained on an ontological level rather than a practical level. The basic difference between qualitative and quantitative methods lies in the nature of the objects they study and their subject matter: qualia and quanta (Shweder, 1996).

From an ontological level, quantitative research examines what is left behind once the world is rid of subjectivities. These objects, events, and processes exist beyond human knowledge and awareness of them. Biology and physics are in this realm. For example, DNA and black holes existed before humans discovered them (Rozin, 2009). In contrast, qualitative research is based on the notion that this view of the world is incomplete. The content of qualia is the realm of experience, meaning making, and intersubjectivities. It is the world of concepts, cultures, self-awareness, and representation. Humans think, feel, want, value, and moralize (Bruner, 1990; Cole, 1996; Markus & Kitayama, 1991; Shweder, 1991, 2003; Wierzbicka, 1993). From this point of view, qualia are added to what is really real, making the world more complete. Qualitative methods thus aim to elucidate and understand this more complete and complex world. Quantitative methods take you only so far in the world of qualia. Humans—and their subjective worlds—need to be understood in social, cultural, historical, economic, political, and legal contexts (Asch, 1952/1987; Power, 2011; Rozin, 2001, 2009; Shweder, 1991). Qualitative and quantitative methods are sometimes confounded with descriptive and experimental. This is not the case in biology, where quantitative methods are often used in description. The description of the double helix of DNA, for example, used quantitative methods (Rozin, 2001). In social psychology, the confounded link between qualitative methods and description, and quantitative methods and experimental procedures, is more obvious. Yet, quantitative procedures can provide useful descriptions of social psychological phenomena, and qualitative ones can provide critical insights to explain results obtained from experiments. For example, quantitative coding highlighted the content and severity of patients’ letters of complaints in hospitals in the United Kingdom to help generate safer and more satisfactory health care systems (Gillespie & Reader, 2016). We highlight other examples that overcome this confounding throughout the article.

From an ontological perspective, if you are a true quantitative researcher, you study quanta. You believe in understanding the world devoid of the content of qualia—human experiences, meaning-making processes, and the role of context. If you use quantitative methods to understand, manipulate, count, or measure qualia, then, from an ontological perspective, you are not studying quanta at all. You are studying qualia with quantitative methods (Shweder, 1996). The majority of studies published in the three premier journals in 2016 were quantified studies of qualia. However, by using methods that are suited to understanding only the content of quanta, there may be incommensurability between quantitative forms of inquiry and the phenomenon being studied.

Qualitative procedures can be used before, during, or after formal quantitative methods. However, they are rarely used systematically or acknowledged in hypothesis generation, testing, or the interpretation and presentation of results. Yet, qualitative methods are needed to create a more accurate, complete, and holistic account of social psychological phenomena. We admire the quantitative work being conducted by our colleagues in social psychology. Each of us has conducted our own fruitful quantitative research. Our aim in this article is not to essentialize research positions or make a political argument about journal space, funding allocation, or hiring practices. We are not suggesting that all social psychological research needs to use mixed methods. Our aim is to provide an integrative framework to guide social psychological research using multiple methods and to make a case for the importance of this approach.

We acknowledge that other researchers have discussed mixed methods in social psychology (Campbell & Fiske, 1959; Denzin & Lincoln, 2000; Gergen, Josselson, & Freeman, 2015; Tashakkori & Teddlie, 2010; Wundt, 1897). However, we believe we are the first to present a novel integrative model that privileges the synthesis of qualitative and quantitative methodologies at multiple levels of analysis. Our focus in this article is not to rehash older mixed-methods debates. Rather, our concern is to describe a novel model to conduct social psychological research by synthesizing previously fractured conceptualizations of mixed methods with the ultimate goal of creating a more holistic model for social psychological research. Our aim is to articulate a SAGE model for social psychological research that is capable of facing and overcoming the many problematic issues concerning replication, validity, and ultimately, the utility of our discipline that was once envisioned but not fully realized in early conceptualizations of psychological science.

The philosophical assumption underlying our model is that although there are ontological differences between quantitative and qualitative methods in social psychological research, each can, and should, complement the other in practice. Multiple methods are indispensable if social science is to advance and deal with the pressing social issues and social psychological phenomena located in the world of qualia. A stubborn and misguided division exists between qualitative and quantitative methods. It is an ontological divide and can often manifest at an epistemological level. This is evidenced in the lack of qualitative studies in the premier journals. Moreover, the establishment of journals publishing only qualitative social psychological work, as a response to the absence in other journals, is another manifestation of this separation (Gergen et al., 2015).

A Synthetic Approach to Research Practice

Our article presents a novel way of overcoming this philosophical problem on a practical level. We believe that there is the necessity for an integrative model in psychology because, currently, the false division between qualitative and quantitative methods must be bridged to advance rich, validated, and insightful scientific knowledge in the field. Our acronym, SAGE, refers to the ways in which we see the utility of qualitative methods in relation to quantitative ones in order to advance research in social psychological science. Within this synthetic model, we highlight three ways in which qualitative methods complement and add to quantitative methodologies in the investigation of social psychological phenomena.

First, qualitative methods are augmentative. They can build on quantitative data collection techniques to deepen knowledge about social psychological phenomena. One function of qualitative methods is thus to expand on quantitative approaches. Second, qualitative methods—used for exploratory psychological research projects—can be generative of new experimental hypotheses. Third, qualitative methods can legitimately be used independently within social psychological research to understand experiences that cannot be understood from experimental reductionism or numerical abstractions. We include experiences within the model because of the utility of solely employing qualitative methods to address particular qualia-oriented research questions. Nevertheless, as with the other parts of the model, this application is part of a broader process; taken together, a synthetic model that encompasses qualitative methods through augmentative, generative, and experiential processes is the SAGE model of social psychological research.

We recognize that mixed methods are a topic with rich research and theoretical literature. Not only does our approach build on this work by providing a new integrative model that we believe can help address issues facing the discipline in this moment, but our model also returns to the foundational principles of psychology to justify such an approach. We argue in the following section that the core methodological conceptualization of psychology pointed toward a potential that is unrealized in the current climate. After explaining this perspective, we then present our model as a path toward realizing that same potential within a contemporary context.

Remembering Visions for the Future: The History of Social Psychological Research

Many researchers have written extensively about the history of qualitative methods in psychology (see Gergen et al., 2015; Rozin, 2001, 2009; and Seligman, Railton, Baumeister, & Sripada, 2013); about the myriad types of qualitative methods, including their utility and the research traditions they inspired (Denzin & Lincoln, 2000; Giorgi, 1999; Laverty, 2003; Moghaddam & Harré, 1995); about the philosophical bases of the qualitative and quantitative divide (Shweder, 1996; Yoshikawa, Weisner, Kalil, & Way, 2008); about the importance of combining the two (Bryman, 2006; Moghaddam, Walker, & Harré, 2003; Onwuegbuzie & Leech, 2005); and about the resurgence of qualitative methods within the discipline of psychology (Gergen et al., 2015). The most comprehensive review of the possibilities and practicalities of mixed methods in psychological research is provided by Tashakkori and Teddlie (2010). These authors surveyed the strengths and limits of mixed-methods approaches to understanding phenomena across the social and behavioral sciences. However, we synthesize and extend this research by presenting a novel model based on previous conceptualizations of mixed-methods research.

At the beginning of its development as a discipline, psychology’s methodologies and formulations were steeped in tensions and possibilities of integrating varied approaches. In Outlines of Psychology, Wundt (1897) detailed a twofold vision of psychology. First, Wundt proposed that basic causal processes of psychophysical experience were to be determined by careful laboratory experimentation. This process entails manipulating independent variables and quantifying observed changes in dependent variables. Second, he also articulated a version of psychology aimed at understanding higher order experiences within diverse social contexts. He advocated for the use of observational and ethnographic techniques to understand people in cultural contexts. The dual approaches were intended to be complementary (Ellis & Stam, 2015; Trinidas, 2007).

However, these dualities were never fully realized in his time. His students at Leipzig opted for basic psychological research conducted in the laboratory and focused on quantitative measurement. American psychologists appropriated only a part of his vision for a scientific psychology analogous to contemporary approaches in the “hard sciences,” such as physics. Emphasis was placed on laboratory experimentation. This formulation of social psychological research—the careful manipulation of independent and dependent quantifiable variables in the context of the psychological laboratory—is most dominant today, particularly in U.S. social psychological research (Power, 2011; Rozin, 2009). The rise of behaviorism reified the experiment and marginalized the importance of understanding context (Rozin, 2001). The gestalt movement, with its focus on context, failed to gain predominance in psychology outside the area of perception. The marginalization of context has been lamented in social psychology (Asch, 1952/1987; Power, 2011).

This emphasis on quantitative, laboratory-based research has challenged the importance of qualitative methods. A further devaluation of qualitative methods has emerged from fundamental critiques that question the possibility of empirical social psychology at all. These arguments focus on various difficulties in studying social psychological phenomena, such as the high number of variables influencing phenomena in the world of qualia, the constantly changing shared meaning systems, and the uniqueness of particular times and places (Schiff, 2017). The randomized controlled trial is one research design used to address these concerns, but this practice can come at the expense of practicality and applicability to real-world issues (Power, 2011; Rozin, 2009; Smedslund, 2009). We believe that a new conceptualization that describes and justifies the synthesis of quantitative and qualitative methods may offer a more nuanced and integrated approach to addressing these critiques of empirical social psychology. Our novel model draws on the historical roots of social psychology, and engages with contemporary issues that highlight limitations of the mainstream approach, to conceptualize the framing of quantitative and qualitative research in contemporary social psychology.

The SAGE Model of Social Psychological Research

Although using different terms and under different frameworks, the augmentative, generative, and experiential potential of research can be argued to already be used in some form in classic social psychological research (e.g., Festinger, Schachter, & Back, 1950; Nisbett & Cohen, 1996; Sherif, Harvey, White, Hood, & Sherif, 1961; Shweder, 2003). Nevertheless, we conceptualize these previously unintegrated processes as an interrelated, unified, and novel whole in a synthetic model. To describe this relationship, we first present our conceptual model and then demonstrate its applicability through an example from our own work. Following this explanation, we briefly detail each of the three components and their bases in historical and modern examples in social psychological research.

Our SAGE model, as depicted in Figure 1, provides a conceptual framework for how dynamic and integrative research producing valid and rich insights could be carried out. Although our model reads left to right, it is a recursive, bidirectional, and dynamic approach. That is, we see a synthetic approach as continually and consistently using different methodologies to check assumptions, research questions, findings, and interpretations. Answers to pressing questions can be triangulated using a recursive combination of methodological approaches and techniques. In this way, the limitations of a single method can be overcome. The complementarity of such research findings can create more nuanced, replicable, and ecologically valid research findings. Therefore, it can lead to sophisticated theoretical growth. Contradictory results could also contribute to achieving this goal. They help question fundamental assumptions behind results from single studies or from single methodological investigations of the same topic. To this end, the SAGE model advances social psychological science by mapping the combination of potentially complementary and contradictory results from multiple perspectives, using multiple methodologies, as a dynamic system that is employed sequentially or concurrently. In the midst of the current emphasis on quantitative methodology, the SAGE model is a framework to help researchers think about the qualia: the world of cultures, representations, meanings, experiences, economics, (in)justices, and laws. It is a framework to more fully understand subjective worlds and lived experiences. We believe that social psychology is too interesting and important to be left to a limited approach.

The SAGE model of social psychological research.

Our SAGE model, as depicted in Figure 1, provides a conceptual integrative framework for how more holistic social psychological research could be carried out. Reading from left to right, we begin by acknowledging and situating qualitative and quantitative research within the historical context of our field (A in the diagram). The field of psychology has developed a rich methodological and theoretical history that informs approaches that researchers can take in studying and understanding psychological phenomena. However, although these approaches have ontological differences, they have been separated on a practical level.

Our integrative model emphasizes the importance of dynamic, inter-enforcing research programs to develop social psychological research. Multiple methods overcome limitations of singular approaches when used in synthesis (B in the diagram). Quantitative methods, grounded in hypothesis and prediction, experimental work, and the manipulation of abstracted variables in controlled research designs, can be fruitfully combined in various ways with qualitative methods. In our model, we highlight augmentative, generative, and experiential aspects of our model to show how both methodological forms can work together (even if not harmoniously). Our SAGE model highlights the potential of synthesizing and connecting quantitative and qualitative in a dynamic model in which multiple methods inform findings from a single methodological approach.

When this occurs in research practice, it can drive empirical findings and theoretical developments that are insightful, groundbreaking, valid, and nuanced to the diversity and complexity of lived social psychological phenomena (C in the diagram). This relationship between methodology and findings is bidirectional; we employ a recursive loop and two-sided arrows to demonstrate how this approach continually develops as findings and lessons can inform more effective triangulation of various methodologies in studying psychological phenomena.

The model is underscored by a broader temporality: To move forward and create more replicable, ecologically valid, rich social psychological knowledge, it might be useful to consider the foundational visions of the discipline (D in the diagram). These legacies inform the different possible options for a particular research project, which is understood in our model as the “present.” An investigator forms a research question and plans out a methodological approach in relation to an area of interest, previous literature, and historical foundations. These possibilities may include quantitative and qualitative, or a mixture of both. In much of the current psychological research, this progression is linear; a chosen approach (i.e., an experiment with quantitative data) produces findings that are then validated through further studies using the same approach or a similar one in different contexts or with slight variations.

In contrast, we argue that these different approaches should be used in synthesis and should be returned to once a given experiment, study, or investigation produces findings. Any methodology should not be used simply in isolation, but its application, applicability, and guiding research questions should draw on insights and possibilities from other approaches. For example, researchers conducting interviews can begin with ethnographic work to better understand the contextualization of the questions they will be asking, or a quantitative survey can be developed first with cognitive interviewing to explore participant understandings of questions and response options.

The findings from these preliminary studies are then further developed by testing them through other methodologies. In our conceptual model, this step is conveyed through the circular arrows indicating that synthesis produces findings that inform procedures, research questions, and ideas and measures for new studies. Employing this loop can lead to insights that are more influential, ecologically valid, and nuanced because understandings are triangulated through various methods, at different levels of analysis, and from multiple perspectives. Social psychological phenomena are thus understood in more reliable and ecologically valid ways. These outcomes are detailed on the far right in the future section of our model, although the arrows returning to the present are meant to underline the continual nature of this process.

Each area of the model is not isolated, static, or all-inclusive; we seek instead to conceptualize the potential of synthesizing and connecting quantitative and qualitative, as well as incorporating new approaches within each. When this occurs in research practice, it can drive empirical findings and theoretical developments that are insightful, groundbreaking, valid, and nuanced to the diversity and complexity of lived social psychological phenomena. We use a recursive loop and two-sided arrows to demonstrate the continual development because findings and lessons can inform more effective triangulation of various methodologies in studying psychological phenomena.

SAGE in Practice

In practical terms, the SAGE model should be used at the development and design stage of a project and again once data are collected and interpreted. For example, a researcher interested in a psychological construct related to experience should ask whether there are phenomenological features of that construct alongside questions of mechanism, frequency, and interactions among other constructs. If so, how might the study of that subjective experience shade or alter possible aggregate findings or measures? Focus groups, cognitive interviews, or behavioral observation can help better understand how participants may answer or interpret questions and whether the proposed measures or tasks are ecologically valid. Furthermore, contrasting interview data and responses on standard metrics may reveal disparities between a participant’s perceived and actual behavior, suggesting desirability effects or blind spots. These mixed data will enable a cautious, holistic design that captures a fuller scope of the phenomena of interest.

Once data have been collected and interpreted, the new insights they provide should inform design retroactively as much as they should inform new research directions. For example, should interview data reveal a trend, that trend should be systematically inspected with a larger sample to determine whether it is more broadly applicable. Large-scale survey data could be broken down in small-sample interviews to investigate subgroups more precisely and to better clarify interactive effects. Behavioral outcomes of experiments may reveal situations in which the same phenomena could be studied in more naturalistic settings. Obviously, every study will be different and require different tools: The SAGE model simply asks that researchers consider their methods as continually under revision and as a fluid component of research. To describe this relationship, we draw from a concrete example to demonstrate the utility of qualitative methods in relation to quantitative approaches.

Power (2015, 2016, 2017, in press) investigated how people understand and experience the aftermath of an economic recession with a particular focus on the localized Irish context between 2008 and 2016. To expand on his initial observations—that unlike some of their European Union neighbors, the Irish did not riot or protest with the introduction of harsh austerity measures after the 2008 worldwide financial crisis—he performed analyses of preexisting data from the European Values Survey (Power, 2015, 2017). These analyses supported initial observations that the Irish case was different from comparable European neighbors, such as Spain, on a number of salient social and political issues, and it warranted closer attention.

To augment this finding, he interviewed a group of people in the public eye in the Republic of Ireland. Analysis of these qualitative data from this group of elites sparked further questions about class. Power therefore interviewed a polar opposite group—unemployed young Irish people—about their economic recession experiences. Members of both groups were interviewed to investigate how they explained the initial passive response of the Irish to austerity, and thematic analysis of interview data demonstrated that the answer to their experiences during the economic collapse lay in culturally ingrained moral logic: In life, “you reap what you sow.” In the Irish context, the implication is that Irish people initially felt partly culpable for the economic crisis—enough to think they ought to tolerate the introduction of some austerity measures—and, therefore, they did not riot or protest. These interviews thus generated a hypothesis about how Irish individuals made meaning of what was occurring in their society.

Following the dynamic nature of a SAGE approach, these findings could be bolstered with a culturally sensitive experiment. It is possible to prime one group of Irish participants to think that in life you get what you deserve relative to the other group. Then, these participants could be asked a series of questions relating to support of civic unrest, the fairness of austerity, and the attribution of blame for the economic crisis. The underlying hypothesis would have been difficult to formulate—especially with precise and culturally meaningful independent and dependent variables—without prior ethnographic fieldwork.

We believe that the results from this experiment reveal what some might consider a weakness of our SAGE model but that we consider a strength. The results from two priming experiments did not support the insights from the qualitative research (Power, 2017). There are multiple reasons for possible incongruent results between qualitative and quantitative investigations into the same phenomena. In the Irish case, the economic and political context had shifted from the time the interviews were conducted to the time the experiments were run. The timing of investigations is important, and in an ideal world, both qualitative and quantitative research projects—particularly in unfolding political, social, and economic contexts—should be conducted concurrently, not sequentially.

In the Irish case study, the contradictory results from multiple psychological studies could then be used to construct similar experimental hypotheses in other cultures and in related contexts, such as the ongoing European financial crisis. Moreover, researchers could draw on these results to develop new survey instruments to gather greater amounts of culturally sensitive data on societal phenomena of interest. These data, gathered from broad, random, and representative samples, could in turn be deepened via ethnographic work and interviewing. Analyses of these qualitative data would provide thick description about the lived experience of a specific cultural group. Indeed, in this example, further ethnographic research was motivated by the unpredicted experimental results to reveal why some Irish people no longer tolerated austerity but began protesting during an economic recovery (Power, 2017, in press). This ethnographic work helped make sense of previous contradictions between the qualitative research with the elites and the unemployed and the experimental studies. The cycle of research could continue and further hone understandings of the phenomena. This integration has the potential to synthesize different methods that are often used in isolation or only in reference to one of the three categories we have discussed. We believe that this synthesis would advance social psychological research.

This use of the SAGE model is just one example of an application of our model. Our argument is that combining some methods is potentially more beneficial for our science. In the forthcoming sections, we provide greater detail of what these methods may entail in order to clarify the individual segments—augmentative, generative, and experiential—of our SAGE model. These individual sections highlight the importance and possibilities of using mixed methods—various combinations of qualitative and quantitative research methods to advance social psychological research of various phenomena—but our overall goal is not to simply dwell on these points but to concretely illustrate how aspects of the SAGE model can and have been applied previously and to then focus on the synthetic integration of these three individual segments. We follow these sections by combining these aspects and articulating the importance of the integrated SAGE model for contemporary problems in our field.

Augmentative

One function of qualitative methods is to deepen our understanding of findings generated by quantitative procedures, both experimentally and in survey items. In this section, we detail the augmentative potential of qualitative methods in relation to the limits of current quantitative approaches.

In doing so, we argue that quantitative methods are not in opposition to qualitative approaches, despite the fact that they are based on different approaches to studying phenomena on an ontological level. Furthermore, we acknowledge that quantitative methods can build on qualitative methods as well, but as quantitative approaches are often used in isolation, we stress the augmentative capabilities of qualitative methods to bring attention back to this relationship. According to our SAGE model of research, qualitative methods can do so by strengthening inferences, discovering new variables, and offering a thick description to explain—and expand on—what may be narrow or decontextualized quantitative-based findings. Moreover, qualitative research can help explain survey items that are often based on statistical aggregates by providing in-depth investigation of meaning making and actual human experiences in ecologically valid contexts (Oishi & Graham, 2010; Smith, Spillane, & Annus, 2006; Weinfurt & Moghaddam, 2001). As an example of this potential, Weinfurt and Moghaddam (2001) demonstrated that the quantitatively validated Social Distance Scale actually depends on participants’ belief that social distance increases as one moves from talking about a family member, friend, neighbor, coworker, and citizen. They accomplished this through a structured interview approach with a diverse group of first-generation immigrants in Montreal. As in this case, qualitative data can reveal new and often vital ways of interpreting quantitative data. We present two historical examples and an additional case from our own work to demonstrate how this augmentative process can improve validity of research findings.

Drawing on work from social psychology’s past, one classic example of the augmentative role of qualitative methods in psychology is The Robbers Cave experiment (Sherif et al., 1961). The researchers used a quantitative experimental design but ultimately used qualitative methods to examine the conditions under which two conflicting groups could be created and then be made to reconcile with one another in a real social context. The researchers created two matched-pair groups of boys who each occupied one side of a camp. They contrived scenarios, which were salient to the children, that produced conflict and then solidarity between the groups. The researchers then observed behavior using in-depth qualitative interviews and ethnographic descriptions. As a final step, they employed scenarios that emphasized cooperation across the two groups to achieve goals beneficial to each group of boys. Again, the researchers charted the boys’ cooperative behavior using observational qualitative methods.

The Robbers Cave experiment highlights how social psychologists can use the design and sampling techniques of quantitative experimental studies—mostly conducted in laboratory settings—and then delve more deeply into exploring the phenomenon in a real-life context with qualitative methods and data. They overcame a potential confounding of quantitative procedures with experiments and qualitative methods with description. This approach provides ecological validity to the study of a social phenomenon in a setting that allows for the nuances of meaning making, perspective taking, and overt behavior to be recorded qualitatively.

Another augmentative purpose of qualitative methods is to provide richer descriptions of emergent phenomena. A historical example is of the authoritarian personality (Adorno, Frenkel-Brunswik, Levinson, & Sanford, 1950). These researchers were interested in understanding the emergent phenomenon of fascism in the post-WWII context. They developed the infamous F-Scale, a series of nine survey questions that clustered together to indicate support for fascism. However, to investigate how people understood and made sense of their perspectives on fascism, these researchers also conducted in-depth interviews with participants. The interviews were critical to creating and validating the questionnaire because these conversational data revealed how the participants understood and interpreted the items. The methodological approach and the findings presented in this research have been critiqued over the years because of a lack of random sampling (Christie & Jahoda, 1954). However, the study highlights the importance of qualitative methods to comprehend how individuals understand emergent phenomena in context.

In this way, qualitative methods can be augmentative by offering stronger validity to quantitative findings. Quantitative research often verifies and builds theoretical understandings by demonstrating interactions between independent and dependent variables in predictable ways. For quantitative research, maximizing internal and external validity is essential. The former is the condition that observed differences in the dependent variable are a direct result of the independent variable identified (Gay & Airasian, 2000). Nevertheless, events in the world—in ecologically valid contexts—often defy clean measurement (Johnson & Christensen, 2000; Schiff, 2017). Social interactions, rituals, convictions, and other ephemeral aspects of the world of qualia can be challenging to recreate in a laboratory setting. Therefore, qualitative methods can strengthen the validity of quantitative instruments and, by extension, their explanatory power by providing additional means to measure psychological variables and to confirm or challenge quantitative measures (Johnson, Onwuegbuzie, & Turner, 2007).

An example from our own work demonstrates this augmentative capability in developmental psychology. In a study on children’s epistemological views, it became clear that children do not exclusively learn and take on social roles in pretest/posttest settings, as assumed by existing, quantitative literature. Instead, these processes occur through interactions with parents, friends, and environments. Whereas traditional quantitative developmental research on learning has discovered a tremendous amount, these measures were incomplete in understanding this phenomenon (Qadafi, 2017).

Similar to the historical paradigms, this example illustrates how qualitative methods can be augmentative of quantitative research. The qualitative analysis generated from in-depth focus groups tested the validity of the quantitative results derived from the initial surveys, adding in-depth understanding of experiences and meaning-making processes of adolescent Black males. Moreover, the qualitative findings offered alternate explanations—not captured by the survey—that showed that these adolescents were using a multitude of coexisting epistemological beliefs. A study that simply relied on standard measures, as internally valid as they are, would have poorly represented the thinking of these students. A mixed-methods approach offered insights into how context changes the epistemological style employed by adolescents (Qadafi, 2017).

Generative

A fruitful combination of psychological methods is the use of qualitative methods to create experimental hypotheses or survey instruments. This generative approach is a different process from the augmentative process outlined in the previous section because it creates rather than refines explanations, theory, and instruments. However, we do not contend that the two processes are mutually exclusive. Rather, we conceptualize them as informing one another, working together as part of a broader investigative process. We acknowledge that experimental manipulations and analysis of statistical aggregates have proven to be generative of insights and theory, but we also wish to stress that these findings can be part of a broader, synthetic model of social psychological research.

Qualitative methods provide rich description and a flexibility of categories that make them suited to generate and develop theories and lines of research. As the social scientist faces a disjuncture or incomplete knowledge, she or he is uncertain of what phenomena or explanations will emerge (Shweder, 1997). Whereas a quantitative approach requires that the researcher-as-explorer assign possible categories to the outcomes from the outset, a qualitative approach is both malleable and sensitive to the relevant context of the subject (Henwood & Pidgeon, 1992). Qualitative study can focus on gathering and capturing the unknown meaning-making processes among individuals and within cultures (Bruner, 1990). For psychology, such an awareness and exploration of cultural processes is integral: “Given that psychology is so immersed in culture, it must be organized around those meaning-making and meaning-using processes that connect man to culture” (Bruner, 1990, p. 12).

To illustrate this aspect of qualitative methods—that is, building explanatory theories of meaning making that are grounded in solid research principles and useful in applied settings—we again turn to a classic work. Social Pressures in Informal Groups exemplifies the ways in which initial qualitative research can produce new insights to be tested using statistically rigorous methods (Festinger et al., 1950).

In this study, the researchers took advantage of the construction of new housing estates at M.I.T. to study the formation of social groups. Two adjoining housing projects, named Westgate and Westgate West, were built and occupied by married veteran students. Westgate consisted of 100 single-family homes and was occupied according to position on a waiting list from spring 1946. Ten months later, Westgate West, 17 former navy barracks that had been converted to apartments, was occupied. There were few preexisting relationships between community members. Social life in the community needed to be generated. It provided ample opportunity for the researchers to investigate the formation and functioning of informal social groups. The authors’ approach to triangulating observations and interviews with surveys led to the development of an influential theory describing group structures and explaining the emergence of group norms.

Specifically, each step in their research addressed the shortcomings of the previous methodology. They started with observations and informal nonrandomized interviews, which then led to the development of a standardized survey that was administered to participants. Sociometric analyses of these data revealed the patterns of interaction between individuals in the two housing estates. The data allowed the investigators to identify the paths along which communication occurred between group members. Moreover, it made it possible to comprehend the genesis and spread of group norms and attitudes. Finally, the researchers used insights from their previous research to develop a field experiment to test hypotheses derived from this primarily qualitative work. The researchers planted rumors concerning media publicity of the new M.I.T. housing estates and recorded the spread of this information the following day across the estates. In this way, quantifiable field experiment data were collected in meaningful ways that were developed from previous qualitative work.

A second demonstrative example of the generative process comes from our work using qualitative inquiry in an exploratory phase and then as a basis for quantified analyses (Tennant, 2015). Tennant conducted a mixed-methods study of the morality of evangelicals and atheists in the United States. He began this research by immersing himself in salient media sources to construct a general sense of what each community was actively engaging with and what issues were of importance to them. Analysis of these sources revealed that both groups placed great importance on moral concerns in society and how these groups related to proscriptive moral claims and public policy. These materials, in conjunction with theological texts on morality and atheist moral texts, served as a foundational body of data for a mixed-methods study (see Harris, 2010; Plantinga, 2000).

These analyses led to the hypothesis that evangelical morality was primarily based on concerns about sin and divine design rather than harm. This finding contrasted with canonical psychological literature emphasizing harm as a universal or essential component of morality (e.g., Gray, Young, & Waytz, 2012). The resulting study consisted of structured interviews and a quantitative survey of demographics and behavioral self-reports. The analysis compared Christian and atheist participants on their moral judgment and justifications (Tennant, 2015). The interview questions were based on the body of data gathered by the initial qualitative research, citing examples of moral writings and opinions from various cultural leaders and news events. Using quantified interview data, qualitative codes, and compared counts of justification types, the study demonstrated that evangelicals employ nonharm morality frequently, even with regard to secular moral dilemmas. The study also found that even in the case of similar moral justifications, the actual content of the justifications varied widely. This suggests that larger cultural differences in belief about the origins of humans and the essentialness of gender influenced the participants’ moral reasoning.

The generative power of qualitative observations, synthesized with quantitative demographic controls and the quantitative coding of interview data, sheds light on two patterns of moral reasoning that were often conflated in the moral reasoning literature. The study is also an example of the ways in which quantification of qualitative data can describe social psychological phenomena. Next, we outline our last area of emphasis, experiential, before discussing some implications of our SAGE model.

Experiential

Augmentative and generative capabilities address the ways in which qualitative and quantitative methods can complement each other to provide validity and depth or to investigate a new phenomenon. As a third process in our synthetic model, qualitative methods offer a unique approach to study experiential phenomena that cannot be holistically understood by quantifying data obtained from investigating the subjective world of qualia. That is, these methods have a unique utility and efficiency in studying complex and dynamic lived experiences that can offer greater description than experiments or through quantification of qualitative data.

The tools of ethnography—inclusive of all forms of interviewing and observation—can build “thick description” that provides a rich, contextualized understanding of people, their perspectives, their personal and communal psychologies, and the cultural mentalities within a localized context (Geertz, 1973). Qualitative methods are uniquely appropriate for such thick description because they can be designed to capture participants’ subjective experiences and meaning-making processes at a level of analysis that evades experimental reductionism. This use of qualitative methods involves detailed and effective explanation of how research participants understand and view the world (Gelo, Braakmann, & Benetka, 2008). Qualitative methods allow the researcher to explore what is unique to the lived experiences of individuals and groups and explain these psychological phenomena as emerging from thoughts, feelings, and actions of embodied and culturally embedded human agents. These understandings remain grounded in the subjective and holistic experiences of people rather than dissociated “variables” that control for context (see Geertz, 1973; Schiff, 2017; Shweder, 1997).

Furthermore, experiential phenomena are not simply identified and explained but arise from synthesizing different accounts and perspectives over time through a variety of ethnographic methods. The researcher must establish that a phenomenon exists, study the conditions in which it occurs, and understand the various interpretations and meanings that individuals have of it. Qualitative research provides this by looking at how phenomena are understood within specific populations, social settings, and parameters (Henwood & Pidgeon, 1992). Reality emerges in the eyes of the participants as qualia that “can only be understood by reference to what they mean, signify, or imply (not in and of themselves and regardless of point of view, but rather) to us (or to me) in this or that time and place” (Shweder, 1996, p. 178).

This experiential capability is demonstrated by the classic study of a small religious group after a predicted apocalypse did not occur (Festinger, 1964). The researchers examined the lived experience of group members by becoming part of the sect. The methods involved participant and ethnographic observation. The research demonstrated that the dissonance of an unfulfilled prophecy did not destroy the belief systems of the group members. The researchers detailed how instead of rejecting the overall religious sect or altering their fundamental beliefs, these believers experienced the event by adapting their understanding of the underlying claim.

Whereas this historical example drew heavily on observational techniques, the flexibility of qualitative research methods provides an array of tools and approaches to explain experiential phenomena. In addition to interviews and ethnographic work, discourse analysis, narrative approaches, and textual deconstruction can provide important modes of understanding how experience is framed or understood. A rich literature details these approaches and their purposes (e.g., Charmaz & McMullen, 2011; Hammack & Pilecki, 2014; Parker, 1992; Potter & Wetherell, 1987). Potter and Wetherell (1987), for instance, argued that these qualitative inquiry methods can serve to illuminate how interpretation of social texts (both oral and written) provides insights into social categories, shared beliefs, causal processes, and historic events.

The human experience is a complex one, and social psychologists are challenged to understand and capture specific perspectives and ways of understanding the world of qualia. Qualitative methods can help researchers investigate how individuals and groups view, construct, and understand their worlds. For social psychological research, these methods offer unique advantages by providing flexibility and lenses derived from experiences and viewpoints of the participants themselves, rather than solely relying on preestablished categories, abstracted variables, and other criteria imposed by the researcher in advance. In this way, qualitative methods offer valuable approaches to augment quantitative findings, to generate experimental hypotheses, and to explain lived experiences.

Implications

In this article, we introduce the acronym SAGE as a broad, novel, and integrative framework for social psychological researchers to think through the uses of diverse methods in contemporary social psychological research. We purposely discuss classical studies in psychology to highlight how the synthetic approach is based in the discipline’s historical roots. This contextualization helps counteract a general contemporary trend toward prizing quantitative methods of inquiry at the expense of qualitative or mixed-methods investigations of psychological concepts and issues. We outlined this trend in our introduction by highlighting the lack of purely qualitative and mixed-methods articles using qualitative research in the premier psychological journals. This movement has been propelled by the tensions introduced into psychology by attempts to integrate cultural frameworks and studies (e.g., the labeling of such methods as “anthropologic,” the balance between generality and concrete, specific insights; see Moghaddam et al., 2003) as well as the historical developments that have led to classifying qualitative, quantitative, and mixed-methods researchers as distinct and isolated (Teddlie & Tashakkori, 2003).

Although quantitative and qualitative are different on an ontological level, we strongly believe that the prevalent contemporary conceptualization of these methods as being in opposition is misguided and negatively affects the development of social psychological science. Both forms of procedural inquiry can inform one another. Mixed methods can be used to overcome the limitations of one approach, from one angle, at one level of analysis. Qualitative methods can be augmentative to quantitative ones by moving beyond drawing inferences from survey and experimental data to capturing the meaning underlying statistical outputs. Qualitative methods can also be generative of new experimental hypotheses that can then be tested in laboratories, with quantitative data sets, and in the field. Finally, qualitative methods can be used to investigate and document experiential phenomena as lived, constructed, and comprehended by people in their unique sociocultural contexts.

The augmentative, generative, and experiential aspects of the methodologies discussed in this article can be synthesized together so that qualitative and quantitative methods can be used to explore psychological phenomena in a progressive loop. This would be a wise, or SAGE, model of social psychological research. We do not believe that our framework is exhaustive, nor do we believe that all research in social psychology should use mixed methods. We simply aim to highlight some benefits from using mixed methods and provide a framework to guide research in this tradition.

The advantages of our approach are to overcome the shortcomings of each social psychological method when used in isolation. However, conducting multimethod analyses may incur several drawbacks and multiple challenges: Multimethod research is more time consuming, requires further methodological expertise, and may struggle to find a home in journals that solely accept quantitative or qualitative methods (Yoshikawa et al., 2008). Being methodically fluid also requires time, practice, and broad expertise to master diverse procedural techniques and the integration of possibly contradictory findings from multiple angles at different levels of analysis. We argue that it can be beneficial to master these challenges and provide a framework to help scholars think through some relevant issues. Human life is complex, and we need methodologies and methods to study this complexity (Gillespie & Cornish, 2010). We offer examples based on our own empirical research to show the ways in which we have employed the SAGE model.

Mixed-methods research can lead to sounder and more nuanced social psychological research that captures people—and the worlds they inhabit—in more meaningful and in-depth ways (Campbell & Fiske, 1959). In turn, this has the potential for social psychology to inform policy creation and development. Most importantly, an integrative model encourages social psychologists to use a broader array of tools, perspectives, and methods to “follow an argument where it leads” rather than to create limited knowledge with narrow implications for our understanding of subjective worlds (Flexner, 1939; Power, 2017).

For example, this approach has implications for the “nudge” literature in behavioral science (Thaler & Sunstein, 2008). The next generation of nudge research needs to investigate why some cultural groups, and not others, can be nudged toward more prosocial activities. An application of the SAGE model of research can help explain cultural or group variance in relation to prosocial activities of interest within nudge projects. By extension, a serious application of our model would have potential implications for many prevalent issues in the 21st century: the support of more ethical and culturally plural societies, fairer and more equal distribution of economic resources, and changed behaviors to reverse the events of global climate change.

Throughout its history, psychology has been met with, and overcome, many crises. The latest one in this series revolves around the replication crisis: the failure to reproduce statistically significant experimental findings when the same design is again used with different people (Open Science Collaboration, 2015). We have two thoughts on this issue. First, the replication crisis is borne out of a narrow view of social psychological research, one that is focused exclusively on manipulating variables and prioritizes quantifying outcomes to comprehend human thoughts, emotions, and behaviors. Second, the SAGE model can help address this by supporting richer comprehension of social psychological phenomena. It provides a more holistic approach—focused on meaning-making processes and contextual understandings—of the participants and contexts in these experiments. Many issues can cause nonreplication, but we suggest that one possible reason that experiments may not replicate is a lack of acknowledgment of the different social, economic, cultural, and historical contexts in which experiments take place. We do not deny that there are many psychological universals, but universals are made manifest in localized contexts (Shweder, 1991, 2003; Shweder & Power, 2013). Our model offers a broad framework to think through using qualitative and quantitative methods in conjunction to design experiments that are more sensitive to the norms of different research populations and contexts. In this way, our methodological model can help address the replication crisis by broadening understanding of social psychological research and how different methods can be integrated to create more ecologically valid, in-depth, and nuanced research findings and theory.

A recent salient critique of psychology has been the focus on undergraduate students from U.S. universities as participants (Arnett, 2008; Henrich, Heine, & Norenzayan, 2010). Whereas this has been an issue in both qualitative and quantitative work, using the SAGE model can help overcome this limitation. The synthetic model of research, in which quantitative and qualitative methods are combined in various ways, has the potential to move away from solely sampling from this narrow and homogeneous slice of humanity. This is because ethnographic methods, including participant observation and interviewing, allow—and even necessitate—an awareness of the worldviews and meaning making of more diverse participants. When employed as part of a synthetic research process, these methods can bring greater attention to the particular time, place, cultural lenses, and perspectives of participants, which in turn may highlight the need for greater diversity. Experiments with undergraduates sampled from universities have value, but its use in psychological research is disproportionate to a more expansive methodological repertoire. This can be remedied not simply through qualitative methods but, specifically, by thinking through the SAGE model while conducting social psychological research.

Conclusion

Although there are deep-seated ontological reasons for a separation in qualitative and quantitative methods, these divisions should not manifest on a practical level. We have returned to the past and drawn on contemporary research to present a framework for thinking through methodological issues in advancing social psychological knowledge. Our SAGE model offers a new conceptualization of how researchers can employ these techniques in social psychological research. A SAGE approach to data collection and interpretation can aid social psychological investigations and help overcome limitations of single methods and related challenges. The hope is to try to realize the holism in social psychology that Wilhelm Wundt envisioned.

Acknowledgments

We thank Alex Gillespie, David Nussbaum, Richard Shweder, Brady Wagoner, and Martha Van Haitsma for their insightful comments on an earlier draft of the manuscript. The manuscript was greatly improved by sage comments from the Associate Editor, Brad Bushman; by a nonanonymous review by Paul Rozin; and by the hard work of three anonymous reviewers.

Footnotes

Declaration of Conflicting Interests: The author(s) declared that there were no conflicts of interest with respect to the authorship or the publication of this article.

Funding: S. A. Power was supported by the Lemelson/Society for Psychological Anthropology Pre-Dissertation Award, made possible by a generous donation from The Robert Lemelson Foundation.

References

Adorno T. W., Frenkel-Brunswik E., Levinson D. J., Sanford R. N. (1950). The authoritarian personality. New York, NY: Harper. [Google Scholar]

Arnett J. (2008). The neglected 95%: Why American psychology needs to become less American. American Psychologist, 63, 602–614. [PubMed] [Google Scholar]

Asch S. E. (1987). Social psychology. New York, NY: Oxford University Press. (Original work published 1952) [Google Scholar]

Bruner J. S. (1990). Acts of meaning. Cambridge, MA: Harvard University Press. [Google Scholar]

Bryman A. (2006). Integrating quantitative and qualitative research: How is it done? Qualitative Research, 6, 97–113. [Google Scholar]

Campbell D. T., Fiske D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105. [PubMed] [Google Scholar]

Charmaz K., McMullen L. M. (2011). Five ways of doing qualitative analysis: Phenomenological psychology, grounded theory, discourse analysis, narrative research, and intuitive inquiry. New York, NY: Guilford Press. [Google Scholar]

Christie R., Jahoda M. (Eds.). (1954). Studies on the scope and method of the authoritarian personality. Glencoe, IL: Free Press. [Google Scholar]

Cole M. (1996). Cultural psychology: A once and future discipline. Cambridge, MA: Harvard University Press. [Google Scholar]

Denzin N. K., Lincoln Y. S. (2000). Introduction: The discipline and practice of qualitative research. In Denzin N. K., Lincoln Y. S. (Eds.), Handbook of qualitative research (2nd ed., pp. 1–29). Thousand Oaks, CA: SAGE. [Google Scholar]

Ellis B., Stam H. (2015). Crisis? What crisis? Cross-cultural psychology’s appropriation of cultural psychology. Culture & Psychology, 21, 293–317. [Google Scholar]

Festinger L. (1964). Conflict, decision, and dissonance. Stanford, CA: Stanford University Press. [Google Scholar]

Festinger L., Schachter S., Back K. (1950). Social pressures in informal groups. New York, NY: Harper. [Google Scholar]

Flexner A. (1939, October). The usefulness of useless knowledge. Harper’s, (179), 544–552. [Google Scholar]

Gay L. R., Airasian P. (2000). Educational research: Competencies for analysis and experience (6th ed.). Upper Saddle River, NJ: Prentice Hall. [Google Scholar]

Geertz C. (1973). Interpretation of cultures. New York, NY: Basic Books. [Google Scholar]

Gelo O., Braakmann D., Benetka G. (2008). Quantitative and qualitative research: Beyond the debate. Integrative Psychological & Behavioral Science, 42, 266–290. [PubMed] [Google Scholar]

Gergen K. J., Josselson R., Freeman M. (2015). The promises of qualitative inquiry. American Psychologist, 70, 1–9. [PubMed] [Google Scholar]

Gillespie A., Cornish F. (2010). Intersubjectivity: Towards a dialogical analysis. Journal for the Theory of Social Behaviour, 40, 19–46. [Google Scholar]

Gillespie A., Reader T. (2016). The Healthcare Complaints Analysis Tool: Developmental and reliability testing of a method for service monitoring and organizational learning. BMJ Quality & Safety, 25, 937–946. [PMC free article] [PubMed] [Google Scholar]

Giorgi A. (1999). A phenomenological perspective on some phenomenographic results on learning. Journal of Phenomenological Psychology, 30, 68–93. [Google Scholar]

Gray K., Young L., Waytz A. (2012). Mind perception is the essence of morality. Psychological Inquiry, 23, 101–124. [PMC free article] [PubMed] [Google Scholar]

Hammack P. L., Pilecki A. (2014). Methodological approaches in political psychology: Discourse and narrative. In Capelos T., Dekker H., Kinnvall C., Nesbitt-Larkin P. (Eds.), Palgrave handbook of global political psychology (pp. 79–98). New York, NY: Palgrave Macmillan. [Google Scholar]

Harris S. (2011). The moral landscape: How science can determine human values. New York, NY: Simon & Schuster. [Google Scholar]

Henrich J., Heine S., Norenzayan A. (2010). The weirdest people in the world? Behavioral & Brain Sciences, 33, 61–83. [PubMed] [Google Scholar]

Henwood K. L., Pidgeon N. F. (1992). Qualitative research and psychological theorizing. British Journal of Psychology, 83, 97–111. [PubMed] [Google Scholar]

Johnson R. B., Christensen L. B. (2000). Educational research: Quantitative and qualitative approaches. Boston, MA: Allyn & Bacon. [Google Scholar]

Johnson R. B., Onwuegbuzie A. J., Turner L. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1, 112–133. [Google Scholar]

Laverty S. M. (2003). Hermeneutic phenomenology and phenomenology: A comparison of historical and methodological considerations. International Journal of Qualitative Methods, 2(3), 21–35. [Google Scholar]

Markus H. R., Kitayama S. (1991). Culture and the self: Implications for cognition, emotion, and motivation. Psychological Review, 98, 224–253. [Google Scholar]

Moghaddam F. M., Harré R. (1995). But is it science? Traditional and alternative approaches to the study of social behavior. World Psychology, 1, 47–78. [Google Scholar]

Moghaddam F. M., Walker B., Harré R. (2003). Cultural distance, levels of abstraction, and the advantages of mixed methods. In Tashakkori A., Teddlie C. (Eds.), Handbook of mixed methods in social and behavioral research (pp. 111–134). Thousand Oaks, CA: SAGE. [Google Scholar]

Nisbett R. E., Cohen D. (1996). Culture of honor: The psychology of violence in the South. Boulder, CO: Westview Press. [Google Scholar]

Oishi S., Graham J. (2010). Social ecology lost and found in psychological science. Perspectives on Psychological Science, 5, 356–377. [PubMed] [Google Scholar]

Onwuegbuzie A. J., Leech N. L. (2005). On becoming a pragmatic researcher: The importance of combining quantitative and qualitative research methodologies. International Journal of Social Research Methodology, 8, 375–387. [Google Scholar]

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349, 944–951. [Google Scholar]

Parker I. (1992). Discourse: Social psychology and modernity. In Doherty J., Graham E., Malek M. (Eds.), Postmodernism and the social sciences (pp. 80–94). Hong Kong: Macmillan. [Google Scholar]

Plantinga A. (2000). Pluralism: A defense of religious exclusivism. In Quinn P. L., Meeker K. (Eds.), The philosophical challenge of religious diversity (pp. 172–192). New York, NY: Oxford University Press. [Google Scholar]

Potter J., Wetherell M. (1987). Discourse and social psychology: Beyond attitudes and behaviour. London, England: SAGE. [Google Scholar]

Power S. A. (2011). On social psychology and conflict. Psychology & Society, 4(1), 1–6. [Google Scholar]

Power S. A. (2016). A violent past but a peaceful present: The cultural psychology of an Irish recession. Peace and Conflict: Journal of Peace Psychology, 22, 60–66. [Google Scholar]

Power S. A. (2017). From the elites to the streets: The psychology of democracy and economic inequality (Unpublished doctoral dissertation). University of Chicago, Illinois. [Google Scholar]

Power S. A. (in press). Actual democracy and a united Europe of states: A case study of austerity and protest in the republic of Ireland. In Wagoner B., Brescó I. (Eds.), The road to actualized democracy: A psychological perspective. Charlotte, NC: Information Age Publishing. [Google Scholar]

Qadafi A. K. (2017). “Because I’ve seen it before”: An exploration of the relationship between identity and the epistemic beliefs and practices of Black male adolescents (Unpublished doctoral dissertation). University of Chicago, Illinois. [Google Scholar]

Rozin P. (2001). Social psychology and science: Some lessons from Solomon Asch. Personality and Social Psychology Review, 5, 2–14. [Google Scholar]

Rozin P. (2009). What kind of empirical research should we publish, fund and reward? A different perspective. Perspectives on Psychological Science, 4, 435–439. [PubMed] [Google Scholar]

Schiff B. (2017). Explorations in narrative psychology. A new narrative for psychology. Oxford, England: Oxford University Press. [Google Scholar]

Seligman M. E., Railton P., Baumeister R. F., Sripada C. (2013). Navigating into the future or driven by the past. Perspectives on Psychological Science, 8, 119–141. [PubMed] [Google Scholar]

Sherif M., Harvey O. J., White J., Hood W., Sherif C. (1961). Intergroup conflict and cooperation: The robbers cave experiment. Norman: University of Oklahoma Book Exchange. [Google Scholar]

Shweder R. A. (1991). Thinking through cultures: Expeditions in cultural psychology. Cambridge, MA: Harvard University Press. [Google Scholar]

Shweder R. A. (1996). Quanta and qualia: What is the “object” of ethnographic method? In Jessor R., Colby A., Shweder R. A. (Eds.), Ethnography and human development: Context and meaning in social inquiry (pp. 175–182). Chicago, IL: University of Chicago Press. [Google Scholar]

Shweder R. A. (1997). The surprise of ethnography. Ethos, 25, 152–163. [Google Scholar]

Shweder R. A. (2003). Why do men barbecue? Recipes for cultural psychology. Cambridge, MA: Harvard University Press. [Google Scholar]

Shweder R. A., Power S. A. (2013). Robust cultural pluralism: An interview with professor Richard A. Shweder. Europe’s Journal of Psychology, 9, 671–686. [Google Scholar]

Smedslund J. (2009). The mismatch between current research methods and the nature of psychological phenomena: What researchers must learn from practitioners. Theory & Psychology, 19, 778–794. [Google Scholar]

Smith G. T., Spillane N. S., Annus A. M. (2006). Implications of an emerging integration of universal and culturally specific psychologies. Perspectives on Psychological Science, 1, 211–233. [PubMed] [Google Scholar]

Tashakkori A., Teddlie C. (Eds.). (2010). SAGE handbook of mixed methods in social & behavioral research (2nd ed.). Thousand Oaks, CA: SAGE. [Google Scholar]

Teddlie C., Tashakkori A. (2003). Major issues and controversies in the use of mixed methods in the social and behavioral sciences. In Tashakkori A., Teddlie C. (Eds.), SAGE handbook of mixed methods in social & behavioral research (pp. 3–50; 1st ed.). Thousand Oaks, CA: SAGE. [Google Scholar]

Tennant J. C. (2015). Harm and non-harm moral judgments and moral justifications in two American religious groups (Unpublished doctoral dissertation). University of Chicago, Illinois. [Google Scholar]

Thaler R. H., Sunstein C. R. (2008). Nudge: Improving decisions about health, wealth, and happiness. New Haven, CT: Yale University Press. [Google Scholar]

Trinidas H. C. (2007). Culture and psychology: A history of the study of their relationship. In Kitayama S., Cohen D. (Eds.), Handbook of cultural psychology (pp. 59–76). New York, NY: Guilford Press. [Google Scholar]

Weinfurt K. P., Moghaddam F. M. (2001). Culture and social distance: A case study of methodological cautions. The Journal of Social Psychology, 141, 101–110. [PubMed] [Google Scholar]

Wierzbicka A. (1993). A conceptual basis for cultural psychology. Ethos, 21, 205–231. [Google Scholar]

Wundt W. (1897). Outlines of psychology (Judd C. H. Trans.). St. Clair Shores, MI: Scholarly Press. [Google Scholar]

Yoshikawa H., Weisner T. S., Kalil A., Way N. (2008). Mixing qualitative and quantitative research in developmental science: Uses and methodological choices. Developmental Psychology, 44, 344. [PubMed] [Google Scholar]