SCID: Single-Case Intervention Design; SCD: Single-Case Design; RCT: Randomized Controlled Trial

A few years ago we presented a class of intervention designs that is relatively unfamiliar to health-sciences researchers [1], namely, single-case intervention designs (SCIDs) out of the applied behavior analysis tradition (see, for example, the Journal of Applied Behavior Analysis). The main thrust of that article was to introduce health-sciences researchers to the virtually unlimited possibilities for conducting intervention studies with only scarce resources (specifically, with small numbers of participants, restricted amounts of time, and financial constraints). In that article, the rationale underlying SCIDs was presented, along with the features and procedural characteristics of basic single-case intervention designs. The uninitiated health-science researcher is referred to that article for an introductory presentation, or “primer”, on the logic, design, conduct, analysis, and associated inferences associated with SCIDs. In the present Comment we expand on the argument that the “scientific credibility” of SCID research can been considerably enhanced through the incorporation of various randomization and replication procedures, along with attention to concerning operational issues. SCIDs, not to be confused with N-of-1 trials designs in the medical-research literature (e.g., [2]), are time-series designs that are characterized by a small number of participants and a large number of measurement/observation occasions [3,4]. Briefly: (1) N-of-1 trials designs are not as methodologically rigorous with respect to controlling unwanted (or experimentally “confounding”) sources of variation – and therefore not as scientifically credible – as are many SCIDs. In addition: (2) with the vast number of SCID design variations available, there are more design-and-analysis alternatives to address directly a researcher’s questions and purposes, as well as to allow for the inclusion of one or more participants; and (3) SCIDs are associated with compendia of desired Standards and Guidelines that researchers must adhere to for their study to be deemed “acceptable” by the scientific community (e.g., [5]). For additional differences between the two methodological approaches, see [1].

As with conventional large-scale randomized controlled trials (RCTs) intervention research, in SCIDs various forms of randomization can turn an observational study into a scientifically credible experimental study [1]. Two common types of randomization in two-treatment (e.g., drug vs. placebo or Drug 1 vs. Drug 2) conventional intervention research involve those that control for or remove unwanted sources of treatment contamination variation. In single-case one-sample alternating treatment or crossover designs a researcher randomly determines, on a participant-by-participant basis, the order in which the two treatments are administered. In two independent-samples designs, a researcher randomly assigns participants to the two different treatments. In addition, these two forms of randomization (phase-order randomization and between-intervention case randomization) can be combined in single-case two independent-samples replicated B (Treatment 1 phase) vs. C (Treatment 2 phase) or crossover designs. Each of these randomization forms serves to prevent a researcher from falsely claiming to have produced an intervention effect either when there is none or when two treatments are essentially equivalent in their efficacy.

As a hypothetical pharmacology example of the latter two independent-samples application, consider the recent speculation that a simple COVID vaccination might prove to be a potential cancer antidote and a likely more economical alternative to a bona fide cancer vaccine [6]. One could design a preliminary SCID study in which a handful of participants are randomly assigned to one of two conditions, a cancer vaccine condition and a COVID vaccine condition. After a series of several cancer-indicating baseline blood tests taken over a period of weeks or months in the two experimental conditions, participants are then administered their respective vaccines. Continuing intervention-series blood tests would be taken repeatedly over several weeks or months to track those cancer indicators and to compare them in the two experimental conditions.

As another related example, suppose that a pharmacology researcher wishes to conduct a preliminary investigation of whether a steroid pill such as dexamethasone taken along with a cancer-treatment vaccine is a more effective regimen than the cancer-treatment vaccine alone. Again, in a two independent-samples SCID, patients would be randomly assigned to either an A (baseline phase), B (cancer drug phase) condition or an A (baseline phase), B+C (cancer drug + steroid phase) condition, with cancer-relevant outcomes repeatedly compared over time. For a recent actual health-sciences SCID example, see [7].

Health-sciences researchers can implement two additional forms of randomization to increase the validity of their inferences concerning intervention efficacy: One form is within-intervention case randomization, where for certain designs (e.g., multiple-baseline designs and multiple-probe designs) the order within the design structure in which participants are treated and tested is randomly determined. A second form of randomization is intervention start-point randomization, where the specific time period or session in which, for each participant, a series of “baseline” (or no-treatment) observations is systematically transitioned to a randomly determined series of intervention (or treatment) observations. This latter form of randomization, which contrasts with a traditional “response-guided” intervention process, serves to remove all potential researcher bias associated with subjective decisions about when to transition from administering a baseline series to administering an intervention series (see, for example, [8–11]).

Replication is the second hallmark component of the scientific credibility equation in SCID research. Because SCID research is typically based on small sample sizes, generalization to other participants and contexts may be limited. Within a single-study, replication of procedures and intervention effects can be provided within a single participant (e.g., in ABAB (i.e., two alternating baseline and intervention phases) and alternating treatment designs, (i.e., two-or-more repeated alternating conditions phases) across participants in one-sample multiple-participant designs, and in two independent-samples designs. Replication across studies can be provided through implementation of the same or similar design and procedures with new participants, thereby strengthening the generalizability of a study’s findings.

In SCID research, various forms of randomization are generally necessary to preserve the integrity of the design and statistical analysis (“internal validity” and “statistical conclusion validity” foci), whereas within- and across-study replication is necessary to permit generalization of the results to other contexts and participant populations (an “external validity” focus) – see, for example, [12]. However, neither randomization nor replication per se are sufficient to allow for confidence in the study’s operations and conclusions. For design-and-analysis integrity and results generalization to be validly claimed, researchers must pay additional attention to, and have evidence to rule out, specific “operational issues” that could have arisen during the study’s conduct. As was originally noted by [13]:

After the data have been collected, the researcher can examine the operational issues of the study as actually conducted (including, for example, attention to unwanted baseline trend, outlying observations, missing data, participant attrition, etc.). Operational issues encompass unplanned-for problems and concerns that could occur during the conduct of the SCD, which, if unattended to, serve to undermine the study’s credibility. These issues are akin to the host of confounding procedural, investigator, and participant “effects” that have been enumerated in the conventional group intervention-research literature and that must be taken into account to preserve the study’s internal validity (e.g., [12,14,15].

Researcher-administered and analyzed surveys, questionnaires, checklists, etc., along with patient-produced mental and physical health protocols are particularly susceptible to unintended operational effects – or worse, biased outcomes. Even a carefully conducted SCID experiment does not rule out the possibility that various types of researcher and patient bias can become internal validity threats. At a minimum, patients should not be aware of the specific test responses that are expected or desirable, thereby negating the “demand characteristics” associated with the intervention. If possible, testers/data collectors should not be aware of (i.e., they should be “blind” to) whether the data being collected are part of the baseline series or the intervention series, and in two-treatment studies, they should be “blind” to the patient’s experimental condition.

To remove the parenthetical “Almost” from this article’s title, at least three additional critical actions are necessary. First, the just-mentioned operational issues concern must be attended to and satisfactorily accounted for. Second, adherence to established SCID Standards and Guidelines must be carefully followed (namely, those provided by [5,13,16,17], and others). For example, an adequate number of baseline and intervention outcome measures must be collected for each participant to constitute an acceptable SCID study. Third, the outcome measures taken must possess a high degree of “construct validity” (i.e., that the outcome measures accurately reflect the participants’ underlying traits, behaviors, or processes of research interest). With these actions stated and successfully achieved, a strong case can be made that Randomization + Replication = Respectability, where Respectability equates to scientific credibility.

In addition, the analysis of SCID data is a specialized topic that should not go unmentioned. Unfortunately, data-analysis options for these designs are not well established in the health sciences research community. As was noted in the opening of this Commentary, SCID data are of a special kind in that they are collected multiple times over the course of a time-series experiment. This feature of data collection requires understanding that multiply measured SCID outcomes within a single participant are invariably “autocorrelated”, rather than “independent” as they are in the singly measured within-participant data of conventional randomized controlled trials studies. The autocorrelated nature of the data in turn means that conventional statistical tests (e.g., traditional t and chi-squared tests, analyses of variance, simple regression analyses, etc.) are statistically inappropriate and invalid when applied to the types of experiments outlined here and in our earlier article (see, for example, [18]). Specifically, when conventional statistical procedures are applied to the analysis of the typically autocorrelated data of SCIDs, the resulting statistical conclusions are too liberal, in the sense of their declaring treatment effects “statistically significant” when in fact they are not or declaring effect sizes larger than they actually are. Our point here is that application of conventional statistical tests will often lead to misleading or erroneous conclusions (namely, about the magnitude of statistically based p-values and effect sizes, along with decisions about the test’s statistical significance), which compromise the “statistical conclusion validity” of the experiment.

What are some appropriate data-analysis options for SCID studies? Although it is beyond the scope of this Commentary to provide the specifics, what can be said here is that different forms of data analysis include visual analysis (e.g., [19,20]), effect-size estimation (e.g., [21,22], directed acyclic graphs (e.g., [23]), descriptive statistics (numerous studies), randomization tests (e.g., [24]), multilevel modeling (e.g., [25,26]), Bayesian analysis [27,28], and artificial neural networks (e.g., [29]). These various single-case data-analysis methods have different purposes and possess different strengths and weaknesses. The statistical power of different valid quantitative procedures to detect various effect types over time (e.g., strong, weak, immediate, delayed, abrupt, gradual, nonlinear, deteriorating) should also be considered when a researcher is deciding on which statistical test to adopt [1]. Readers are referred to textbooks that feature different procedures related to the analysis and interpretation of SCID data (e.g., [3,4]).

We conclude this Commentary by providing a sample of recent relevant single-case intervention articles [30–37]. Although such techniques might initially seem imposing and challenging to implement in applied research settings, they are now commonly adopted in numerous cognitive, behavioral, mental health and other health-related domains, the latter of which are illustrated through empirical studies in the accompanying References (e.g., [7,8,28,31]). Specifically, as can be appreciated from an inspection of the content in those References, dozens of scientifically credible SCID studies have been conducted with children experiencing cognitive or behavioral difficulties and who are taking various medications. Preliminary SCID investigations of different drug efficacies, dosage levels, or combinations could well lead to more definitive large-scale RCTs. Researchers in the pharmacological-sciences field are well-advised to take note of SCID methodology, with the goal of implementing the tactics therein to expand the design, conduct, and analysis of their own intervention-research arsenals.