MEASUREMENT
General PrinciplesUsing Existing DataUsing Self-Anchored Rating ScalesUsing Standard InstrumentsUsing Direct Behavior ObservationsHow Many Measurements and How OftenObtaining the BaselineGeneral Principles
It can easily be argued that the quality of scientific research rests on the precision of the measurement. This is certainly no less true in single-case research designs. In fact, the reliance on repeated measures in these designs makes this a particularly crucial feature.
As in all applications in school psychology and counseling, the tools you use for measurement in single-case research should meet typical criteria for adequate reliability and validity. Feasibility also warrants special consideration in single-case applications in the schools. The frequency of the measurement process requires attention to insure that the selected tool is feasible in view of not interrupting the instructional process. The tool(s) you select must also be sensitive to even small variations in the condition being studied. In single-case applications, you also need minimal reactivity in the tool(s) you select.
Reactivity is the condition in which the measurement process itself results in change in the behavior. For example, a researcher might ask a depressed person to keep a log of the number of positive thoughts during a 24-hour period. It would not be especially surprising if just the process of keeping the log resulted in a decrease in the feelings of depression. While this would be fine for the person with the symptoms, the reactivity of this log would confuse interpretations made about other treatment. Reactivity issues are magnified in single-case designs because the measurement process is repeated so frequently.
Using Existing Data
For many of the applications in school psychology and counseling, there may be existing data to meet the measurement need. For example, if you are recommending or providing treatment to reduce the extent of tardiness, there often will be data to provide a baseline, and the data for continuing measurement will be routinely gathered without special action on your part.
In like manner, if your treatment recommendation is in reference to a specific academic problem, exams already being used in the classroom may provide the data needed for the research study. This has an additional advantage in that it facilitates collaborative involvement with the classroom teacher.
Using Self-Anchored Rating Scales
In these scales a rating is made, typically by the student, on a dimension operationally defined for a specific application. Sklare (1997) describes the integration of a self-anchored scale in school counseling applications. The focus of the scale is the problem which brought the student to the counselor. The problem is translated into a goal. The student provides a self-rating of present status toward the goal on a scale of 1 to 10. This scale is then used to identify progress in future sessions. The source cited in the references section provides detail about how this scaling enhances the effectiveness of the school counseling process.
The process described above could be generalized to a number of single-case applications in school psychology and counseling. For example, scales of progress could be completed by the student and/or by the referring teacher. Baseline data for single-case research applications could be obtained by asking the referring teacher to provide such ratings for a period of time before the treatment intervention is initiated. Self-anchored scales are problematic in group research designs because of the unknown differences in response set (e.g. your "good" might be my "great"). The use of such scales is more easily justified in single-case designs because comparisons are made within rather than between students.
Using Standard Instruments
There are advantages in using standard published instruments for the measurements in single-case research. Reliability and validity data will already be available. You can usually thus be more confident in the quality of the measurement data.
Caution, though, is needed in selecting the tool. Many instruments typically used in school psychology and counseling are intended for measurement at the trait level and thus would not be sensitive to the incremental changes when treatment is introduced. Also, the length of many of our standard instruments would preclude use on a repeated measures basis.
With those cautions considered, there are a number of published instruments which are appropriate for use as repeated measures. Examples include the variety of symptom checklists designed to be completed by student, teacher, and/or parent. A useful source of such instruments cited in the references section has been compiled by Corcoran and Fischer (1987).
Using Direct Behavior Observation
Single-case research grew from the tradition of applied behavior analysis in which the focus was on direct observation of behavior without an intermediary test or questionnaire. Use of the single-case research model does not require embracing a "behavior modification" philosophical position, but you may very well have applications in which direct observation will provide the best form of measurement.
Direct observation of behavior is a focus not only in the tradition of behavior analysis. It is a method at the heart of contemporary qualitative research designs. One of the sources cited in the references (Neuman & McCormick, 1995) provides an especially interesting (and perhaps controversial) recommendation for integrating single-case and qualitative inquiry in the school setting.
When using direct observation as the measurement tool, you will want to be especially cautious about comparability in the data. The outcome of direct observation is influenced by a number of factors including who is making the observation, when it is being made, and so forth. If more than one observer is being used it is particularly important to be sure that differences in the data are related to differences in the student and not the person doing the observation.
How Many Measurements and How Often
The answer to these questions is, of course, in part determined by the specific nature of your single-case research design. In general terms, the answer to the first question (how many) is "enough to provide a valid assessment of the baseline condition and the possible impact of the treatment". The answer to the second question (how often) is "often enough to insure that you have valid assessment of the baseline condition and the possible impact of the treatment, but not so often that it interferes with obtaininga valid assessmenty".
More specific guidance for the question of how many measures are needed is provided by the statistical tools. The time series procedure, for example, is known to be reasonably effective with at least eight data points for the analysis. Thus, a goal for the baseline and the treatment data would be to have at least eight measurements during each phase.
In deciding how often the measurement should occur, feasibility can be a primary concern. Asking the student, teacher, and/or parent to complete a questionnaire once each day may or may not be an unreasonable request dependent on the particular circumstances. The risk of instrument reactivity is certainly increased as frequency of measurement is increased.
The general guideline here is simply to remember the focus of your inquiry. You need to know the status of the condition before your treatment begins. You need to know that status of the condition after the treatment is introduced. In some cases this will require daily measurement. In others a weekly measurement process will be sufficient. In all cases it is important to remember that we work in the context of a school setting which necessitates sensitivity to any requests which require the time of others.
Circumstances in your particular setting will have the major impact in the repeated measurements required in single-case design. You may or may not be able to obtain the number or frequency desired for an optimal study. This, though, does not prevent you from conducting the study or the analysis. Instead, any features which may limit the findings are addressed in your interpretation of them.
Obtaining the Baseline
This program was written with particular attention to applications of single-case research in school psychology and counseling. There are constraints in "real-life" as opposed to clinical research settings which present some special difficulties, especially in regard to the baseline measurements. For example, when a student is referred to you for assistance with a conduct problem in a classroom, it may be unreasonable (and could be unethical) for you to require a series of baseline measures before you will see the student.
Unless you are able to see the student immediately, it will usually not be unreasonable, though, to request the referring teacher's assistance in gathering at least some baseline information. In fact, the very process of doing this may be of help to both you and the teacher in working with the student.
For example, if the referral request is in regard to a conduct problem, collaborative work with the teacher in identifying when and under what conditions the misbehavior occurs can be of mutual benefit. Such discussion could lead to the gathering of specific information to provide the baseline for a single-case study. With caution required, it also may, in some instances, be possible for a baseline to be retroactively constructed from recall. And, in some cases, the needed baseline data may be already available in student records before a referral is made.
For obvious reasons the "retrospective" creation of a baseline based only on memory recall should be a last resort. A variety of features, including simply bad memory, can distort such data. Building collaborative relationships with referring teachers to assist in obtaining baseline data for single-case research studies could, on the other hand, have benefits which go beyond the gathering of research data.
Single-case designs do not always require the gathering of baseline data. A study using the alternating-treatments design, for example, can be conducted without the baseline phase. And, although technically not a research design, there is still another potentially valuable application of single-case statistical analysis when a baseline is not available. Assume that you begin the measurement process when you begin the treatment and continue until treatment is successfully terminated. These data could, in effect, become a form of "baseline" which could be contrasted with a follow-up period of data gathering to determine if the effect of intervention continued after the treatment ended.
_________________________________________________________________________
continue to statistical analysis overview
measurement review questions
access statistical analysis tools
back to program guide
Jones Home Page