Guide to ReportsNEUROPSYCHOLOGICAL ASSESSMENT LABPAUL JONES, ED.D.
UNIVERSITY OF NEVADA, LAS VEGAS
____________________________________________________________________
INFLUENCE OF PRACTICE CONDITIONS:
Assessment Lab Report 2-2
December/1998
A study conducted in this lab in the spring of 1998 (Lab Report 1-1: Practice Effect) confirmed the importance of an orientation session when the ANAM/TWB cognitive processing scales are used for neuropsychological assessment. In that study the practice and trial sessions were conducted on two occasions with a minimum of two days between the two sessions. Consistent with other data, accuracy scores were typically above 90% on both practice and actual trial sessions; the difference between practice and trial sessions on accuracy was negligible. However, differences in the cognitive efficiency (correct responses/minute) scores were clearly evident.
Statistically significant differences in the cognitive efficiency scores between practice and trial sessions were found on all scales except Matching to Sample. Using criteria suggested by Gastaldo, Reeves, Levinson, and Wenger (1997) and Kay (1995), clinical significance was also apparent. On scales other than Matching to Sample, a significant number of the practice test efficiency scores would have been identified as clinically impaired performance. The percentage of such scores ranged from 38% on the math and spatial tasks to 46% on procedural reaction time.
In the study above, participants self-selected the time period between practice and actual test trial sessions. The time between practice and trial sessions ranged from two to 67 days with a median of one week. Actual test performance did not appear to be a function of the length of time delay between practice and test sessions, and that outcome was suggested to provide tentative support for conducting both practice and actual test administration in a single session in clinical application. The present study provides further investigation of the influence of practice conditions.
Method
Participants
Two participant samples provided data for this study. Subjects for the time-delay condition participated in the series one studies in the spring of 1998. Complete practice and test trial data on all scales were available for 52 subjects in that study. Subjects for the single-occasion condition participated in the series two studies in the fall of 1998. Complete data were available for 71 subjects. Both samples were drawn from an accessible population of undergraduate teacher education students.
Table 1 below compares demographic data for the 52 subject time-delay sample in the spring of 1998 and the 71 single-occasion subjects in the fall 1998 study. A perusal of Table 1 indicates that the two samples are comparable in gender, age, and previous grade point average. Using percentages from the spring study for expected frequencies, there was no significant difference in gender (chi-square (1, N = 71) = .05, p > .05). Age differences were not statistically significant (chi-square (2, N = 71) = 3.22, p > .05). There was also no significant difference between the groups in reported grade point average (chi-square (2, N = 71) = .79, p > .05).
____________________________________________________________________| Group 1 Spring 98 n = 52 | Group 2 Fall 98 n = 71 | ||||
|---|---|---|---|---|---|
| gender | n (%) | n (%) | |||
| female | 36 (69) | 50 (70) | |||
| male | 16 (31) | 21 (30) | |||
| age | |||||
| 18-25 | 38 (73) | 47 (66) | |||
| 26-35 | 9 (17) | 18 (25) | |||
| 36 & above | 5 (10) | 6 (8) | |||
| grade point average | |||||
| 2.9 & below | 22 (42) | 27 (38) | |||
| 3.0 - 3.4 | 18 (35) | 28 (39) | |||
| 3.5 & above | 12 (23) | 16 (23) |
Instrumentation
Instruments used in this study were drawn from the Tester's Workbench, a series of cognitive test instruments developed by the Office of Military Performance Assessment Technology. Five of these scales were used in both the series one studies (spring 1998) and series two studies (fall 1998):
Procedural Reaction Time- Standard Version
Procedural Reaction Time- Degraded Stimuli
Mathematical Processing
Spatial Processing- Simultaneous
Matching To Sample
The number of stimuli (items) used in the actual test trials was the same in both spring and fall studies. Table 2 below identifies the number of items used in the practice sessions in the two conditions for each of the five scales.
____________________________________________________________________| time-delay practice items | single-occasion practice items | ||||||
|---|---|---|---|---|---|---|---|
| Procedural Reaction Time- Standard Version | 17 | 6 | |||||
| Procedural Reaction Time- Degraded Stimuli | 17 | 6 | |||||
| Mathematical Processing | 10 | 5 | |||||
| Spatial Processing- Simultaneous | 10 | 5 | |||||
| Matching To Sample | 10 | 5 |
Procedures
In both conditions, the stimuli (items) in the practice trial were comparable, but fewer in number, than those used in the actual test with feedback (correct or incorrect) provided after each stimulus in the practice trial. The two study conditions differed in: 1) the length of time between practice and actual test trial sessions; 2) the number of items included in the practice sessions; and 3) the contiguity of practice and test trials.
In the time-delay condition there was a minimum of two days between practice and test trials with typically a span of a week between them. In the single-occasion condition, the practice test immediately preceded the test trial for each scale. Fewer practice items (1/2 or less) were provided in the single-occasion condition.
Several scores are available for each of these cognitive scales, including task accuracy, mean response time for accurate responses (ms), and an efficiency score. The latter is defined as the number of correct responses per minute. Levinson and Reeves (1997) and Schlegel, Gilliland, and Crabtree (1992) have noted that use of the accuracy scales on the these measures is problematic because the nature of the tasks typically results in a low ceiling and thus little differentiation among the scores. Levinson and Reeves (1997) suggested that the cognitive efficiency score which is more sensitive to individual differences is the more useful score, particularly with nonclinical samples. The cognitive efficiency score was used to compare the performance for the two practice conditions.
Results
Table 3 below provides efficiency (mean and standard deviation) scores for the time-delay (spring 1998) and single-occasion (fall 1998) test conditions on the five cognitive scales used in both studies. Differences in cognitive efficiency between the time-delay and single-case conditions were not statistically significant on any of these scales.
____________________________________________________________________| time-delay n=52 | single-occasion n=71 | ||||||
|---|---|---|---|---|---|---|---|
| mean (s.d.) | mean (s.d.) | t | p | ||||
| react-1 | 101.2 (13.50) | 105.0 (11.93) | 1.65 | .102 | |||
| react-2 | 95.9 (19.16) | 100.9 (17.20) | 1.51 | .133 | |||
| math | 24.5 (6.61) | 23.8 (5.83) | .61 | .546 | |||
| spatial | 28.5 (6.77) | 30.8 (7.86) | 1.68 | .096 | |||
| match | 30.5 (10.38) | 33.5 (9.11) | 1.67 | .098 |
____________________________________________________________________react-1: Procedural Reaction Time- Standard Version
react-2: Procedural Reaction Time- Degraded Stimuli
math: Mathematical Processing
spatial: Spatial Processing- Simultaneous
match: Matching To Sample
Summary
The results of this study provide additional support for combining practice and actual test trials in a single session. The two samples appear comparable in gender, age, and general ability (as measured by self-reported grade-point-average). Differences were not statistically significant on any of the five scales.
References
Gastaldo, E., Reeves, D., Levinson, D., & Wenger, C.B. (1997). ANAM normative data: USMC-1995 hyponatremia outbreak studies. San Diego, CA: National Cognitive Recovery Foundation.
Levinson, D.M., & Reeves, D.L. (1997). Monitoring recovery from traumatic brain injury using Automated Neuropsychological Assessment Metrics. Archives of Clinical Neuropsychology, 12, 155-166.
Kay, G. (1995). Cogscreen: Aeromedical edition: professional manual. Odessa, FL: Psychological Assessment Resources, Inc.
Schlegel, R.E., Gilliland, K., & Crabtree, M.S. (1992). Development of the UTC-PAB Normative Database: Technical Report. Springfield, VA: National Technical Information Service.
________________________________________________________________________________________________________________________________________