Guide to ReportsNEUROPSYCHOLOGICAL ASSESSMENT LABPAUL JONES, ED.D.
UNIVERSITY OF NEVADA, LAS VEGAS
____________________________________________________________________
PRACTICE EFFECT:
Assessment Lab Report 1-1
May/1998
In the procedure used for the series one studies, each of the cognitive test instruments was administered on two occasions with a minimum of two days between each test session. Results from the two test periods are compared here for analysis of the extent of change between practice and actual test sessions.
In a large-scale study involving multiple trials with comparable instruments, Schlegel, Gilliland, and Crabtree (1992) reported a significant training effect on the ANAM/TWB cognitive processing scales. They noted that the greatest change is between the first and second trials and that a number of trials may be necessary to obtain a stable level of performance.
Gastaldo, Reeves, Levinson, and Wenger (1997) used ANAM V3.11 with a group of USMC recruits with a retest one month later. ANAM V3.11 Math Processing, Spatial Processing, and Matching to Sample tests are comparable to the instruments used in the series one studies. They observed no practice effect between the two test periods, noting a similar outcome in a study of college graduates tested and retested two to three months later (Levinson & Reeves, 1996).
Results: Practice Effect
Tables 1 and 2 below provide accuracy and efficiency (mean and standard deviation) scores for the two test occasions in the series one studies. Preceding this analysis, the data files were assembled in a spreadsheet and inspected to identify responses indicating that the participant did not understand the instructions prior to beginning the practice session (e.g. significant number of response lapses and/or premature responses). The screening resulted in an unequal number of participants in the analysis of some scales. Such screening is typical for analysis of ANAM/TWB data (Gastaldo, Reeves, Levinson, & Wegner, 1997; Schlegel, Gilliland, & Crabtree, 1992) and was appropriate in this study in attempt to insure that observed changes in performance could be attributed to a practice effect.
Table 1 indicates that there was only one statistically significant difference in accuracy scores between the practice and actual test trials. This is consistent with other findings and is anticipated from the low ceiling of the accuracy scores. The statistically significant difference on the degraded stimuli condition of the Procedural Reaction would not appear to represent a clinically significant change in performance (95.4% to 97.8%.
In contrast, Table 2 indicates an overall significant degree of improvement in the cognitive efficiency (correct responses/minute) scale between practice and actual test trials. Statistically significant differences were evident in all cognitive tests except Matching to Sample.
____________________________________________________________________| practice | test | ||||||
|---|---|---|---|---|---|---|---|
| mean (s.d.) | mean (s.d.) | t | p | ||||
| react-1 (n=52) | 95.0 (12.13) | 97.9 (3.31) | 1.70 | .095 | |||
| react-2 (n=54) | 95.4 (6.36) | 97.8 (3.55) | 2.36 | .022 | |||
| mem-4 (n=54) | 95.6 (11.10) | 96.3 (5.37) | .43 | .670 | |||
| mem-6 (n=56) | 97.3 (5.56) | 97.5 (4.28) | .17 | .865 | |||
| math (n=55) | 90.7 (11.03) | 93.6 (5.94) | 1.82 | .074 | |||
| spatial (n=56) | 95.0 (11.28) | 95.8 (7.91) | .62 | .538 | |||
| match (n=56) | 90.7 (10.93) | 90.9 (9.91) | .09 | .93 |
____________________________________________________________________react-1: Procedural Reaction Time- Standard Version
react-2: Procedural Reaction Time- Degraded Stimuli
mem-4: Sternberg Memory Search- Four-Letter Set
mem-6: Sternberg Memory Search- Six-Letter Set
math: Mathematical Processing
spatial: Spatial Processing- Simultaneous
match: Matching To Sample
| practice | test | ||||||
|---|---|---|---|---|---|---|---|
| mean (s.d.) | mean (s.d.) | t | p | ||||
| react-1 (n=52) | 83.4 (26.12) | 101.2 (13.50) | 6.00 | .000 | |||
| react-2 (n=54) | 70.0 (25.31) | 94.2 (20.79) | 7.81 | .000 | |||
| mem-4 (n=54) | 73.1 (18.97) | 80.6 (17.45) | 2.71 | .009 | |||
| mem-6 (n=56) | 74.1 (20.12) | 80.7 (17.70) | 2.62 | .011 | |||
| math (n=55) | 21.6 (6.97) | 24.4 (6.45) | 3.42 | .001 | |||
| spatial (n=56) | 22.3 (5.67) | 28.3 (6.74) | 9.68 | .000 | |||
| match (n=56) | 28.2 (8.63) | 30.0 (10.22) | 1.56 | .123 |
The series one studies procedure allowed participants to self-select the time period between practice and actual test trial sessions with a minimum of two days between the two sessions. The time delay in this dataset ranged from two days to 67 days with a mean of 15.1 and standard deviation of 16.7. The median number of days between the two sessions was seven.
Table 3 below displays the product moment correlations for the number of days between sessions and the absolute difference in cognitive efficiency scores between practice and trial sessions. The only statistically significant relationship was with the 6-letter set of the Sternberg Memory Search.
For further exploration of this relationship, the number of days between trials was categorized into three levels: less than one week (n=23), a week or more but less than a month (n=19) and a month or more (n=14). The absolute difference between practice and trial sessions among the three groups was not significant, F(2,53) = 3.04, p > .05.
____________________________________________________________________| react-1 | react-2 | mem-4 | mem-6 | math | spatial | match | |
|---|---|---|---|---|---|---|---|
| -.11 | -.10 | .18 | .28* | -.09 | -.12 | .08 |
____________________________________________________________________* significant at .05 level
Summary
Consistent with other data, the difference between practice and trial sessions was negligible on the overall accuracy score with most participants achieving 90%+ in both trials. The cognitive efficiency scale which includes both accuracy and speed, however, was significantly influenced by the practice session.
In comparison to other published studies, it should be emphasized that the practice and trial sessions were not conceptualized to provide test/retest data. Each scale was significantly shorter in the practice session. The practice session in this study, in effect, served as an orientation session which would appear the most likely explanation for the marked overall improvement in efficiency between the two sessions.
The results of this study clearly support the importance of a practice session when the efficiency score is used in clinical applications of these cognitive scales. The lack of significant relationship in length of time delay between practice and actual test administration provides tentative support for providing both practice and actual test administration in a single session. Additional study is needed to investigate whether such procedure would introduce a significant fatigue factor and to further investigate the impact of the number of sessions needed to reach a stable efficiency score.
References
Gastaldo, E., Reeves, D., Levinson, D., & Wenger, C.B. (1997). ANAM normative data: USMC-1995 hyponatremia outbreak studies. San Diego, CA: National Cognitive Recovery Foundation.
Levinson, D.M., & Reeves, D.L. (1996). ANAM V3.11 normative performance of college graduates. (Report No. NCRF-TR-96-02). San Diego, CA: National Cognitive Recovery Foundation.
Schlegel, R.E., Gilliland, K., & Crabtree, M.S. (1992). Development of the UTC-PAB Normative Database: Technical Report. Springfield, VA: National Technical Information Service.
________________________________________________________________________________________________________________________________________