There has been a true explosion of digital cognitive testing platforms, which is not surprising given the enormous benefits of digital cognitive testing and the number of medical conditions that affect cognitive functions such as Alzheimer’s disease, stroke, and traumatic brain injury. In all of these conditions it is important to regularly track disease progression, either to know whether a patient is deteriorating over time or to know whether she is improving, e.g. because of a treatment. Given the importance of regular cognitive testing, it is crucial to know whether a measured change in cognitive functions is due to a change in the patient’s ability rather than to other factors.
A change in a patient’s cognitive functions between two assessments is usually due to some combination of true change and measurement error. There can be various sources of measurement error, including the test-retest reliability of the cognitive test, practice effects, the duration of the retest interval, and clinical factors. In order to know whether an observed change in cognitive functions between two or more assessments is beyond what would be expected by chance, it is necessary to calculate the reliable change index1. Despite its importance for clinical decision-making, the reliable change index is rarely used in clinical practice because of the complexity of formulas to calculate it.
The reliable change index can be calculated in different ways2, and formulas ideally include corrections for factors that can influence cognitive performance, such as the patient’s age and level of education as well as practice effects of a cognitive test and the duration of the retest interval. Confidence intervals can then indicate how likely it is that an observed change is reliable, i.e. due to the patient’s ability.
The figures illustrate two patients, one with a reliable decline (patient A), and one with no change in cognitive function (patient B). The lines indicate the 90% confidence interval of the reliable change index. Patient A changed from a score of 12 on the first visit to a score of 6 on the second visit. The score on the second visit falls outside of the 90% confidence interval and it can therefore be concluded that patient A shows a true decline in cognitive function. Patient B on the other hand changed from a score of 15 on the first visit to a score of 20 on the second visit. The score on the second visit falls within the 90% confidence interval and is therefore due to other factors rather than to a true improvement in cognitive ability.
If you are interested in learning more about reliable change in digital cognitive testing, have a look at my recent publication:
Klaming, L., Spaltman, M., Vermeent, S., van Elswijk, G., Miller, J.B., & Schmand, B. (2024). Test-retest reliability and reliable change index of the Philips IntelliSpace Cognition Digital Test Battery. The Clinical Neuropsychologist.
- Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19. ↩︎
- Chelune, G., Naugle, R., Lüders, H., Sedlak, J., & Awad, I. (1993). Individual change following epilepsy surgery: Practice effects and base-rate information. Neuropsychology, 7(1), 41–52; McSweeny, A. J., Naugle, R. I., Chelune, G. J., & Lüders, H. (1993). T-scores for change: An illustration of a regression approach to depicting change in clinical neuropsychology. Clinical Neuropsychologist, 7(3), 300–312. ↩︎