The goal at Quantified Mind is to provide cognitive assessment tools that require nothing but a computer and a single enthusiastic user. To do this, we have scoured, condensed, and adapted what is known in the science of psychometrics. Here is an outline of our approach, with some detail on each area of cognitive performance that Quantified Mind can test.
While the history of measuring cognitive performance is somewhat controversial, there is ample evidence demonstrating a strong connection between assessed intelligence and success in a variety of “real world” tasks. For example, IQ and related cognitive measurements closely predict school performance, completion of higher education, occupation, and income.
This is the case even where variables such as social class, parental education, and other obvious potential confounds are controlled for [1][2][3]. While the precise mechanisms and causal interaction metrics of intelligence are not yet well understood, it is clear that cognitive performance has practical importance.
Quantified Mind measures a wide range of specific cognitive skills rather than having a focus on IQ. Our tests are patterned on scientifically validated and widely trusted assessments, modifying them where possible according to the following goals:
Repeatability
Repeatability is essential for a battery that is to be used for testing the effects of different interventions and lifestyle patterns over time. Repeatable tests must be generated automatically and minimize practice effects. We aim for tests that allow people to rapidly plateau through practice, so that any changes in performance trends once the plateau has been reached can be attributed to genuine improvements or degradations of the skill in question. In addition, the tests dynamically challenge users as much as possible, adaptively adjusting their difficulty in response to previous performance and providing constant feedback.
Efficiency
To minimize the time demands on users, we have put a great deal of effort into making our tests as efficient as possible -- yielding statistically reliable results quickly, and extracting maximal information from a short testing period. Having tests that dynamically adjust ensures that time and energy are never wasted while users mechanically negotiate trivial versions of a task waiting to progress to the point where they are challenged. Adaptive testing also improves the quality of our performance statistics, because statistical power is greater when people are measured at the point where they are optimally challenged.
Purity
The purity of a test is a measure of how well the test can isolate and reliably evaluate a specific cognitive skill. For instance, if a test were to evaluate performance on a specific task, but there were many different ways of performing that task or distinct strategies that could be used to improve performance, the test would be very impure. Maintaining purity is important because scores in pure tests are much easier to interpret than scores on impure tests; it is much easier to get a clear picture of what cognitive ability is being employed and how skillfully. Pure tests also tend to be less susceptible to practice effects.
Validity
Validity is the extent to which a test measures what it purports to measure, or the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of the tests. The Quantified Mind tests are based on well-validated tests in the scientific literature.
Engagement
The tests must be fun and easy to navigate, with reminders to come back to use them again, so that people will be engaged and participation in studies will be as high as possible. Having Quantified Mind as an online platform allows us to design the flow of going through the tests in a maximally engaging way, and to measure and improve where we find that people are dropping off.
Repeatability, efficiency, validity, purity and engagement are guiding principles in the test design process for Quantified Mind. The remainder of the document will discuss our approach to assessing specific cognitive skills of various sorts.
What is reaction time?
Reaction time is the measured time lapse between the presentation of a stimulus and the participant’s production of a simple behavior. Normally, tests of reaction time involve assessing behavior that requires little (or no) inference or complex cognitive processing.
What tests of reaction time have been developed?
There are many kinds of reaction time tests. Simple ones test the participant’s ability to react to a single, simple stimulus, like a light that turns on. More complex ones assess reaction times in scenarios where multiple different stimuli might be presented, and different reactions must be chosen depending on the specific stimulus.
The Test of Variables of Attention (T.O.V.A.) is an example of the sophistication possible in a simple test. The T.O.V.A. assesses sustained attention, inhibition, and impulsivity by measuring whether and how quickly a user presses a button in response to a target stimulus [4].
What is the connection between reaction time measured in these tests and real world cognitive performance? There is a fair body of experimental evidence demonstrating that performance on both simple and choice reaction time tests is modestly correlated with IQ test scores [5][23][43]. The cause of the correlation is unclear, and may involve more efficient information processing, better attentional control, or the integrity of neuronal processes. Reaction times have also been shown to predict important real-life outcomes that rely on processing speed [6].
Interestingly, the T.O.V.A. has been shown to be an indicator of attention problems closely associated with ADHD [7]. There also appear to be signature differences between the performance of children with and without autism on the test [8].
How does Quantified Mind test reaction time?
Quantified Mind has three reaction time tests. The first is “Simple Reaction Time,” which assesses how quickly a person can respond to a single stimulus (a circle that turns from white to green) by pressing the space bar. The second is “Choice Reaction Time,” which assess how quickly a person can respond to one of three stimuli (one of three numbered circles that randomly turn from white to green) by pressing the appropriate number key that corresponds to the green circle.
The third test is “Go/ No-Go,” which is similar to the T.O.V.A. Target stimuli are small white squares that appear at the top of a black screen. There are also nontarget stimuli, small white squares that appear at the bottom of the black screen, which the user is instructed to ignore. Stimuli are presented one at a time, and statistics are kept on how often a target stimulus is ignored by the user, how often the user presses the button in response to a non-target stimulus, how quickly the user presses the button in response to target stimuli, and how often the user presses the button too soon for target stimuli. This combination of measurements assesses sustained attention--the ability of a user to pay attention in spite of boredom and inactivity, and inhibition control--the ability to react quickly and accurately to target stimuli while managing to avoid reacting to non-targets or reacting too early to targets.
What is executive function?
The term ‘executive function’ refers to a range of loosely related cognitive abilities associated with high-level processing [44]. These include inhibition control, planning, inference, organization, self-regulation, and initiation of behavior. When executive function is impaired, a person's functional outcome, medication compliance, and capacity to give informed consent are also negatively affected [40].
What tests of executive function have been developed?
A classic variety of executive function tests is the so-called “Stroop Task,” named after the English psychologist John Ridley Stroop. Stroop tasks measure the ability of participants to attend alternatively to reading printed color words (e.g., red) and noticing the color those words are printed in.
Another executive function test is the Wisconsin Card Sorting Test (WCST), which is sensitive to three distinct cognitive processes: cognitive flexibility, problem-solving, and response maintenance. In the WCST, a set of fixed cards is displayed, and new single cards are presented sequentially. The participant must match the new card to some characteristic of the fixed cards, for example color, number, or design [25].
What is the connection between executive function measured in these tests and real world cognitive performance?
There is evidence that scores on Stroop tests among healthy adults correlate highly with scores on Raven’s Progressive Matrices, often considered a good measure of fluid intelligence [9]. Stroop scores in this population also correlate with with several Wechsler Adult Intelligence Scales (WAIS) subscores that measure processing speed and conceptual ability [10].
Interestingly, depression and a variety of mental illnesses impede scores on Stroop tasks. The validity of Stroop and sorting tests has also been demonstrated in terms of relation to activation and damage to the frontal lobes [24]. And Stroop tests were shown to have strong positive predictive value for diagnosing persistent postconcussion syndrome [22].
How does Quantified Mind test executive function?
Quantified Mind’s executive function tests focus on attention, inhibition and context-switching. The Quantified Mind “Sorting” test measures the ability to switch contexts by asking people to match displayed images to fixed background images according to one of several different rules, with a potential shift of rule on every trial. In addition, Sorting also tests inhibition control since stimuli can be concordant or discordant to various degrees.
The Stroop tasks that are commercially available and the Quantified Mind test “Color-Word” are similar. “Sorting” is different from tests that are commercially available, but it assesses task-switching abilities similar to those in Stroop tasks, and includes a minimal working memory component as well, since remembering which species and colors go with which numbers speeds responses slightly. Quantified Mind also has two attention tests: one called “Cued Attention”, which is based on the popular Posner spatial cueing task [26], and one called “Attentional Focus”, where a person must detect subtle changes in line length [27].
What is verbal learning?
Simply put, verbal learning is the learning of words. This learning can be tested by free recall, prompted recall, or recognition. We also understand verbal learning to include learning of nonsense words, which are pronounceable but which do not form meaningful units of English.
What tests of verbal learning have been developed?
Verbal learning has historically received significant attention from psychometricians, and as a result there are many tests of verbal learning available with subtly different methods and emphases. One of the most well-known tests is the Auditory-Verbal Learning Test (AVLT), where a 15-word list is read to the participant at the rate of one word per second on five successive trials. After each trial, the participant is invited to recall as many of the words as possible. After this, an interfering list of 15 words is read followed by an immediate request for the participant to recall the original list, and another request is made after a delay of 20 to 45 minutes.
Another well-known test is the Wechsler Verbal Paired Associates [11]. Unlike the AVLT, the Verbal Paired Associates test assesses the participant’s ability to recall one member of a word pair, rather than an individual unpaired word. The original Wechsler format involves an experimenter reading ten word pairs, six forming “easy” associations (e.g., ‘baby’- ‘cries’) and four forming “hard” associations (e.g., ‘cabbage’-‘pen’). The list is then read three times, with a memory trial following each reading [12].
What is the connection between verbal learning measured in these tests and real world cognitive performance?
There is considerable evidence that verbal learning correlates reasonably strongly with performance in a number of important practical tasks. For instance, verbal learning tests have been demonstrated to be highly correlated with prospective remembering in real life, which means remembering to perform a planned action at the appropriate time [29]. In addition, the Verbal Paired Associates test from the Wechsler Memory Scale is one of the most widely used instruments for assessing explicit episodic memory [30].
The California Verbal Learning Test (CVLT), a formal verbal memory test, has been used on people with frontal lobe lesions. Frontal patients show deficits on this test, including poorer recall, reduced semantic clustering, and impaired yes/no recognition performance [13]. Also, in a study that asked participants to later recall words presented earlier (delayed verbal recall), it was shown that this delayed verbal recall was associated with motor learning in older adults [28].
How does Quantified Mind test verbal learning?
Quantified Mind has two tests for this: “Verbal Learning” and “Verbal Paired Associates,” which ask people to recall a series of words presented visually on the screen in three successive trials, either as single words or paired with other words. The words are either English words or nonsense words that are pronounceable but have no English meaning.
Corresponding to each of these tests is a test that assesses later recall. These tests are meant to be taken at least 5 minutes after the initial corresponding test, and include no rehearsal of the items that should have been learned. The user is simply prompted to recall them in the same way as on the original test.
What are working and short-term memory?
Working and short-term memory refer to memory available for immediate cognitive processing, storage, and manipulation of information, in contrast to long-term memory, which stores important information for future use. In other words, working memory is the cognitive ability to hold a certain amount of information in mind in an accessible way that can be used in real-time mental tasks [40].
What tests of working and short-term memory have been developed?
One of the most common kinds of assessments of working memory is the digit span test. Digit span tests present the participant with a list of numbers and then ask for the numbers to be repeated back. These tests come in different varieties that can be classified according to how the digits are presented or how the digits are to be recalled by the participant. The stimuli are usually presented auditorily. Where recall is concerned, some versions require the digits to be recalled forwards (“forward digit span tests”) and others require the digits to be recalled backwards (“backward digit span tests”) [31][32].
In addition to digit span tests, there are spatial span tests. Rather than presenting digits, these tests typically require people to view and then reproduce an ordered spatial pattern. The popular electronic children’s game “Simon” is a version of a spatial span test, albeit one with only four possible locations that the pattern can trace its way through. What is more typical is a square grid with 9, 16, 25, or more squares. These squares light up in an ordered pattern, and any of the squares could be the next one in the pattern. Just as with digit span tests, some versions require the user to recall the pattern forwards and others backwards [40].
Another variety of working memory test is similar to spatial span tests, except that rather than presenting the patterns in order and requiring people to reproduce them in the same or opposite order, these tests present a group of squares all lit up at once, and people must then afterwards indicate which squares were lit up and which were not. Quantified Mind refers to these tests as “Design Copy” tests.
There are also N-back tests, which are useful for looking at working memory and higher cognitive functions such as fluid intelligence [38][39]. In fact, N-back has been referred to as the gold standard working memory assessment technique in cognitive neuroscience [40]. In an N-back test, a series of stimuli is presented, and the participant is asked to recall which stimulus was presented a specified number (n) back in the sequence. Stimuli can be numbers, letters, or pictures. If n is one, the correct response is the stimulus presented immediately prior to the last one. If n is two, the correct response is the stimulus presented two prior to the last one.
What is the connection between working and short-term memory measured in these tests and real world cognitive performance?
Working memory is considered one of the most important and most studied mental skills today [40]. Forward digit span tests have been shown to measure efficiency of attention in addition to memory [14][15][31], and performance can be impeded by stress [12][16].
Children with impaired working memory have been found to have short attention spans, high levels of distractibility, problems in monitoring the quality of their work, and difficulties in generating new solutions to problems [41]. Working memory is strongly linked to the ability to learn reading and math, and measurements of working memory are effective predictors of later academic achievement. [42].
How does Quantified Mind test working and short-term memory?
Quantified Mind has tests in each of the areas discussed above. We have both forward and backward digit span tests, with digits displayed on-screen. While spoken recall is typically used in clinical settings for digit span tests, recall for our versions is via keyboard typing.
We have both forward and backward spatial span tests, where some squares in a white grid turn black. The same setup is used for our pattern reproduction tasks, “Design Copy” and “Design Recognition.” The main difference between Design Copy and Design Recognition is that Design Copy requires the person to fill in the design from memory, while Design Recognition asks the person to choose the correct pattern out of four options that matches the original pattern displayed.
In addition, Quantified Mind has another test called “Visuo-Spatial Paired Associates.” This test also uses a grid, but requires the participant to remember where distinctive shapes have been placed in the grid and then recall these positions later when cued by a display of the particular shape.
There is also a self-paced 1-back, where a cartoon animal is shown, and people must react in one way if the next cartoon animal shown matches the previous one, and a different way if it fails to match the previous one. The test is called “self-paced” because the next cartoon animal is shown immediately upon response--people can take as much or as little time as they are comfortable with. 2-back and 3-back are also available.
As with many of our tests, the tests in this area (aside from the self-paced n-back) adapt to user skill by increasing or decreasing the length of any given series that must be repeated.
What are visual perception and cognition skills?
Visual perception and cognition skills are skills that have to do with the perception of visual information and the ability to process and manipulate visual images.
What tests of visual perception and cognition skills have been developed?
A variety of tests of visual perception and cognition has been developed. One group of tests seeks to measure the ability to perceive the location of stimuli that appear or disappear quickly. Those familiar with the popular brain training website Lumosity.com will recognize the games “Birdwatching” and “Eagle Eye” as being based on tests in this category.
Another group of tests assess a person’s ability to rotate given shapes or objects mentally. Such tests have been a longtime feature of IQ batteries like the WAIS, for instance. There is also a test called SDMT, or Symbol Digit Modalities Test, that asks participants to match a given symbol to a number, using a decoding key that lists which symbols correspond to which numbers [36][37].
What is the connection between visual perception and cognition skills measured in these tests and real world cognitive performance?
The SDMT test is reported to be the most sensitive test to the presence of acute or chronic ‘organic’ cerebral dysfunction, and so it has potential diagnostic properties [36]. And performance on visual acuity tests that evaluate a person’s ability to spot and identify objects that appear quickly is inversely correlated with the frequency of driving accidents.
On the fun side, this same visual acuity tends to correlate with performance on action video games. The same goes for mental rotation abilities. There also appears to be a causal process in the opposite direction as well, from frequent video game play to performance on these tests [17].
How does Quantified Mind test visual perception and cognition skills?
The Quantified Mind test “Mental Rotation” presents people with highlighted patterns on a grid, then gives them a set of possible answers as to how the pattern would look if rotated and placed elsewhere in the grid. The placing of the object in the correct answer’s grid and the angles of rotation are varied randomly.
“Visual Matching” places two highlighted grids next to one another and asks a person to judge, as quickly as possible, whether the two grids are identical. Quantified Mind also has tests to measure visual attention and the ability to notice quick changes in the visual field.
Finally, there is a mixed visual memory, attention, and acuity test called “Coding.” Takers of this test must match the presently displayed image to one of the one images displayed above under each of nine numbers. Good visual memory improves performance on this task by making the answering process more efficient, but visual memory is not strictly necessary, since the user always has the option to directly compare the displayed image with the answer choices.
What are motor skills?
Motor skills are a family of abilities related to the speed and coordination of bodily movements.
What tests of motor skills have been developed?
There have been numerous tests of motor skills developed, though many focus on motor impairments of one sort or another. One test of manual motor skill that has been quite prominent is the “Finger Tapping Test.” A simple assessment, this test has participants tap a finger from each hand (one hand at a time) as many times as possible in a given time period. Generally, trials are repeated multiple times and a mean taken [4].
What is the connection between motor skills measured in these tests and real world cognitive performance?
Finger Tapping is believed to be one of the most sensitive neuropsychological tests for determining brain impairment [18][19][34]. There is evidence of correlation in adolescents between performance in this test and various verbal skills [20]. Surprisingly, there also seems to be a modest correlation with WAIS assessments of Perceptual Organization, which focus somewhat heavily on visual processing [21]. And finally, Finger Tapping is being used as a way to monitor cognitive decline in terminal conditions such as Parkinson’s Disease [35].
How does Quantified Mind test motor skills?
We have implemented our own version of the Finger Tapping Test, simply called “Finger Tapping.” This test involves two alternating trials for each hand. The trials are 10 seconds each, and involve tapping the space bar as many times as possible in this time.
If you have gotten this far, here's a big thank you **HUG** for reading about the Science of Quantified Mind! Before we finish, we just wanted to point out that Quantified Mind is not the same as formal neuropsychological testing and is not diagnostic. Make sure to consult a medical professional if you have any concerns about your mental health [33].
1. Jencks, C. (1979). Who Gets Ahead? The Determinants of Economic Success in America. New York: Basic Books.
2. Rehberg, R.A. and Rosenthal, E.R. (1978). Class and Merit in the American High School: An Assessment of the Revisionist and Meritocratic Arguments. New York: Longman.
3. Brody, N. (1992). Intelligence, 2nd Edition. Academic Press, 1992.
4. Strauss, E., Sherman, E. M. S., and Spreen, O. (2006). A Compendium of Neuropsychological Tests: Administration, Norms, and Commentary, 3rd edition. Oxford: Oxford University Press
5. Detterman, D. K. (1987). What does reaction time tell us about intelligence? in P.A. Vernon (ed.), Speed of Information-Processing and Intelligence. Norwood, NJ: Ablex.
6. Jensen, A. R. (2006). Clocking the Mind : Mental Chronometry and Individual Differences (2006)
7. Schatz, A. M., Ballantyne, A. O., Trauner, D. A. (2001). Sensitivity and specificity of a computerized test of attention in the diagnosis of attention-deficit/hyperactivity disorder. Assessment, 8(4), 357-365.
8. Schatz, A. M., Weimar, A. K., and Trauner, D. A. (2002). Attention differences in Asperger syndrome. Journal of Autism and Developmental Disorders, 32(4), 336-336.
9. Anstey, K. J., Dain., S., Andrews, S., and Drobny, J. (2002). Visual abilities in older adults explain age-differences in Stroop and fluid intelligence but not face recognition: Implications for the vision-cognition connection. Aging, Neuropsychology, & Cognition, 9, 253-265.
10. Graf, P., Uttl, B., and Tuokko, H. (1995). Color- and picture-word Stroop tests: Performance changes in old age. Journal of Clinical and Experimental Neuropsychology, 17, 390-415.
11. Wechsler, D. (1987). Wechsler Memory Scale--Revised manual. San Antonio: The Psychological Corporation.
12. Lezak, M.D., Howieson, D. B., and Loring, D. W. (2004). Neuropsychological Assessment, 4th Edition. Oxford: Oxford University Press.
13. Baldo, J. V., Delis, D., Kramer, J., & Shimamura, A. P. (2002). Memory performance on the California Verbal Learning Test-II: findings from patients with focal frontal lesions. Journal of the International Neuropsychological Society: JINS, 8(4), 539–546.
14. Kaufman, A.S., Kaufman-Packer, J. L., McLean, J. E., and Reynolds, C. R. (1991). Is the pattern of intellectual growth and decline across the adult life span different for men and women? Journal of Clinical Psychology, 47, 801-812.
15. Fowler, P. C., Richards, H. C., Boll, T. J., and Berent, S. (1987). A factor model of an extended Halstead Battery and its relationship to an EEG lateralization index for epileptic adults. Archives of Clinical Neuropsychology, 2, 81-92.
16. Mueller, J. E. (1979). Test anxiety and the encoding and retrieval of information. In I. G. Sarason (ed.), Test Anxiety: Theory, Research, and Applications. Hillsdale, NJ: Erlbaum.
17. Dye, M. W.; Green, C.S.; Bavelier, D. (2009). Increasing speed of processing with action video games. Current Directions in Psychological Science, 18(6), 321-326.
18. Leckliter, I. N. and Matarzazzo, J. D. (1989). The influence of age, education, IQ, gender, and alcohol abuse on Halstead-Reitan neuropsychological test battery performance. Journal of Clinical Psychology, 45, 484-512.
19. Horton, A. M. (1999). Above-average intelligence and neuropsychological test score performance. International Journal of Neuroscience, 99, 221-231.
20. Stanford, M. S., and Barratt, E. S. (1996). Verbal skills, finger tapping, and cognitive tempo define a second-order factor of temporal information processing. Brain and Cognition, 31, 35-45.
21. Berger, S. (1998). The WAIS-R factors: Usefulness and construct validity in neuropsychological assessment. Applied Neuropsychology, 5, 37-42.
22. Cicerone, K. D., and Azulay, J (2002). Diagnostic utility of attention measures in postconcussion syndrome. The Clinical Neuropsychologist, 16(3), 280-9.
23. Deary, I. J., Der, G., and Ford, G. (2001). Reaction times and intelligence differences: A population-based cohort study. Intelligence, 29(5), 389-399.
24. Demakis, G. J. (2004). Frontal lobe damage and tests of executive processing: a meta-analysis of the category test, stroop test, and trail-making test. Journal of Clinical and Experimental Neuropsychology, 26(3), 441-50.
25. Greve, K. W., Love, J. M., Sherwin, E., Mathias, C. W., Ramzinski, P., & Levy, J. (2002). Wisconsin Card Sorting Test in chronic severe traumatic brain injury: factor structure and performance subgroups. Brain injury: [BI], 16(1), 29–40.
26. Posner, M. I. (1980). Orienting of attention. The Quarterly journal of experimental psychology, 32(1), 3–25.
27. Cohen, M. R., & Maunsell, J. H. R. (2011). When attention wanders: how uncontrolled fluctuations in attention affect performance. The Journal of neuroscience: the official journal of the Society for Neuroscience, 31(44), 15802–15806.
28. Tunney, N., Taylor, L. F., Higbie, E. J., & Haist, F. (2001). Declarative Memory and Motor Learning in the Older Adult. Physical & Occupational Therapy in Geriatrics, 20(2), 21–42.
29. Titov, N., & Knight, R. G. (2000). A procedure for testing prospective remembering in persons with neurological impairments. Brain injury: [BI], 14(10), 877–886.
30. Uttl, B. (2002). Verbal Paired Associates tests limits on validity and reliability. Archives of Clinical Neuropsychology, 17(6), 567–581.
31. Groth-Marnat, G., & Baker, S. (2003). Digit Span as a measure of everyday attention: a study of ecological validity. Perceptual and motor skills, 97(3 Pt 2), 1209–1218.
32. Wechsler D. Manual for the Wechsler Adult Intelligence Scale— third edition. San Antonio, TX: Psychological Corp; 1997
33. Gualtieri, C. T., & Johnson, L. G. (2006). Reliability and validity of a computerized neurocognitive test battery, CNS Vital Signs. Archives of Clinical Neuropsychology, 21, 623-643.
34. Mitrushina, M. N., Boone, K. B., & D’Elia, L. F. (1999). Handbook of Normative Data for Neuropsychological Assessment. New York: Oxford University Press
35. Riggare, S. (2012). MyOwnPROMS. Poster presented at the 10th anniversary conference of Medical Management Center at Karolinska Institutet. http://www.riggare.se/2012/06/19/myownproms/
36. Smith, A. (1982). Symbol Digit Modalities Test (SDMT). Manual (Revised). Los Angeles, Western Psychological Services.
37. Lezak, M. D. (1994). Domains of behavior from a neuropsychological perspective: the whole story. Nebraska Symposium on Motivation. Nebraska Symposium on Motivation, 41, 23–55.
38. Kane, M. J., Conway, A. R. A., Miura, T. K., & Colflesh, G. J. H. (2007). Working memory, attention control, and the N-back task: a question of construct validity. Journal of experimental psychology. Learning, memory, and cognition, 33(3), 615–622.
39. Jaeggi, S. M., Buschkuehl, M., Perrig, W. J., & Meier, B. (2010). The concurrent validity of the N-back task as a working memory measure. Memory (Hove, England), 18(4), 394–412.
40. Hill, B. D. (2008). The construct validity of the clinical assessment of working memory ability. A dissertation submitted to the Graduate Faculty of the Louisiana State University in the Department of Psychology.
41. Alloway, T. P., Gathercole, S. E., Kirkwood, H., and Elliott, J. (2009). The Cognitive and Behavioral Characteristics of Children With Low Working Memory. Child Development, 80(2), 606-621.
42. Holmes, J., Gathercole, S. E., & Dunning, D. L. (2009). Adaptive training leads to sustained enhancement of poor working memory in children. Developmental Science, 12(4), F9–F15.
43. Deary, I. J., & Der, G. (2005). Reaction Time, Age, and Cognitive Ability: Longitudinal Findings from Age 16 to 63 Years in Representative Population Samples. Aging, Neuropsychology, and Cognition, 12(2), 187–215.
44. Diamond, A. (2013). Executive Functions. Annual Review of Psychology, 64, 135-168.