How many trials dey make reaction time test reliable?

The old numbers come from Hamsher and Benton, wey their abstract talk clearly say the conclusion na only for the conditions wey dem use for the study. No use am as certification rule for modern browser game. Reliability na whether one defined score dey come out again the same way; power na chance to detect one defined effect. Dem dey related, but dem no be the same question. de S. Hamsher et al., 1977 Miller, 2023

Why repeating the same task dey help

If trials na independent draws and variability no change, standard error of their mean go equal trial standard deviation divide by square root of number of trials. The four-times example follow this formula. Real trials fit break these assumptions because people dey learn, tire or change strategy. So calculation of trial number na model, e no guarantee say one person score go stable. Miller, 2023

The modern analysis of trial number still show say effects fit change as people dey practise. More trials no simply make more copies of one first trial wey no change. Research plan need choose one outcome and consider how the task dey behave throughout the session. Miller, 2023

Which kind reliability you want?

Consistency inside one run different from consistency across days, and both different from consistency for how people rank. Research about reliability paradox show say tasks fit get strong group effects but less reliable differences between individuals. Big average Stroop or reaction effect no automatically mean say short score fit assign person percentile. Hedge et al., 2017

Mean, median and minimum dey answer different questions

The fastest response measure another part of performance from the mean. E dey especially depend on how many chances you give yourself. Very slow results fit push mean upward, but deleting them without rule wey you set before fit bias the result. Research about reaction-time outliers na reason to keep scoring rules fixed before you compare runs. Ratcliff, 1993

Research about sleep show why the outcome you choose matter: psychomotor vigilance researchers dey separate slow responses and lapses from central reaction-time summaries. You no fit assume say quick game wey pick the best from some tries go show the same weakness to sleep loss as task wey test vigilance for long time. Basner et al., 2011

Wetin fit change the number of trials wey e look like say you need?

Device jitter dey add variability, and fixed delay from device dey add bias. More trials fit help with the first one, but dem no dey calibrate away the second one. Both timing evaluations and laboratory calibration results support make we separate these sources of error. Anwyl-Irvine et al., 2020 Bridges et al., 2020 Woods et al., 2015

For informal tracking, choose method wey you fit manage, use the same number of tries each time, and record every run you finish instead of only your personal records. Treat am as way to compare, no be confirmed instruction for one particular minimum. If you want scientific percentile, the task need sample of people wey represent the population, and measured reliability, plus enough trials. Hedge et al., 2017 Miller, 2023

Age, preparation and sleep fit still change performance during or between sessions. If you keep conditions fairly steady, e go easier to understand the comparison, but no amount of standardisation fit turn browser result into clinical assessment. Woods et al., 2015 Basner et al., 2011