Problem Verification Task

Problem Verification Task

Example item as it appears on screen

Equations appear on the screen one at a time as above; some equations are mathematically correct, and some are incorrect. Participants respond by pressing keys F, J or K. They press F if they think the equation is correct; J if they judge the equation to be incorrect; or K if they don’t know. For every item, a reminder of which keys to press is shown on screen as in the above example.

The test consists of 2 practice and 48 main test trials. These items, with the correct answers, are tabulated below.

The test begins with the tutorial, comprising text instructions with a voice-over, and two practice items. This script for the tutorial has been provided separately, but is reproduced at the end of this document for reference. After the practice test, there are options either to repeat the practice test or to start the main test – subjects may repeat the two practice items as many times as they wish before starting the main test.

The come-back-later button is presented once only, half-way through the test, after item 24. At this point, the text on screen states:

You have now reached the half-way point in this activity. If you would like to stop for a while and complete the test later, click on the “come back later” button now.

If you want to continue press the SPACE-BAR.

There is no intermediate screen between items. After responding to one item, a subject is immediately presented with the next item on screen.

Test rules

·         No branching rules

·         No discontinue rule

·         Come back later option only after item 24

·         Time limit of 10 seconds for every item. A time bar appears at the top of the screen during every item, to remind subjects of the time remaining.

·         The response to each item is coded as 1=F/right, 2=J/wrong, 3=K/don’t know, 4=timed out

·         The scores for each item are 1=correct response, 0=incorrect response or don’t know or timed out

·         The reaction time (answer time) is also recorded for every item, to the nearest millisecond. For timed out items, the default time of 10 seconds is recorded.

·         No data is recorded for the practice items, and these do not contribute to the test score

·         The total test score is the sum of the item scores for the 48 main test items. The total score ranges from 0 to 48.

·         The test is effectively abandoned if interrupted at any point during the 48 main test items, except at the come-back-later point after item 24. This may happen if the subject closes the browser, clicks on the browser’s “back” or “refresh” buttons, or simply fails to respond for a length of time exceeding the session timeout. An abandoned test may not be resumed or repeated; when the subject next logs in, they are presented with the next activity. Abandoned tests are not included in the data analysis files.

Test items