How to ensure a test is valid

Validity is commonly understood as referring to the outcomes of an assessment and whether the evidence known about the assessment supports the way in which the results are used. Test users need to be sure that the particular assessment they are using is appropriate for the purpose they have identified. They may find it useful to review evidence in the accompanying teacher’s guide or the technical guide.

Different forms of evidence can be used to develop an argument for the validity of the assessment. These include:

  • How comprehensively the test samples from the content domain chosen to be assessed – such as the curriculum for a particular subject.
  • The sample of pupils on which the test was standardised and how representative it is of the population on which the test will be used.
  • How well test performance matches expectations based on evidence available from other relevant sources.
  • The extent to which aspects that may be irrelevant to performance of the skill in question have been removed or minimised – such as the reading demand of a maths test.
  • How well test performance predicts attainment in the subject at a later date.

For a test to be valid, it must also be reliable (it is possible, however, for a test to be reliable and not valid). Test scores are reliable to the extent that they are not erratic or random. For example, a reliable test is one that would produce the same or very similar results when taken by the same pupil more than once within a close period of time.

NFER takes steps to ensure the validity of its assessments by:

  • Undertaking extensive research into effective assessment development to ensure that the methods and techniques used are scientifically robust, and assessments are underpinned by the highest measurement and psychometric standards.
  • Being committed to the principles of universal design.
  • Ensuring a deep knowledge and understanding of the domain to be assessed (such as the national curriculum) to ensure assessments provide adequate and appropriate coverage.
  • Scrutinising trial data to ensure that the items, sections and the whole test function well; that the test is internally reliable and free from bias.
  • Working with a wide range of schools and large groups of experienced teachers from a range of backgrounds to develop and trial test content, as well as set thresholds of achievement on some tests.
  • Publishing data concerning sampling, and item and test functioning.
  • Standardising tests with large and nationally representative samples drawn from the population that will be assessed.
  • Providing comprehensive teacher guides which give guidance on creating consistency during the test administration and marking process.

Although issues of validity are considered throughout the development of NFER’s assessments, the validity of a perfectly well-developed test may be compromised if it is misused or inappropriate inferences are drawn from the results. Therefore, there is also a key role for schools to play in ensuring the validity of test outcomes.

Schools can take steps to ensure the validity of their assessment processes by:

  • Choosing assessments that have been developed robustly, and which provide evidence concerning the properties of the assessment and guidance about the tests’ use.
  • Choosing assessments that are relevant to and provide adequate coverage of the content domain – such as the appropriate parts of the curriculum.
  • Using assessments for the purposes for which they were designed.
  • Following the teacher guides provided with assessments in order to create consistency in the test administration and marking process.
  • Drawing conclusions from assessment data in an appropriate way, considering other factors that may influence pupils’ test scores – e.g. physical, emotional and psychological factors.
  • Ensuring that those who mark the tests have adequate training and that any marking service used is robust.

For more on the effective use of assessment, head over to the NFER Assessment Hub where you'll find a host of free guidance and resources. You can also sign up to our monthly assessment newsletter for exclusive assessment-related content delivered direct to your inbox.

For more information on NFER’s popular range of termly standardised assessments for key stage 1 and 2, visit www.nfer.ac.uk/tests.