Evaluation of the Critical Thinking for Achievement Programme (GA/ASE)
Background
You, or staff in your school, are taking part in the Critical Thinking for Achievement TLIF Programme provided by the Geographical Association/Association for Science Education (GA/ASE). This project is funded by the Department for Education (DfE) through Round 2 of its Teaching and Leadership Innovation Fund (TLIF), a three-year investment programme, which supports high-quality professional development for teachers and school leaders in the areas and schools in England that need it most. The programme aims to improve the outcomes of children and young people, thereby making a significant contribution towards tackling social mobility.
The evaluation
The National Foundation for Educational Research (NFER) and the Sheffield Institute of Education (SIoE) have been commissioned by the DfE to evaluate TLIF. The evaluation has a number of components. These include an overarching fund-level evaluation, and bespoke evaluations of each project that is funded as part of the programme. Each component includes process (formative) and impact (summative) evaluations. The evaluation will be conducted between September 2018 and October 2021.
The objectives of the evaluation are to:
- assess the effectiveness of the fund overall and the individual projects that have received funding
- assess the outcomes and impacts of the fund overall and the individual projects, and explore how these vary depending on the focus of the project, delivery model, and participant and school characteristics
- compare and contrast the outcomes and additionality (added value) of individual projects or types of projects across the fund
- examine how TLIF has influenced the wider CPD market, and how it has interacted with other targeted funds such as Opportunity Areas and the Strategic School Improvement Fund
- share emerging evidence with the Department on what works in teacher CPD and leadership development.
Data we will collect
During the project we will undertake:
- Telephone interviews with:
- subject leaders (30 minutes) and teachers (45 minutes), 40 interviews will take place in total, these will collect information on participants' perceptions of the success of the programme and the programmes impact on; confidence, pupil engagement and achievement.
- members of the GA/ASE delivery team, four 45 minute interviews will take place to understand the CPD model and gather their perspectives on the implementation of the programme and its impact.
- GA project managers, three interviews will take place (2018/19, 2019/20, 2020/21). These will assess progress towards targets, perceptions of implementation and reported outcomes for the programme overall. Questions will explore the extent to which the logic model assumptions and mechanisms have worked in practice, and areas of learning and further improvement. Interviews will take approximately 45 minutes.
- School case study visits with up to 4 schools who have hosted the GA/ASE training. Case studies will involve:
- an observation of a lesson in science or geography where Critical Thinking for Achievement materials are used.
- focus groups with staff who have had the training, to discuss perceived impacts and how they have implemented the programme.
- an interview with a member of the senior leadership team OR with the subject leader, for a strategic-level comment on the programme.
- pupil focus groups – to gather pupils’ views, and perceptions of the impacts of the programme.
In addition we use the following sources of secondary data:
- Management information collected by the GA team. DfE will provide us with analysis of programme recruitment and retention data which will include in our evaluation report.
- National pupil database - at the end of the evaluation, we will compare pupil outcome data - attainment at KS2 and GCSE - from participating schools with those from a comparison group of similar schools who are not receiving support through the TLIF. This will enable us to assess what would have happened if the fund had not been introduced, and to investigate the extent to which funded activities add value to existing practices.
- Schools workforce database - at the end of the evaluation we will use schools workforce data to examine the impact that the GA/ASE TLIF programme and the fund as a whole has had on teacher retention and progression.
Ethics and data protection
Personal data collected by the Geography Association from those involved in Network and Development Days
As outlined in the DfE Privacy notice the GA/ASE will have sent details of participants to DfE when they started the project. Teachers and leaders participating in projects funded by TLIF are required to provide personal information (such as your name, teacher number and email address). GA/ASE is contractually bound to provide that information to DfE. This information will be shared with NFER and the SIoE for research purposes only, including linking to national data-sets such as the Schools Workforce Census. Any personal data shared between the NFER, the SIoE and DfE will be handled securely and confidentially. We set out how we will use your personal data and the legal basis for processing your personal data in the evaluation Fair Processing Notice.
Voluntary participation, confidentiality and right to withdraw
Participation in telephone interviews is voluntary. All telephone interview data will analysed anonymously.
You can withdraw your data at any point until it has become fully anonymised and aggregated into the analysis at which point we would be unable to identify it. If you wish to withdraw your data please notify Dr Bronwen Maxwell (contact details below). Your decision to take part or withdraw from the evaluation will not influence your participation in the TLIF programme in any way.
In all reporting, no individual participant in a school will be identified or identifiable and the school will not be named. Where appropriate, GA/ASE staff involved in programme design and delivery may be identified in reporting by their role title in relation to the project. This may make them identifiable to readers that are familiar with the organisation. Care will be taken to ensure full anonymisation of any sensitive data.
Further information on our ethics code of practice
NFER and SIoE will conduct the project in line with NFER’s Code of Practice, which complies with Social Research Association (SRA) and Market Research Society (MRS) guidelines. Ethics approval has been obtained through Sheffield Hallam University's Faculty of Development and Society's Ethics Committee.
Reporting
We will produce interim reports for DfE in September/October 2019 and September/October 2020 to share learning on effective CPD and inform funding prioritisation during the lifetime of the fund. These reports will not be published. A final report will be completed in September/October 2021 and published on the DfE website.
We may also present our findings at conferences and publish articles in professional and academic journals.
For further information about this evaluation, please contact:
Suzanne Straw (NFER Project Director)
Deputy Head of Centre for Evaluation and Consultancy
National Foundation for Educational Research, Unit G3, IT Centre
York Science Park, Heslington, York, YO10 5NP
01904 567661
Dr Bronwen Maxwell (SIOE Project Director)
Head of Commissioned Research and Deputy Head of Centre for Research and Knowledge Exchange, SIoE, Sheffield Hallam University S1 1WB
0114 225 5166