“We valued the transparency and organisational skills which NFER brought to the project. The team were extremely well-organised, easy to work with, and always on hand to answer statistical queries and offer clarity.”
Code Club is a nationwide network of free after-school clubs for children aged 9-13 years that provides projects and materials to support the teaching of Scratch, HTML/CSS, and Python. Children taking part in Code Club are taught to program by making games, animations, websites and applications. The aim is that they develop skills that will be useful to them in their future hobbies, schooling, and career, and may inspire them to pursue programming and other digital making activities in the future.
Why was an evaluation needed?
NFER was commissioned to undertake a randomised controlled trial of Code Clubs between June 2015 and September 2016, in order to robustly assess the impact of attending Code Club on children’s computational thinking and attitudes towards computers and coding more generally. In addition, NFER undertook a process evaluation to methodically explore whether Code Clubs had been delivered as intended, key success factors, any barriers to delivery, and the perceived outcomes for pupils, teachers, and schools.
What was involved in the evaluation?
NFER recruited twenty-one schools to participate in the trial with eligible pupils randomised into one of two groups: pupils who attended Code Club and pupils who were not offered a place in Code Club and who continued as they would do normally. The primary outcome measure was the Bebras Computational Thinking Assessment, which pupils completed at baseline and endpoint. Secondary measures included a Coding Quiz and Pupil Attitude Survey. The pupil-randomised design enabled NFER to compare pupil outcomes in the intervention group with pupil outcomes in the control group, in order to identify a causal impact of the Code Clubs on computational thinking and attitudes. The process evaluation included consultations with all teachers running Code Clubs, exploring the progress of schools’ Code Clubs over the academic year, including key success factors and any barriers encountered, as well as outcomes for pupils, teachers and schools.
What were the results?
The evaluation found that attending Code Club for a year did not impact on pupils’ computational thinking over and above changes occurring anyway, as both the intervention and control group increased their Bebras scores between baseline and endpoint. However, attending Code Club was found to significantly improve pupils' coding skills in Scratch, HTML/CSS and Python, even when control children learn coding within the regular curriculum. The evaluation therefore evidenced the real benefits of Code Club to pupil outcomes within the timeframe measured, adding credibility to the intervention and enabling Code Club to correctly position it for stakeholders to make confident, informed decisions about its value to pupils. The process evaluation also shed light on the factors integral to the success of programme, arming Code Club with valuable evidence to inform its ongoing delivery strategy.
More information about the Code Club evaluation, including the full report, can be found here.