The Secret Life of SATs Papers: 35 Facts You Didn’t Know

The KS1 and KS2 SATs can be a cause of consternation for teachers, and the process of creating a SATs test is surprisingly opaque. So here are your answers to all the facts about the SATs you always wanted to know such as who writes the KS2 SATs Papers and why are they so “hard”?

Are papers just written by someone at the DfE before they are unleashed on unsuspecting schools? Where do some of the questions come from (chicken, anyone?) And what is the “science” behind the SATs exams?

To find out the answer to questions like these, and more, we went undercover and talked to an individual involved in the test development process. We think that, like us, you’ll be surprised by some of the answers. Which is why we’ve put together the 35 most surprising insights about how SATs papers are developed.

N.B – Some names of individuals and companies have remained anonymous in order to maintain the security and privacy of the SATs examination process.

1. Let’s start with the most important fact: SATs papers aren’t actually put together by the DfE but they’re created by the Standards and Testing Agency (STA). The STA is actually managed separately from the DfE and has its own budget for this very reason.

The Secret Ingredients to SATs: Test Development Frameworks & Content Domains

Behind every good SATs paper, there is a robust test development framework and well defined content domains. After all, if we don’t know exactly what is being tested, then what is the point of the test?

2. Did you know that the papers that pupils sit during for their national assessments are actually put together according to the test development framework not the National Curriculum?

3. The test development framework, which is publicly accessible and should be essential reading for Year 6 Teachers, sets out all the objectives from across KS2 that can be tested within the KS2 SATs papers. This is what is referred to as the content domain.

4. The test framework also sets out the proportion of questions that should be tested from different content areas. For example, 83-93 marks each year will be from Number (including calculations and fractions), Ratio or Algebra content areas.

5. Also, of all the content domain references available, each year only around 40% covered in SATs are from Year 6 objectives. So here it is in black and white that KS2 SATs aren’t just about Year 6,  and it is theoretically possible for children to achieve expected standard, without being proficient in any of the Year 6 content.

Keep this in mind, when you’re creating your own programme of revision for maths KS2.

The “Science” Behind Testing: Why are SATs questions so hard?

Just like good teaching, good testing requires a strong theoretical understanding of learning and, especially in the case of SATs, how to objectively assess learning. With all this theory, it’s no wonder the SATs questions can seem so hard! Cue the four cognitive domains…

6. Alongside content domains, the test framework also sets out the four different cognitive domains that are crucial to how the government structures KS2 SATs.

In short, cognitive domains are the type of cognitive skills and abilities that each question is assessed against. And, interestingly, they are an often neglected part of the test framework when people create mock or sample papers, but they are an essential element of the test development process.

The four cognitive domains are as follows:

• The first cognitive domain is Depth of Understanding. This strand is used to assess the demand associated with recalling facts and using procedures to solve problems.
• The second cognitive domain is Computational Complexity, which is used to assess the computational demand of a question.
• The third cognitive domain, Spatial Reasoning and Data Interpretation, is used to assess the demand associated with the representation of geometrical problems involving shapes and position and movement. This strand is also used to assess the demand associated with interpreting data presented in tables, charts and graphs.
• The fourth cognitive domain is Response Strategy, which describes the demand associated with constructing a response to a question.

7. In the SATs papers, each question is scored against a 1-4 scale for each cognitive domain that it tests. In which one represents low (or none) and 4 represents high. A definition of what represents a low, intermediate and high level of ‘demand’ for each cognitive domain is provided in the test framework.

8. Questions which involve a higher level of content (i.e. from Year 5 or 6) often have cognitive domains that sit towards the lower end of each scale, or only have a high level of cognitive demand in one or two of the domains.

Usually, there are only a very small number of high content and high-cognitive domain questions (around 4-6 marks per year).

Third Space have worked with assessment experts to create their own sample practice year 6 maths SATs papers.

Third Space Learning also offers one to one, online SATs tuition aimed at helping to prepare Year 6 students for the upcoming SATs exams. With each programme specially tailored to the individual students’ needs, our SATs revision programme fills gaps in learning and gives students more confidence going into SATs.

The Minds Behind the Maths: Who Writes SATs Questions

You might wonder how some SATs questions end up in the papers, but the road to creating a good SATs question is actually a long and intensive one!

9. As teachers, SATs can be an understandable source of frustration. When individual questions, or the mark schemes can feel bewilderingly unfair, it might feel like they were put together all too quickly.

I’m finding this level of pedantry frankly bewildering #SATsshambles Struggling to spot the difference between K1 and K2 pic.twitter.com/eI83t9Qqsf

— Debra Kidd (@debrakidd) July 11, 2017

An example of an ‘unfair’ question, picked up by teachers lamenting the 2017 #SATsShambles.

But actually each SATs paper starts its journey at least two years before it arrives at your school. It’s far from a simple “write a paper and release it” process.

10. Questions (or “items” in test development speak) are written by a number of “item writing agencies”, who use the test framework to write individual questions. These agencies include well-known names within the UK educational scene, as well as those which are better known in other countries.

11. Once received into the Standards and Testing Agency, these “items” are reviewed by a diligent team of civil servants who oversee the test development process.

(These civil servants often have an education background themselves, and have specialised in test development)

12. All SATs questions are then entered onto a central database of “items”. This means that – if any are not used in the current test development process, they can still be called upon in future years.

13. The “items” that are selected as possible candidates for the papers are then taken to a series of expert review groups. There are several separate “groups”; including specialist teachers, academic experts and inclusion experts from various subjects.

The SATs Secret Society Behind the Test Development Process

Great assessment isn’t just about the strength of individual questions. Putting together a series of structured questions that pupils are required to answer in a set time, is a process in and of itself.

Stage 1 – SATs Question Level Review

14. The first meeting of the Expert Review Group for a given paper takes place over two years before the date of the assessment.

It goes without saying that these meeting are very strictly confidential. No notes can leave the room, and there is high security surrounding the meeting.

This also means that the test development process for at least two separate years of assessments takes place concurrently.

15. In this meeting, all of the items that have been selected for possible inclusion in the year are discussed over a two day period.

These are analysed in depth by the members of the group, and honest and open feedback given.

At least 150 SATs reasoning paper items are presented at these initial meetings – more than three times the number needed.

16. The expert review groups can suggest many things, such as whether or not questions are suitable, or whether amendments to wording, contexts, illustrations etc, need to be made. They even pour over which words should be in bold!

There is often discussion and debate amongst the different group members. A discussion on any particular question can last between five to 30 minutes. So you can imagine how long it takes!

17. The inclusion group focus on the inclusivity of each question, considering a whole range of factors. They also discuss any adaptations needed for the modified versions of the papers.

18. Then all groups consider the context of any questions, ensuring that contexts are as realistic and recognisable to pupils as possible.

This goes somewhere to explain the sometimes limited range of contexts used, as there are surprisingly few contexts that are universally “real” to most 11 year olds.

19. Finally, the feedback from each different expert review group is collated and used by the STA staff to make any amendments to the questions, or to refer questions back to the item writing agency.

GET THE ANALYSIS ON PREVIOUS SATS

SATs have been running in their current incarnation for 7 years; there were no government standardised assessments in 2020 and 2021 as a result of the Covid-19 pandemic. Get all the analysis and results from previous national assessments below.
And find out which are the top 20 year 6 maths revision topics to focus on this year.

– SATs papers 2024
– SATs results 2024
– SATs papers 2023
– SATs results 2023
– SATs papers 2022
– SATs results 2022
– SATs papers 2019
– SATs results 2019
– SATs papers 2018
– SATs results 2018
– SATs papers 2017
– SATs results 2017
– SATs results 2016

Stage 2 – SATs Sample Papers

20. Once the questions (or “items”) have been decided, they are formed to create a large number of sample papers.

These are put together in a way that ensures the test framework is rigorously followed and the appropriate mix of content and cognitive domains are covered across a “suite” of papers (i.e. Paper 1, 2 & 3).

21. Once the sample suite is assembled, they are given to Year 6 children to sample in schools across the UK.

Any UK state school could be selected to sample SATs papers and – if a school is selected – their Year 6 will be given at least one sample paper to be administered by an external agency, usually in June or July.

Schools cannot refuse involvement in this process and, accordingly, results from sampling are not given back to schools (nor linked to schools or pupils in any way!) Although teachers are usually asked to provide an assessment of those who sit the paper, as a baseline.

Note – There is no risk of your Year 6 actually sitting a sample question in their real SATs, as they are only exposed to questions that might be used two years later.

22. Interestingly, the sample papers are designed with a whole host of different variations. This means that things like the effect of question order can be tested and analysed.

23. After Year 6 have sat the sample papers, they are marked by a specialist team of markers. During this process the mark scheme and guidance is also reviewed and created where needed.

24. The markers flag any unexpected responses. For example, is a question “leading” children to give a certain incorrect answer? Incorrect answers like these are “coded” to describe the type of error.

25. Once a wide range of sample papers have been sat and marked, the results are statistically analysed in immense detail.

While the papers are analysed, many aspects of the data are weighed up and considered at this point, such as:

• Potential gender bias (more boys answering correctly than girls, or vice versa)
• The number of children getting the question correct; the impact of question placement
• The performance of high, medium and low ability children (as identified by their teachers)

26. During this process, the types of errors children have made on each question are also analysed heavily. To ensure that questions aren’t universally causing a certain error.

27. In questions with two or more marks, the number of children gaining partial marks (i.e. one mark from a two mark question) is also examined. This is to ensure that it’s possible both one and two marks. Not just the one partial mark.

28. Finally, once the in-depth analysis is complete, the questions are taken back to the second round of expert review groups.

Here, the amended questions (along with sampling results) are reviewed, and more suggestions are made.

We are now around one year away from the actual test date for these SATs Papers release.

29. Finally, all “items” are amended. Any questions that are still not quite ready are deferred to a later year to be part of another sampling round.

The final countdown – the last stage of SATs paper creation

30. The six month countdown to the test date is now underway. Using the combined insight from sampling, expert review groups, and other stages in the test development process, the STA put together a final suite of SATs papers.

31. There is a lot of statistical analysis that goes into creating the final suites of papers. This is to make sure that they:

• Assess the full range of the test framework
• Have the correct breakdown of content areas (as given in the test framework)
• That the demand from the different cognitive domains are fair

32. Rather than creating three SATs Papers, the STA actually create six (so two suites). One suite is kept as a backup, in case of leaks, issues in the testing process, or other emergencies. For each test year, 2 complete suites of papers are created.

33. These completed papers are then taken to a smaller, combined expert review group. Security around this group is, understandably, even tighter. Here, final tweaks and amendments are made to the tests and mark schemes.

Then the complex final sign off process is begins!

34. Once papers have had final sign off, they are securely printed, stored and then when the time comes, are sent out to schools. The backup set is also printed.

35. If there are any leaks, issues or concerns during the lead up to the SATS test date, then the second set of papers would be sent to schools and used. However, this hasn’t happened in a very long time.

So, there you have it. You now have more facts about SATs than you’d ever dreamed about. You can also see the hugely complex journey the SATs papers have gone on before they get to you (and a bit more about what is done to ensure they’re as fair as possible!)

FREE SATS REVISION RESOURCES

As part of the Third Space Learning offer to schools, the personalised Year 6 SATs revision programme of online one to one SATs tutoring can be supplemented by hundreds of free SATs revision resources. The most popular of these include:
– free SATs papers
– free Year 6 maths reasoning questions
– over 250 more free SATs questions

The best way to access these and more is to join the Third Space Learning online maths hub. Registration is free, and if you like the free resources, you can sign up for the premium resources too such as Fluent in Five and Rapid Reasoning.