The role of curriculum

In the second NPQSL face to face session we looked at leading the quality of teaching and learning within a school. We were asked to think about what high quality teaching and learning looks like in our schools and what this means to us personally. This provided some good reflection time for my own thinking about this means for me. I concluded that high quality teaching and learning is where students are forced into thinking about the topics of the subject under discussion. Thinking takes variety of forms. For me and my project, focused on implementing the DP, TOK is the key to horizontal collaboration within the DP programme, catalyzing not only a change in the way that student think but also how teachers think. Going forward I need to Establish a working group of teachers who are interested in improving their links to TOK.

At the start of the session on “driving the quality of teaching and learning” we were asked to list three priorities with regard to the quality of teaching and learning. Mine were:

  1. Making thinking the basis of both
  2. Developing good knowledge of the whole curriculum (Martin Robinson’s story)
  3. Developing knowledge of good practice – can the teacher make reasoned judgements about why they do what they do.

We then considered learning centered leadership: – how do we model, monitor and have dialogue. My group felt that it was important for leaders to be:

  1. Modelling preparedness, calm, openness and friendliness
  2. Still teaching?
  3. Using data
  4. Observations
  5. Conversations
  6. Diagnostic audit of peoples and there skills

Next we were asked to list ten ingredients for great teaching and to discuss why leaders may want to observe lessons, what the purposes of lesson observation were. My ingredients for great teaching were:

  1. Dialog
  2. Content knowledge
  3. Pedagogical content knowledge
  4. Evidence for teaching practice
  5. Prior knowledge
  6. Contextual – relevance for kids
  7. Focussed on concepts
  8. Timing – knowing when an intervention is appropriate or not
  9. Collaborative – outside the silo
  10. Firm friendliness

I also felt that observation is a great way to learn and be coached and time for teachers to observe each other is valuable if we want to enable coaching, mentoring and further development.

After sharing these within our groups we had to decide on the groups final five. We had a lot of good discussion about how learning is often confused with performance and other proxies, and that learning is actually quite a hard thing to actually observe in a lesson. Any attempt to observe a lesson for accountability purposes was doomed if you are hoping to look measure learning. Instead my group agreed that the best we could hope for was to look for proxies that may indicate high quality teaching. My group decided that our priorities were to look for :

  1. Positive relationships
  2. Feedback
  3. Knowing the students
  4. Knowledge of content and pedagogy
  5. High Expectations

I reflected that evidence is a key thing here: Knowledge in education is so tentative and unsure that no one can say with certainty this is right, or this way is wrong. Thus if we focus on the thinking behind what teachers are doing and why – are teachers able to engage with discussions and evidence why they are doing somethingt. To ensure great teaching I think it is important for leaders to smile, be open and approachable. We need to encourage discussion between teachers about their practice, provide opportunity for observation between teachers and focus more on teaching and learning, instead of getting drag into secondary tasks.

Going forward I need to work to facilitate this in my community and help to provide opportunity for this to happen, time for teachers to observe each other and time for them to have discussions with a view to improving the quality of teaching within the school. I need to support a focus on developing an understanding of the links between the subjects – horizontally and vertically – and encourage teachers to come out of the their silo.

How might this session influence your staff professional development policy?

How can you measure the impact of CPD? Carry out observations of trying out TOK activities, carry out a staff survey, have the CPD, start the written curriculum and then observe more activities and carry out an additional survey. Invite staff to take part in TOK and ATL collaboration.

Side effects in Education

Recently, in my NPQSL course we have been asked to reflect on the question “What is the moral purpose of education?” Education can be argued to have many moral purposes, and it comes down to an educators point of view; this is an opinion that I think many teachers and leaders would accept.

For example you could argue that the moral purpose of education is to allow individuals to experience a fulfilled life where they can experience and appreciate the whole of their humanity. You could also argue that education’s purpose is to serve society and better the community at large.

Where ever you stand on this spectrum, the very fact that there is a difference of opinion here makes education, as a profession, a little unique. Doctors, for example, would largely agree that the moral purpose of education is to save life.

In what works can hurt, Yong Zhao asks if educational research should be concerned with side effects in education. However appealing this analogy is misleading. In medicine there is a clear moral purpose: do no harm. This is a moral purpose that all medics subscribe too. Medics are driven by the desire to save and prolong life.

No single unifying moral principle exists in education and different schools and different teachers have different moral purposes.

Yong Zhao cites the medical profession as one that requires researchers to investigate side effects as well as the main affects of interventions. In light of calls for educational research to adopt similar methodologies to medical research and become more scientific he argues that this is an area that is overlooked.

Educational research is exclusively focussed on what works without looking at how much it hurts.

Reasons for this may be that education is universally perceived as good, although I would argue that medicine is also. I think the reasons that the education does not consider side effects so much is that the moral purpose of education is much less clear. As well as, this damage due to eduction may take a very long time to be observable and you can only measure that which can be observed – also there are huge numbers of conflating variables.

Zhao writes that education is dominated by a narrow focus on cognitive abilities derived in a small number of subjects measured by standardized tests so that scores in these tests become the measure of effectiveness. Other outcomes are rarely measured so we don’t know about any adverse effects.

More evidence is unlikely to stop the battles within education, but a consideration of side effects might. The education pendulum swings but there is really no progress. I can agree with some of this as any look at the history of the debates does see that these arguments do go on quite a way back.

A way forward to resolving the traditional/progressive debate may be the consideration of both main and side effects in education interventions.

Zhao highlights that direct instruction is effective but can stifle creativity and reduce confidence. He cites the progress of some Asian countries, where students have a lot of knowledge drilled into them but students suffer from lack of confidence, versus Western countries where students know less but have more confidence, as evidence that what works can hurt.  I wonder if this may be a relic of the Dunning-Kruger effect. Student may well be further along the knowledge curve and therefore less confident.

Many interventions that have sought to improve reading scores have reduced access to other subject areas, by eliminating subjects to make more time for reading prep. The negative effects of these interventions are now well documented: reducing access to other subjects only serves to reduce literacy scores.

I do think that by focussing only on what can be measured can lead us down the wrong path. Measurement is important and does have a place, but there are elements of human life that we don’t know how to measure or have barriers to measuring like cost and time. We shouldn’t ignore these areas.

Developing a progression model for IBDP biology

I recently completed Daisy Christodolou’s “Making good progress?”. You can see my notes here. In the final chapters, after presenting an argument building up to this, she outlines the key aspects of what she terms a “progression model”. In this post I want to line up some ideas about what this may look like in delivering the IB DP Biology course.

In her book Christodoulou suggests, and I agree, that to effectively help students make progress we have to break down the skills required to be successful in the final assessments into sub-skills and practice these. This is a bit analogous to a football team practicing dribbling, striking or defending in order to make progress in the main game.

In the book she also stresses the difference between formative and summative assessments, what they can and can’t be used for respectively and why one assessment can’t necessarily be used for both.

A progression model for biology

A progression model would clearly map out how to get from the start to the finish of any given course, and make progress in mastering the skills and concepts associated with that domain. In order to do this we need to think carefully about:

  1. What are the key skills being assessed in the final summative tasks (don’t forget that language or maths skills might be a large component of this)?
  2. What sub-components make up these skills?
  3. What tasks can be designed to appropriately formatively assess the development of these sub-skills or, in other words, What does deliberate practice look like in biology?
  4. What would be our formative item bank?
  5. What could be our standardised assessment bank?
  6. What are appropriate summative assessment tasks throughout that would allow us to measure progress throughout the course?
  7. What could be our summative item bank?
  8. How often should progress to the final summative task be measured i.e. how often should we set summative assessments in an academic year that track progress?

Key skills in biology

This is quite a tricky concept to pin down in biology specifically and in the sciences in general. What skills exactly are kids being assessed on in those final summative IGCSE or IBDP/A Level exams. I haven’t done a thorough literature review here so currently I am not sure what previous work has been in this area.

However,  I would contend that most final written summative exams are assessing students conceptual understanding of the domain. If this is the case then the skill is really, thinking and understanding about and with the material of the domain. Students who have a deeper understanding of the links between concepts are likely to do better.

In addition, those courses with a practical component, like the IBDP group 4 internal assessment are assessing a students understanding of the scientific process. While it may seem like these components are assessing practical skills per se, they only do this indirectly, as it is the actual written report that is assessed and moderated. To do well the student is actually demonstrating an understanding of the process, regardless of where their practical skills are in terms of development.

Indeed if we look at the assessment objectives of IBDP biology we see that this is very much the case. Students are assessed on their ability to: demonstrate knowledge and understanding and apply that understanding of facts, concepts and terminology; methodologies and communication in science etc.

Sub-skills

How can we move students to a place where they can competently demonstrate knowledge and understanding, apply that understanding as well as formulate, analyse and evaluate aspects of the scientific method and communication.

The literature on the psychology of learning would suggest breaking down these skills into their subcomponents. This means we need to look at methods that develop knowledge and understanding from knowledge. Organising our units in ways that help students see the bigger concepts and connections between concepts within the domain will also help. For more on this see my previous post here. I think that understanding develops from knowledge.

I recently read that Thomas Khun claimed that expertise in science was achieved by the studying of exemplars. Scientific experts are experts because they have learned to draw the general concepts of the specific examples.

Useful sub-skills would be:

  • Fluency with the terminology of the domain
  • Ability to read graphs and data
  • Explicit knowledge of very specific examples
  • Explicit knowledge of abstract concepts illustrated by the specific examples
  • Ability to generate hypothesis and construct controlled experiments

Deliberate practice in biology

Thinking about these sub-skills, then, we can see what may constitute deliberate practice in biology and thus what would make useful formative assessments within the subject.

Fluency with the terminology can be gained through the studying of terminology decks like those available on quizlet. In addition, the work of Isabel Beck. Suggests that learning words isolated from text is not that helpful to gaining an understanding of those terms. To gain this, students need to be exposed to these words in context. Therefore there is a lot to be said for tasks and formative assessments that get students reading. Formative assessments could then consist of vocab tests and reading comprehension exercises of selected texts.

Reading and interpreting data can be improved through practice of these skills. This is an area where inquiry alone won’t help students make progress. Students need to be shown how to interpret data and read tables and graphs before making judgements. Ideally, in my opinion they should do this once they have learned the relevant factual knowledge of a related topic. Formative assessments focussing on data interpretation should therefore come a little later once students have covered a bulk of the content.

To build up conceptual understanding, students need to be exposed to specific examples related to those topics as I outlined in this post. Tests (MCQs) that assess how well students know the specific details of an example could be useful here to guide learners to which parts they know and those they don’t.

Following this we can begin to link examples together to build knowledge of a more abstract concept. Concepts can then be knitted together to develop the domain specific thinking skills: thinking like a biologist.

Formative assessments

Formative assessments could take the form of MCQs but as outlined above, vocab tests, reading comprehension activities, and other tasks may well have their place here.

Summative assessments for measuring progress

I am now thinking that to truly assess student progress against the domain, individual unit tests just won’t cut it. As Christodolou argues, summative tests exists to create shared meaning and do that need to be valid and reliable. Does scoring a 7 in a unit test on one topic of an 11 topic syllabus mean that the student is on track to score a 7? Not necessarily. Not only is the unit test not comparable to the IB 7 because it is only sampling a tiny portion of the full domain, but the construction and administration of the test may not be as rigorous as that of the actual IB papers.

Clearly it isn’t ideal to use the formative assessments described above as these are nothing like the final summative assessment of the course, plus their purpose is to guide teaching and learning, not to measure progress.

I would argue that summative assessments over the two-year course should use entire past papers. These past papers sample the entire domain of the course and performance against them is the best method of progress in the domain. A past paper could be administered right at the start of the course to establish a base line. Subsequent, infrequent, summative tests, also composed of past papers could then measure progress against this baseline.

Why should summative assessments use past papers? What not use unit tests? Unit tests, aggregated, is not the same thing as performance on a single assessment sampling the whole domain. They cannot produce the same shared meaning as an assessment that samples the entire domain. In addition the use of many single unit, high stakes tests will cause teaching to the test as well as much more student anxiety. Instead lots of formative testing and practice of recall should help to build students confidence in themselves.

Notes on making good progress?: Chapter 9

In this series of posts I record my notes from Daisy Christodolou’s book “Making good progress? The future of Assessment for Learning” It is quite excellent. You can buy a copy here.

An integrated assessment system

An accurate and useful progression model is the foundation of any assessment system because it explains how pupils make progress. SoWs and lesson plans and the curriculum can all be based on this progression model. Textbooks would be the most efficient way to bring all of these items together.

The first item in a progression model is a collection of formative questions which match up to the curriculum. The bank should be online so that pupils can access it at home or at school. Pupils should have formatives at the end of lessons or chapters and these would contain questions on new material and also on material previously taught. It could be linked up to question level analysis and if it was automated could point pupils straight back to a relevant video or worksheet on what the students had just got wrong.

Next we should have summative item banks. For difficulty model assessments this could be whole past papers or if online a computer adaptive system that makes testing shorter and more accurate. For quality based model we could use a comparative judgement system.

Finally a standardised test bank would contain be non-curriculum linked questions like the CEM tests. These are useful for setting targets.

Such a system would be beneficial because it would be coherent: a clear structure of progressing to novice to expert. It has the ability to give pupils ownership of the curriculum. It could also be self improving.

Notes on making good progress?: Chapter 8

In this series of posts I record my notes from Daisy Christodolou’s book “Making good progress? The future of Assessment for Learning” It is quite excellent. You can buy a copy here.

Improving summative assessments

The aim of summative assessments is for them to provide an accurate and shared meaning without becoming the model for every classroom activity. Rubrics of prose assessment statements are not particularly good at delivering reliability, and they can end up compromising the creative and original aspects of the task. Prose descriptors can be interpreted in many different ways. Judging in absolute terms is extremely difficult. Markers will overgrade and undergrade depending on the sample. We are much better at making comparative judgements than absolute ones.

very prescriptive rubrics end up stereotyping pupil’s responses to the task removing the justification for having them (grading creative and original work). Responses that are coached to meet the rubric pass and truly original work that doesn’t fails. Rubrics encourage coaching.

Comparative judgement offers the possibility of dropping rubrics by defining quality through exemplars not prose and by not relying on absolute judgement. It simply asks markers to make a series of paired judgements about responses. It relies on tacit knowledge of the subject expert – knowledge that is not easy to express in words.

Comparative judgement is criticised for offering little in the way of formative feedback. This is precisely the point. It decouples the grading process from the formative process. It allows classroom practice to be refocussed away from the rubric and towards helpful analyses of quality. One extremely useful resource that could be produced would be a set of annotated exemplar scripts.

Decisions about the difficulty and content of national summative exams are made by national exam boards. What if a school wants to summatively assess more frequently? To what extent can they be linked to the curriculum that the pupils are following? One solution is to outsource summative assessments, but there is still a gap between the remote standardised assessments like CEM and the formative assessments of classroom practice. It is not easy to create or interpret the results of school made curriculum-linked assessments. It can be difficult to tell if the test is difficult enough, or if it has the right spread of difficulty. Tests taken by small numbers of pupils don’t produce reliable grades. We can compare the results of teacher made tests to national assessments. The content studied over one term is simply not broad enough domain to sample from. Assessments have to sample from what pupils have learnt in that subject, not just in previous terms but in previous years.

A summative assessment can be linked to the curriculum and the most recent unit of study. However if a grade is awarded it will not be based solely on that unit and cannot be seen as reflecting performance on solely that unit. A student can make great strides with a unit but not be reflected on the summative unit as the assessment is not sensitive enough.

Summative assessments need to be far enough apart that pupils have the chance to improve on them meaningfully. However pupils will make relatively slow progress on the large domains that summative assessments are sampling. There are risks with using summative assessments too frequently.

Using scaled scores can overcome this to some extent. A scaled score converts raw marks which are not comparable (from different assessments) into ones that are. They show the continuum of achievement. Grades suggest that pupil performance falls into discrete categories when in fact it is continuous.