Summary of Assessment Principles and Practices

Last term the IB published a new document – “Assessment principles and practices – Quality assessments in a digital age”. Below I post my summary and notes on the document.

Introduction

The aim of this document is to explain the principles that the IB has adopted to make sure that assessment is meaningful, fair and in the best interests of the students involved.

All assessments are a balance between conflicting demands and many concerns about testing fail to take this into account. An example is the tension between reducing the assessments burden and the risk of candidates only having one opportunity to show what they can do.

The IB aims to be a holistic programme of study and this should be reflected in the assessments. Decisions should therefore focus on the impact of the overall programme, not just on one subject, discipline or assessments. The IB focusses on what it is important to assess and not what is easy to assess.

Assessment is all about balancing conflicting and competing demands. Poor quality asssessments will lead to poor quality outcomes, even if assessment is “only” formative.

Assessment principles are what the IB thinks are important in creating qualifications and assessments. They come from what is considered important about an IB education. Assessments should support education, not distort it.

Assessment practices are the ways in which the principles are delivered in a meangingful and practical way.

IB exams should represent an opportunity for candidates to show what they understand, rather than being a unique experience which they need to master. Technology therefore should be driven by the assessment needs and not the other way around.

The IB is exploring eAssessment as a way to ensure that students are able to genuinely demonstrate what they know. In this way, assessments will seek to utilise technology where it may make the assessment more authentic. The IB has already moved to eMarking for many DP components and will seek to implement eAssessment in the DP, although it aware of the risks:

  1. Burdens on schools
  2. Risk of failure
  3. Security
  4. Tech for the sake of Tech
  5. Bias against certain groups of students and device effects
  6. Changing standards
  7. Barriers to schools offering IB programs

Assessment

Assessment can mean many different things but generally is divided into summative and formative although today there is a drive towards “assessment as learning”. Assessments need to be designed carefully to meet the purposes its results are used for. Excellent formative assessments may be poor summative assessments (See here).

Assessment can mean any of the different ways in which student achievement can be gathered and evaluated and there are different assessment models, namely the compensation model and mastery model. The mastery model requires a basic minimum in all criteria to be met, whilst the compensation model will allow poor performance in one criteria to be balanced by very good performance in another criteria.

Formative assessments aim to provide detailed feedback to teachers and students about the students strengths and weaknesses. In contrast summative assessments focus on measuring what the student can do at a specific time. Summative assessment seeks to make a judgement about a candidate, not inform future teaching and learning.

The balance between measuring achievement and identifying correctly what still needs to develop is called assessment validity. It is important to note that the balance between quality of feedback and attainment is opposite in formative vs summative assessment.

Different national systems have adopted different approaches to assessment and these reflect the tensions between the wider aims of the society, the time and resources available. There is no one size fits all or perfectly optimal assessment system. Additionally summative assessment is increasingly being used to analyse teaching quality.

The backwash effect is the influence that an assessment has on the teaching of the content. This can be positive or negative. Snyder’s hidden curriculum is the meaning that students create about a discipline based on this assessment tasks. Assessment needs to be designed around constructivist learning theory.

Marks and grades are not the same thing. Marks refer to credit given to a candidate in line with a mark scheme, and has no other meaning. Grades describe the quality of a candidates work.

Generally IB assessments are not norm referenced. The IB generally uses marks as an indication of overall performance and then looks at how well candidates with x marks performed matched to the grade level descriptors. They then place boundaries based on the descriptors.

Validity means asking if an assessment is fit for purpose. The IB’s first concern is whether the programme is valid, then whether the elements of the programme are valid and finally whether assessments are valid. Validity is not an objective concept and is a balance between competing issues. We cannot prove validity but can construct a validity argument based on evidence.

eAssessment offers new opportunities for interaction within exams and therefore improves the validity of some aspects of assessment. It also removes some of the security concerns while introducing others.

Validity chains can be used to think about validity. There are five elements to the chain:

  • Reliability
  • Construct relevance
  • Manageability
  • Fairness
  • Comparability

All of the above are necessary to achieve validity but there are also tensions between each of them. The IB places the most value in construct relevance – assessments that actually test what they intend to test, but not at the exclusion of all else.

Reliability is the extent to which a candidate would get the same test result if the testing procedure was repeated. There are several sources of unreliability. Consistent outcomes are not the same as the right outcome. The aim of marking reliability is to ensure that all examiners make the same judgement as the senior examiner.

Construct relevance is concerned with accurately measuring the thing that the assessment is attempting to measure.

Manageability can be discussed in terms of the candidate, the school and the IB.

Fairness and bias is concerned with ensuring that the test does not give an advantage to one group over another. Bias can arise from:

  • The delivery of the assessment
  • Bias arising from marking
  • Bias related to assessment questions

Comparability of assessment is concerned with how the grades from assessments can be compared between years or subjects. The IB seeks to maintain three principles about comparability:

  1. The standard of work to achieve grades within a subject or discipline is comparable between years.
  2. Grades between subjects have a consistent meaning so that different routes to achieve the program award are comparable.
  3. Although the IB aims to focus on higher order skills, IB assessments are broadly comparable with similar exams offered by individual nations or other awarding bodies.

IB’s approach to validity

The IB believes that construct relevance and authentic assessment are more important than maximising reliability. The IB believes in rounded, holistic education. Its priority is for strong arguments of validity at programme level. Validity is a complex and multi-faceted balancing act and there is not single right answer, where you place the balance is ultimately a judgement based on the value of the organisation that is developing the assessments. The IB aims to do more than other curricula by developing inquiring, knowledgable and caring young people who are motivated to succeed.  We need to consider how the aims of individual subjects fit into the holistic aims of the IB.

IB assessments are weakly criterion referenced. That is candidate performance is matched against behavioural descriptors.

Comparative marking represents an alternative to traditional marking. The basis is that the human mind is better at making comparisons than absolute judgements. In these examples examiners make win/lose comparisons between pieces of work. In subjective marking mathematics, combined with an importance statement will allow a team of examiners to compare and “mark” students work. However comparative judgement requires many marking decisions because each piece of work must be looked at several times by several examiners.

For the IB the underlying principle is to test what is important and assessments should encourage good teaching. Comments on summative work are used to support examiner marking. Comments on formative work are give feedback to learners.

IB programme-specific processes

Key elements that link all IB programmes are:

  • The learner profile
  • Approaches to teaching and learning
  • international mindedness

IB programmes are conceptual, that is, they focus on powerful organizing ideas that are relevant across subject areas and that help to integrate learning and add coherence to the curriculum. We need to consider how are assessments within each program meet the broader objectives of the DP.

ATLs

The IB recognises the need for schools and individual teachers to have the space to be creative and the ATLs are suggested as guidance and to help highlight good practice and enable discussion. The ATLs are not meant to be prescriptive. Skills can only be improved over time and if taught in a sustained fashion.

Notes

There is definitely some new information here that I am happy to receive. I was not aware that teachers could be observers at grade awarding meetings or at the final awarding committee, and this is something that I would definitely pursue in the future or recommend my teachers do. I was also pleased to see the section on ATLs and the nod from the IB that these are not mean’t to be prescriptive and that schools and teachers should be free to be creative.

Towards a strategy: mission, vision and data

This term I am starting my National Professional Qualification for Senior Leadership (NPQSL) and last Saturday travelled to Beijing to attend the first face to face session on succeeding in senior leadership.

The pre-session activities had been focussed on culture, mission and vision statements. Prior to this, I had thought of these statements as being nebulous and having no real bearing on the real business of schools. To be fair in the majority of the schools I have worked in, these are not things that have been referenced or referred to in the daily working of teachers. In fact, it is only in one school which I was involved in setting up where I can remember a meeting discussing these things. We were asked, in a meeting, to suggest ideas for our mission and vision statements and then we never referred to them again. I can remember one close friend being frustrated because no one he was working with seemed to understand what the mission and vision statements of a school were for or how they might be used. Looking back, I question whether any of the leaders I had worked for in the past really new what these things were.

The session reading began to shine a new light on these ideas for me and I summarise my notes on what I read at the end of this blog post.

In essence, a vision statement should be future orientated and formulate what the community is working towards in the long-term. I write community because whilst a school will have a vision statement, I think it is important they groups within and above the level of the whole school should also pay attention to what their vision is. If this is right I should think about what the vision is for KS5 and this vision should be linked to my schools vision as well. Vision statements should be reviewed on a regular basis in consultation with the people who work within those communities.

Mission statements are focussed on the present and the action that a community is taking to bring about that vision. Like a vision statement this mission statement will need to be reviewed regularly with input from all community members.

Both the mission and the vision of the community should influence decisions that are made in the daily “workground” and should be a lens through which policy is produced, disseminated and questioned. One of the colleagues I met had started as HOD for in a new department. They had placed a board up with titles of areas that they didn’t know about and asked their team members to write down things that they thought the HOD needed to know. From this they formulated a vision and mission for the department which is the lens by which they examine actions in department meetings.

These statements need to carefully formulated so that they can be used and communicated appropriately. Aside from their uses as described above we need to think about how they are communicated with the parental community and how they can be used to work with the parental community. There was the suggestion that we can interview parents when students are interviewed to carefully explain our vision and values in one to one situations.

When vision, mission and value statements are combined with data analysis they can be the backbone for creating a strategic plan to allow the school or community to move forward. Senior leadership has a focus on vision creating. Compared to middle leadership who have a specific team focus, and will be implementing that vision, senior leaders need to think broadly.

At this stage I think that identifying my moral purpose and the vision for my area is absolutely paramount. Both on an individual level and for the IB DP at my school. My thesis and the schools thesis need to be synthesised together for the IBDP and the UGO office.

Pre-Reading Notes

Moral Purpose – Michael Fullan

Five attributes of leadership; moral purpose, understanding change process, strong relationships, knowledge adding and coherence among multiple priorities. Moral purpose is both ends and means but we have mixed motives and that is fine. Moral purpose doesn’t exist in a vacuum. It interacts with the other components of leadership. Two case studies illustrate this: the national literacy and numeracy strategy in England and Monsanto. For leadership to be effective it must have 1. An explicit making a difference purpose. 2. Use strategies that mobilise many people to tackle tough problems 3 be held accountable by measured indicators of success 4 mobilise others sense of moral purpose Acting with moral purpose in a complex world can have insidious consequences. coherence making can be guided by moral purpose. Moral purpose must contend with diversity. Difference in morality from different cultures

Moral leadership by John West Burnham

Behaviour which is consistent with personal and organisational values which in turn are derived from a coherent ethical system. Decisions in education are naturally ethical decisions so leaders need to be ethically literate. The trad vs progressive debate. Professions have ethical purpose. Communities have a consensus over the values that they think are important. Leadership needs to secure this agreement. Culture is the expression of and reinforcement of the ethics of a community. Leaders exemplify what that society and community most value. Moral leadership can be taught through engagement with the meta narratives of ethical classics, reflection in action, coaching and networking.

Vision, strategy and planning in education by Mark Brundrett

To be effective agents of change, leaders in schools need a clear idea of some end goal. Vision is focussed on some goal to be achieved in the future. Leaders need to conceptualise and articulate a vision to influence followers through five mechanisms: 1 giving direction and purpose 2 organise action around future goals 3 provides a sense of identity and meaning 4 provides a common framework 5 vision may develop organisational norms. Personal visions possibly arise from a variety of different causes. Vision must be both individual and collective. Visions in schools and organisations will only succeed if they are bought into by the staff at all levels. Vision and strategy need to be linked at the outset and on an ongoing basis. If strategy is about achieving goals then it is inseparable from vision. Good strategy involves the organisation as a whole just like the vision. Strategies must encourage the involvement of as many people as possible so that they have a sense of ownership. Strategy must take account of long-term intentions and aspirations, the external environment, the internal strengths of the organisation, the prevailing organisational culture, expectations of stakeholders, future resources. Priorities change in education so it can be difficult to plot a straight line approach to strategy and planning. Four abcd approach to translating strategy into action: articulate, build (images metaphors experience) create (dialogues cognitive) define (formal plans)
Targets can help move things forward. School leaders need to have a vision of the school they wish to create. This will be personal but will accord with the aspirations of the wider community. Vision needs shared ownership.

What makes a school a learning organisation – OECD report

The model focuses on:

  • Developing a vision centered on the learning of the students
  • Creating and supporting continuous learning opportunities for staff
  • Promoting team learning and collaboration among all staff
  • Establishing a culture of inquiry, innovation and exploration
  • Embedding systems for collecting and exchanging knowledge and learning
  • Learning with and from the external environment and larger system
  • Modelling and growing learning leadership

Trust time technology and thinking are the four T’s that underpin the above elements. Developing a shared vision is the result of a process that involves all stakeholders. We need integrate all including those on the margins of society as alienation poses a threat to democracy. The vision needs to aim to enhance the lives of all learners. Excellence and equity are not mutually exclusive goals. Some countries have managed to successfully improve outcomes of the most disadvantaged. (Disadvantaged is often implicitly financial but also language as in EAL learners in a globalised society.)

Setting direction vision and values

Generating culture is a high priority for leaders and the concept has a synergy with vision and value statements. Values statements are concerned with behaviour. It can be used as a lens to view decision-making and the formulation of policy. Mission statement are concerned with the present and present actions. It is about what the organisation does or is attempting to do. The vision statement is about the future and linked to ideas about future goals. These statements need to be actualised and this is achieved through culture as shown in Scheins model (see picture). Culture will make or break a school and changing culture is time-consuming. All of the statements above, like all good plans, should be time limited. Changing culture involved conjoining content and process. They are always a work in progress. Various models can be used to implement culture change. This eight step model suggests:

  1. Establish a sense of urgency. Don’t change the culture if the arguments for changing it have not been convincing.
  2. Form a powerful guiding coalition
  3. Create a vision
  4. Communicate the vision. Instead of the presentation use listening and questioning during conversations to broker by in. (Turning statements into questions could be a good way to build relationships with peeps)
  5. Empowering others
    Coaching here through conversation is paramount
  6. Planning for short-term wins
  7. Consolidating improvements and producing more change
  8. Institutionalising new approaches

Reflections: first term in senior leadership

In August 2018 I started a new position as the IBDP Coordinator at a school that had recently adopted the IBDP to replace its A Level program. This was my first time in a senior leadership position and I want to take sometime to reflect on that first term.

Prior to this role I had worked as part of the founding team of a secondary school. During that time we took the school through IB MYP and DP authorization, CIS accreditation, where I planned resourcing of the science department, as well as planned and developed the university and careers program. It was a busy time, and I learned a lot about things that I would not have got exposure to if I had stayed at my previous school. Through observation, I spent a lot of time watching and reflecting on the actions of various senior leaders.

So, naturally, entering into senior leadership for the first time, I felt that I had some ideas of do’s and dont’s of leadership, to help guide me in my initial steps. I also felt that I knew some of my natural weaknesses that I needed to work on. Looking back I think the biggest lesson I learned at my previous work was an echo of a prior lesson that I learned in my first year of teaching: It is ok to not have all the answers. It is ok to admit to knowledge gaps.

Starting in a new school always brings new challenges, and this term was no exception. It was odd, being new, being in leadership, and feeling like I was expected to have answers in the first week about processes and systems that I was still just getting my own head around. In addition to picking up new classes (I am still in the classroom 10 hours a week) one of which was a Y13 biology class, and having to learn the ropes of IT systems I hadn’t used before as well as the expectations of policy and procedure, I was implicitly and explicitly asked by staff about various points of procedure which I just didn’t know how the school did it. I had my own ideas of how things should be done, but I certainly did not want to impose those from the start.

In my first week, I was, figuratively, thrown under the bus being required to present to the whole primary and secondary school about priorities for KS5 and the rest of the secondary school. I also was front and center, leading multiple sessions during the staff inset. During all of these sessions I was careful to thank those who had put the work in before my arrival and show humility in the way I talked about the DP and my ideas. I was also happy to admit that this was my first DP Coordinator post. I think that this went some way to helping me build relationships with staff members.

On top of planning and marking (It was the first time I was teaching IGCSE biology in 4 years) during that first term my time was consumed mostly by:

  • Managing teachers
  • Developing policy
  • Developing exams processes and procedures
  • Developing the DP options process

Before becoming a senior leader I thought that most of my time would be taken up with managing student behaviour across the board but this term I was surprised to find just how much time is taken up by teachers! 🙂 Not all of this is good or bad, much of it is healthy relationship building, and I had already clocked in my last job that this was an area I needed to work on: relationship building.

I like to work on my own, not surrounded by other colleagues to talk to or distract me. I find planning and prioritisation hard when my time is broken up, and I like long chunks of uninterrupted time to focus on my work. In my last job I had already told myself that I needed to get into the staff room more but this term this has been an imperative. Why send an email if I can have a conversation? I have come to understand that this is so important for a senior leader and as I write I am reminded of an old Head who always used to admonish me to come and see her and not send emails. However, I currently believe that this is a responsibility of senior leadership not teachers. At the time I was teaching four classes of biology, two classes of TOK, running the Duke of Edinburgh’s Award and setting up the careers program. I didn’t have time to try and find people (in fact a lot of that year I spent in tears). This Head should have been coming to find me, instead of showing a profound lack of understanding about what I was doing with my time.

As DP Coordinator I need to be aware of the pressures my teachers are under and get out there to support them. However, when issues between staff arise, this can take up a huge amount of temporal and emotional space as I learned this year. I have also seen how destructive certain habits can be for individuals.

Developing policy has been a key focus for the term. As the DP program is quite young, there are a lot of policies and procedures that need attention. Initially we have been focussed on developing academic honesty policy and I wrote about that here and here.

Since October, my time has mostly been absorbed by planning the administration of the mock exams to be held in January (getting the dates in place is another story) and developing a process for the DP options/subject choices procedure for year 11.

I also had the importance underscored, more than ever, of working in a team and holding the party line. When decisions are made as a leadership team that might be unpopular and that you are either personally neutral to or don’t really agree with, it is really important to support the party line. Not doing so, can serve to undermine the effectiveness and efficiency of that team. While it may be tempting to admit privately that you disagree with a decision or that the decision wasn’t yours, the result of doing so will not be positive.

Developing a school wide Academic Honesty Policy II

In November I shared the first stages of my thinking about developing our academic honesty policy. In this post I want to follow-up, to document the most recent steps.

Last Monday, while trying to find my mind, which I appeared to have left somewhere between the UK and China, I lead the second inset session on academic honesty. Running inset when you are jet lagged isn’t fun – especially when all the team is equally as tired!

During this session, I followed a very similar model to October’s inset, using chalk talk as a way to elicit thoughts and ideas about the academic honesty policy.

I am keen not to simply impose my ideas about academic honesty on the teaching body but to encourage by in, I want to grow a policy as a team. It may seem like this is a little esoteric, but having a shared understanding of the why’s, what’s and how’s of teaching academic honesty is a really important part of what we do as educators. Understanding the issues of good practice in this area impacts on many other aspects of classroom practice and should engender a change in the way that we approach planning of units, lessons and tasks.

We started the session with a Quizlet live game of academic honesty terminology before moving on to practice the chalk talk again. I was inspired by my Christmas reading of Martin Robinson’s Trivium 21C and so started with the question:

–What is more important:

–developing students knowledge of facts or ability to critique and discuss facts?

Following from that we rotated around the three questions of:

  1. Why teach academic honesty
  2. How teach academic honesty
  3. When teach academic honesty

Finally, the team was asked to present the discussion points and ideas on the question sheets that they started with to the rest of the group.

Before ending the session with a call for volunteers to join a working party, I shared the results from the staff citation survey that I conducted in November. These results certainly gave me some food for thought as there were some good arguments presented for not having a centralised citation policy. Most staff thought it would be a good idea to have a centralised policy but the arguments against it were that essential that students should be exposed to a variety of different ways, no one way is right, students joining the school may have learned different systems and should be allowed to demonstrate that learning. All valid points and I must have admit shifted my thinking on this one a little.

Another advantage of decentralising the policy could be that departments take responsibility for agreeing a policy together and therefore think about and implement a procedure that is useful for them. Considering that only one member of staff has taken me up on my offer to form a working party to develop a policy, this could be another avenue for ensuring buy in to the new policy. I may ask HODs to formulate a policy and to let me know what system they are going to teach to ensure that we avoid the attitude of “let some other department deal with it”.

Now I need to think about how to take this forward so that we can launch a policy next August. I have now had staff contribute to the triage of where the school is at. Now we need to decide what to put in the policy and write it and I need to think of how best to achieve this, if teachers don’t volunteer to join the working group?

Notes on Trivium 21c

Martin Robinson’s Trivium 21c was an absolute delight to read. Thought provoking and enlightening it presents an eloquently articulated history of the educational ideas and, through this radical history, a persuasive argument for the great synthesis of traditional and progressive teaching methods, united via the ancient arts of the trivium.

The trivium in teaching

The trivium of grammar, dialectic and rhetoric, originally used as the basis of the school curriculum in the middle ages are explored from a variety of different paradigms and meanings for Robinson to final expound his view of how this mantra could be utilised in modern schooling.

Grammar, Robinson argues, is not only the teaching of the rules of the language arts but is equivalent to the transmission of all of the basic facts and building blocks of knowledge that make up a particular discipline or subject. Grammar is also the transmission of the structure and rules of culture, which, of course, encompasses all the academic disciplines as well as other elements. To Robinson, modern grammarians are those of us on the traditionalist side of the great education debate on methods. Grammarians value knowledge and the methods best shown to help students gain this knowledge.

Dialectic, to Robinson, is the art of critiscm, skeptiscim and questioning of grammar. It is an art that needs to be taught in order to enable future adults to be able to think clearly about and with knowledge, in order to not simply be absorbers of knowledge but users and producers of it. Dialectic is important as it allows students to manipulate and use the knowledge acquired through grammar, by questioning it, reflecting on it and potentially rejecting or changing it. If grammar represents tradition, then dialectic represents progression; the dialecticians are those of us in education who aspire to the more progressive methods in the great educational debate.

The third art of the trivium, rhetoric, is the art of communication. Not only should learners be taught to acquire knowledge through grammar, taught to question it through dialectics but they should also be taught to communicate their thoughts through the arts of rhetoric.

Should the purpose of education serve the common good or enable someone to live a good life?

As someone who has moved from being deeply religious to being so no longer, I found myself agreeing with Robinson’s sentiments that curiosity is not best served by prejudice and that teachers must not model the closed mind of someone who thinks there is only one path that leads to meaning or, I guess, truth. In this vein he asks us to attempt to live, as teachers, with the uncertain position of holding the traditional and the progressive together, investigating ideas from across the range of opinion.

Robinson asks if all teachers in any given school understand the narrative of a the curriculum? He argues that only by seeing how their part fits into the wider curriculum can teachers deliver an education to students that allows them to be knowledgable, critical and reflective. He claims that students must learn the unifying concepts, the concepts that come up again and again, of each discipline again and again. This put me in mind of Thomas Khun who claimed that expertise as a scientist only arrives through exposure to many examples. Scientists are experts because they have been able to generalise from the many specific examples and they apply this knowledge in new scenarios.

Robinson also claims that teachers must move away from omniscience, which reminded me of an early career conversation a chemistry teacher who claimed that not knowing in a teacher is a sign of weakness, and that students don’t like it. I agree with Robinson that all teachers need to honest about what society doesn’t know, they need to embrace the uncertainty in their discipline.

A good teacher has mastered the core knowledge and more of their discipline as well as holding an appreciation of what society doesn’t yet know in their field.

We need to understand that teachers should have the authority to teach but recognise that all knowledge is probable and uncertain.

Teachers should use language in such a way that ensures uncertainty has its place.

Robinson’s book draws on many sources and aside from his main argument is highly informative of the history of educational ideas. His arguments are compelling and interesting but the book is worth reading not only for this but also for your education in the history of educational ideas that it draws upon. This book has helped me begin to see the synthesis of the progressive and traditional narratives and has got me wondering about how I can go about making argumentation an important part of my biology classroom in the second stage of the trivium. How can I use debate to really challenge kids to think and learn all sides of an argument? How can I introduce students to the, Dissoi Logoi in science, the art of seeing both sides of an argument as true within their contexts. Instead of dialectic argument as being right vs wrong we can make both as right. Dual thinking explores the possibility that both sides can be right.