Category: Teaching & Learning

Categories
Education Teaching & Learning

Side effects in Education

Originally posted on March 22, 2019 @ 6:05 pm

Recently, in my NPQSL course we have been asked to reflect on the question “What is the moral purpose of education?” Education can be argued to have many moral purposes, and it comes down to an educators point of view; this is an opinion that I think many teachers and leaders would accept.

For example you could argue that the moral purpose of education is to allow individuals to experience a fulfilled life where they can experience and appreciate the whole of their humanity. You could also argue that education’s purpose is to serve society and better the community at large.

Where ever you stand on this spectrum, the very fact that there is a difference of opinion here makes education, as a profession, a little unique. Doctors, for example, would largely agree that the moral purpose of education is to save life.

In what works can hurt, Yong Zhao asks if educational research should be concerned with side effects in education. However appealing this analogy is misleading. In medicine there is a clear moral purpose: do no harm. This is a moral purpose that all medics subscribe too. Medics are driven by the desire to save and prolong life.

No single unifying moral principle exists in education and different schools and different teachers have different moral purposes.

Yong Zhao cites the medical profession as one that requires researchers to investigate side effects as well as the main affects of interventions. In light of calls for educational research to adopt similar methodologies to medical research and become more scientific he argues that this is an area that is overlooked.

Educational research is exclusively focussed on what works without looking at how much it hurts.

Reasons for this may be that education is universally perceived as good, although I would argue that medicine is also. I think the reasons that the education does not consider side effects so much is that the moral purpose of education is much less clear. As well as, this damage due to eduction may take a very long time to be observable and you can only measure that which can be observed – also there are huge numbers of conflating variables.

Zhao writes that education is dominated by a narrow focus on cognitive abilities derived in a small number of subjects measured by standardized tests so that scores in these tests become the measure of effectiveness. Other outcomes are rarely measured so we don’t know about any adverse effects.

More evidence is unlikely to stop the battles within education, but a consideration of side effects might. The education pendulum swings but there is really no progress. I can agree with some of this as any look at the history of the debates does see that these arguments do go on quite a way back.

A way forward to resolving the traditional/progressive debate may be the consideration of both main and side effects in education interventions.

Zhao highlights that direct instruction is effective but can stifle creativity and reduce confidence. He cites the progress of some Asian countries, where students have a lot of knowledge drilled into them but students suffer from lack of confidence, versus Western countries where students know less but have more confidence, as evidence that what works can hurt.  I wonder if this may be a relic of the Dunning-Kruger effect. Student may well be further along the knowledge curve and therefore less confident.

Many interventions that have sought to improve reading scores have reduced access to other subject areas, by eliminating subjects to make more time for reading prep. The negative effects of these interventions are now well documented: reducing access to other subjects only serves to reduce literacy scores.

I do think that by focussing only on what can be measured can lead us down the wrong path. Measurement is important and does have a place, but there are elements of human life that we don’t know how to measure or have barriers to measuring like cost and time. We shouldn’t ignore these areas.

Categories
Teaching & Learning

Developing a progression model for IBDP biology

Originally posted on November 24, 2018 @ 4:00 pm

I recently completed Daisy Christodolou’s “Making good progress?”. You can see my notes here. In the final chapters, after presenting an argument building up to this, she outlines the key aspects of what she terms a “progression model”. In this post I want to line up some ideas about what this may look like in delivering the IB DP Biology course.

In her book Christodoulou suggests, and I agree, that to effectively help students make progress we have to break down the skills required to be successful in the final assessments into sub-skills and practice these. This is a bit analogous to a football team practicing dribbling, striking or defending in order to make progress in the main game.

In the book she also stresses the difference between formative and summative assessments, what they can and can’t be used for respectively and why one assessment can’t necessarily be used for both.

A progression model for biology

A progression model would clearly map out how to get from the start to the finish of any given course, and make progress in mastering the skills and concepts associated with that domain. In order to do this we need to think carefully about:

  1. What are the key skills being assessed in the final summative tasks (don’t forget that language or maths skills might be a large component of this)?
  2. What sub-components make up these skills?
  3. What tasks can be designed to appropriately formatively assess the development of these sub-skills or, in other words, What does deliberate practice look like in biology?
  4. What would be our formative item bank?
  5. What could be our standardised assessment bank?
  6. What are appropriate summative assessment tasks throughout that would allow us to measure progress throughout the course?
  7. What could be our summative item bank?
  8. How often should progress to the final summative task be measured i.e. how often should we set summative assessments in an academic year that track progress?

Key skills in biology

This is quite a tricky concept to pin down in biology specifically and in the sciences in general. What skills exactly are kids being assessed on in those final summative IGCSE or IBDP/A Level exams. I haven’t done a thorough literature review here so currently I am not sure what previous work has been in this area.

However,  I would contend that most final written summative exams are assessing students conceptual understanding of the domain. If this is the case then the skill is really, thinking and understanding about and with the material of the domain. Students who have a deeper understanding of the links between concepts are likely to do better.

In addition, those courses with a practical component, like the IBDP group 4 internal assessment are assessing a students understanding of the scientific process. While it may seem like these components are assessing practical skills per se, they only do this indirectly, as it is the actual written report that is assessed and moderated. To do well the student is actually demonstrating an understanding of the process, regardless of where their practical skills are in terms of development.

Indeed if we look at the assessment objectives of IBDP biology we see that this is very much the case. Students are assessed on their ability to: demonstrate knowledge and understanding and apply that understanding of facts, concepts and terminology; methodologies and communication in science etc.

Sub-skills

How can we move students to a place where they can competently demonstrate knowledge and understanding, apply that understanding as well as formulate, analyse and evaluate aspects of the scientific method and communication.

The literature on the psychology of learning would suggest breaking down these skills into their subcomponents. This means we need to look at methods that develop knowledge and understanding from knowledge. Organising our units in ways that help students see the bigger concepts and connections between concepts within the domain will also help. For more on this see my previous post here. I think that understanding develops from knowledge.

I recently read that Thomas Khun claimed that expertise in science was achieved by the studying of exemplars. Scientific experts are experts because they have learned to draw the general concepts of the specific examples.

Useful sub-skills would be:

  • Fluency with the terminology of the domain
  • Ability to read graphs and data
  • Explicit knowledge of very specific examples
  • Explicit knowledge of abstract concepts illustrated by the specific examples
  • Ability to generate hypothesis and construct controlled experiments

Deliberate practice in biology

Thinking about these sub-skills, then, we can see what may constitute deliberate practice in biology and thus what would make useful formative assessments within the subject.

Fluency with the terminology can be gained through the studying of terminology decks like those available on quizlet. In addition, the work of Isabel Beck. Suggests that learning words isolated from text is not that helpful to gaining an understanding of those terms. To gain this, students need to be exposed to these words in context. Therefore there is a lot to be said for tasks and formative assessments that get students reading. Formative assessments could then consist of vocab tests and reading comprehension exercises of selected texts.

Reading and interpreting data can be improved through practice of these skills. This is an area where inquiry alone won’t help students make progress. Students need to be shown how to interpret data and read tables and graphs before making judgements. Ideally, in my opinion they should do this once they have learned the relevant factual knowledge of a related topic. Formative assessments focussing on data interpretation should therefore come a little later once students have covered a bulk of the content.

To build up conceptual understanding, students need to be exposed to specific examples related to those topics as I outlined in this post. Tests (MCQs) that assess how well students know the specific details of an example could be useful here to guide learners to which parts they know and those they don’t.

Following this we can begin to link examples together to build knowledge of a more abstract concept. Concepts can then be knitted together to develop the domain specific thinking skills: thinking like a biologist.

Formative assessments

Formative assessments could take the form of MCQs but as outlined above, vocab tests, reading comprehension activities, and other tasks may well have their place here.

Summative assessments for measuring progress

I am now thinking that to truly assess student progress against the domain, individual unit tests just won’t cut it. As Christodolou argues, summative tests exists to create shared meaning and do that need to be valid and reliable. Does scoring a 7 in a unit test on one topic of an 11 topic syllabus mean that the student is on track to score a 7? Not necessarily. Not only is the unit test not comparable to the IB 7 because it is only sampling a tiny portion of the full domain, but the construction and administration of the test may not be as rigorous as that of the actual IB papers.

Clearly it isn’t ideal to use the formative assessments described above as these are nothing like the final summative assessment of the course, plus their purpose is to guide teaching and learning, not to measure progress.

I would argue that summative assessments over the two-year course should use entire past papers. These past papers sample the entire domain of the course and performance against them is the best method of progress in the domain. A past paper could be administered right at the start of the course to establish a base line. Subsequent, infrequent, summative tests, also composed of past papers could then measure progress against this baseline.

Why should summative assessments use past papers? What not use unit tests? Unit tests, aggregated, is not the same thing as performance on a single assessment sampling the whole domain. They cannot produce the same shared meaning as an assessment that samples the entire domain. In addition the use of many single unit, high stakes tests will cause teaching to the test as well as much more student anxiety. Instead lots of formative testing and practice of recall should help to build students confidence in themselves.

Categories
Education Teaching & Learning

Remembering stuff

Originally posted on November 9, 2018 @ 9:00 am

Someone once said that the educational debate in the UK is lightyears ahead of the debate internationally. It is a shame really because you would hope that the minds engaging with educational debate from every country would add to making the debate more urgent.

The modern education system is sometimes characterised as being one where kids are mindlessly forced to rote learn and that we have to fight against this industrial factory like education. It’s anecdotal I know but I have worked in five schools and visited a few more and never seen anything like this. Where are all these schools that are battery farming their kids? Most schools are definitely more progressive in their outlook than traditional, in this sense. Although I would contend that good schools and teachers in them know when to adopt different techniques as necessary.

If you engage in debates about the aims and methods of education it is common to read thinking like this, a popular view exposed by many educators, and widely influenced by romanticism:

“The point I am making is that DI is very successful in a certain thing that we are measuring. Remembering stuff. For an education system that measures how well you can remember stuff sat at a table for two hours (of which the DP is really no different from any other offering) then I’m sure DI is highly effective…. but really why do we care? We all pretty much know that that such a metric is a) a terrible way for Unis and businesses to know that they have recruited an effective colleague b) it just isn’t they way to make it in the world past examinations. Once our smartphones can answer any knowledge based examinations (not far off from now) then DI will just about be a waste of everybody’s time. What I’m interested in is what type of instruction leads to creative, communicative, empathetic, collaborative, entrepreneurs and explorers? If DI does that then I’m interested. But first we have to develop a way of measuring these things to see if a certain practice achieves it. Any other research is basically past its sell-by-date, as I suspect are exam based remembering courses.”

Remembering stuff. It’s the practical equivalent of the old, male and stale ad-hominem stereotype trotted out in arguments in post-modernist education discussions at times. It’s uncool. It’s useless and why would anyone who cared about kids and their futures insist on paying attention to it in their classroom or school? It’s outdated. We don’t need to remember because we have google now. We don’t need knowledge because AI will take over our jobs and if we make sure kids know and remember stuff then they are doomed to be job-less, on future the scrap heap, in a world where 65% of the jobs haven’t yet been invented yet.

Why do we care about remembering stuff? 

First, let us not conflate remembering and knowing. They are not the same thing. Technically, remembering is simply the process of retrieving information from your long term memory that you know. Knowing something is having it stored there in the first place. It is possible to know something and not remember it.

This argument above mentions remembering initially but then refers to knowledge based exams and questioning the value of knowing stuff when our smartphones can do that for us later, effectively conflating the two. Both knowing stuff and being able to remember stuff are important. It’s no good knowing stuff and not being able to remember it and you can’t remember what you don’t know. So, in my view education has to help students do both of these things. Why is knowing stuff and then remembering it important and why should we care?

Well, actually, knowing stuff is still pretty important. Believe it or not. Some educator’s use Bloom’s Taxonomy to assert that remembering stuff is at the bottom of the pile, a low order skill useless on its own. However, despite the fact that this taxonomy is not informed by the cognitive psychology of how people learn and it is often presented uncritically, this interpretation is also not what Bloom intended. He put knowledge at the bottom as it is the foundation on which all else is built.

You can’t do much if you don’t know anything. And in fact the more you know, the more you can do, including learn more. The Matthew Effect is a well documented psychological phenomenon by which the rich get richer and the poor get poorer. The more you know, the easier it becomes to learn more and therefore become a life-long learner. That is one reason why we should care, especially if we want to make life-long learners.

These days it is fashionable for international educators to discount knowing  stuff because the international consensus is that 21st century skills are more important than knowledge per se. These 21st century skills are generally recognised to be the four C’s of communication, collaboration, creativity and critical thinking. The line of reasoning is, generally, that we need to teach these skills instead of knowledge.

There are a few problems with this line of thinking. Firstly these skills are not actually 21st Century in and of themselves, and there is no reason to think that they are more important this century than they were in the time of Julius Cesar. Indeed, calls for skills based curriculums go back at least a century already.

Secondly, we can’t have people skilled in these areas who aren’t also knowledgable. Most psychological research to date suggests that creativity requires knowledge and it is only possible to think critically about what you already know about. If you really think about it – to be a great communicator you actually need to know about what you are communicating about. Could you imagine the BBC Earth documentaries not only without a knowledgable David Attenborough but the teams of knowledgeable researchers who write the scripts?

Thirdly, the idea of teaching generic skills is also flawed. The generic skills method of teaching postulates that authentic tasks are ones that mimic real life i.e. science teaching that gets kids to act like scientists. Authors like Daniel Willingham and Daisy Christodoulou point out that the most effective way of teaching skills is through the deliberate practice method. Just as a football team doesn’t practice by playing games, but by breaking the skills needed to win (dribbling, passing) down to their component tasks and practicing those.

In short, knowing stuff (and remembering it) is the foundation of the skills we want to instil in our kids, it is also the foundation of understanding and the foundation of life-long learning.

We all pretty much know that that such a metric is a) a terrible way for Unis and businesses to know that they have recruited an effective colleague b) it just isn’t they way to make it in the world past examinations.

Do we? How exactly do we know this? It seems hard to make that claim as it is pretty much unmeasurable. Even if you could survey every employer and university there are too many conflating variables. We are all products of this system. This claim is made without any proof and the burden of proof lies with the one making the claim.

Once our smartphones can answer any knowledge based examinations (not far off from now) then DI will just about be a waste of everybody’s time.

Oh no. Seriously? We still honestly think this? It is right up there with the “we can google it” claim that knowledge isn’t worth having. In addition to what I have written above I should highlight here the distinction between working and long term memory.

Working memory is what you can hold in your awareness and it is limited. The environment and long term memory are accessible from working memory and long term memory is unlimited in its store.

If we rely on google and not our long term memory we will find it very hard to make sense of the world around us as our working memories will constantly be overwhelmed. We wont be able to chunk information.

Knowledge isn’t just what we think about it is what we think with. If you rely on google on your smartphone you won’t be able to think well, you certainly won’t be able to think creatively nor critically nor communicate well.

Also, google is blocked in China. Do we really want to give governments that much power over knowledge and what we know and can know?

What I’m interested in is what type of instruction leads to creative, communicative, empathetic, collaborative, entrepreneurs and explorers? If DI does that then I’m interested. But first we have to develop a way of measuring these things to see if a certain practice achieves it.

Yes, it can do. DI has been shown to effectively increase what people know and remember. If knowing and remembering is the foundation of being able to think well, collaborate well,  and create well then we shouldn’t just throw these out.

One of the problems with international education in my view is that it over emphasises inquiry learning, making ideologues get hot under the collar when DI and other guided instruction is mentioned. We are trained to think schools are battery farming kids, when to be honest, they really aren’t.  I think we need to try to find out what works in what context and focus on that. I think that there is a place for guided instruction.

Anyway, DI does not always equate with rote learning. Why make it out to be?

I am also now reminded me of this article and this tweet. They are based on similar assumptions and outlooks, and I had wanted to write something in response to these claims.

I agree with Noah Harari when he writes that we often conflate intelligence and consciousness.  I am not convinced that AI can actually know anything. I think it is intelligent and can process a lot of information quickly, but I would contend that to know anything and remember anything you need to be conscious.

If this is true what is the real risk presented by AI? Probably automation of tasks that rely on data processing in some form. Doctoring for example, requires the ability to process symptoms and match them to known illnesses. But not every job is at risk of automation because not every job relies purely on data processing. As Harari contends in his books, the highly prized human jobs of the future will be the ones that rely on human ability to relate to other humans. Therefore Doctors are at much more risk of being automated than Nurses. However, Nurses still need to know an awful lot of stuff as well as be at good at relating to other people to be able to do their jobs.

Humans need knowledge to be able to think well and to specialise in areas. If we don’t ensure that people know things they definitely will not be better placed to work with or instead of AI. The people that are replaced by AI will be the ones who don’t know much.

Additionally, the fact is knowledge rich curriculums demonstrably reduce inequality and with the way social divides are opening up in our modern society perhaps the way for international education to contribute to a peaceful world is to close those gaps? Seeing as DI has been demonstrably shown to reduce social inequality (See Why Knowledge matters by E.D. Hirsch) and as international curriculum’s like the IB is placed in many public schools in poorer areas, I find it’s focus on inquiry teaching quite worrying.

I wonder if international educators can afford to ignore this stuff because generally our kids come from educated and affluent homes?

Categories
Teaching & Learning

Notes on making good progress?: Chapter 1

Originally posted on October 28, 2018 @ 7:59 am

In this series of posts I record my notes from Daisy Christodolou’s book “Making good progress? The future of Assessment for Learning” It is quite excellent. You can buy a copy here.

Why didn’t Assessment for Learning transform our schools?

Formative assessment is when teachers use evidence of student learning to adapt instruction to meet student need. It’s focus is on what students need to do to improve, on their weaknesses. Feedback then, needs to be tailored thoughtfully to direct the student in how to improve and allow students to act on that feedback.

Formative assessment should be used to diagnose weakness and feedback should tell students how to improve explicitly. AfL is not just about teachers diagnosing weakness and being responsive it is about students responding to information about their progress. Could be a good model for appraisal too.

There is a tension then, between summative and formative assessment. One is about measuring student progress against the aims of education while formative assessment is about the students finding out what they need to do to improve.

If we can agree on the aims of assessment, there is still a discussion about methods. We either favour the generic-skill method which states that to get better at a particular performance you just need to practice that performance. So if you need to practice critical thinking you just practice it or if you want to get better at writing an essay, you just write lots of them. Or we favour the deliberate practice method. This method breaks the final skill down into its constituent parts and practices those. So footballers practice dribbling, passing, defending and shooting not just playing whole games all the time.

Summative assessment is about assessing progress against the aims of education. Formative assessment is about the methods you choose to meet those aims: generic or deliberate. Depending on which one of these you subscribe too will affect your formative assessment, and thus whether assessment tasks can be used formatively and summatively.

If you believe that skill acquisition is generic then formative assessment tasks will match the final summative task. You will write lots of essays, feedback can be given and a grade awarded. If you believe that the method of deliberate practice is better then you may need to design formative tasks that don’t look like the final task. These tasks cannot be used summatively because they don’t match the final task.

Interestingly, belief in generic skills leads down the road of test prep and narrow focus on exam tasks because this model suggests that to get better at the exams you do need you need to practice taking them.

In my mind the key questions for a school, curriculum level or department that wants to adopt the deliberate practice model should be:

  • What are the key skills being assessed in the final summative tasks (don’t forget that language or maths skills might be a large component of this?
  • What sub-components make up these skills?
  • What tasks can be designed to appropriately formatively assess the development of these sub-skills?
  • What does deliberate practice look like in my subject?
  • How often should progress to the final summative task be measured i.e. how often should we set summative assessments in an academic year that track progress?
Categories
Teaching & Learning

A summary of the structure of knowledge

Originally posted on June 27, 2018 @ 1:20 pm

In the final term of this year, I completed an online course on “Theory of Knowledge” from the University of Oxford’s department for continuing education. As part of this course, I have to submit two assignments. The first, which is a summary of the structure of knowledge and limited to around 500 words, was due on the 5th June and I am posting a copy of it below.

A summary of the structure of knowledge

According to Pritchard (2014), we can distinguish between two types of knowledge: knowledge of something or knowledge of how to do something also referred to as propositional knowledge and ability knowledge respectively. It is the first of these that we are interested in in this summary.

Knowledge is valuable because knowledge has instrumental and non-instrumental value. Having knowledge is instrumentally valuable in the sense that it helps us achieve our goals, but it is also non-instrumentally valuable in the sense that having knowledge enriches our lives in and of itself.

To claim to know something is to make a claim or a proposition that a) you believe something and b) that your belief is true. If I claim that it is raining in London while I am living in Lausanne, and assuming that I have no ill intent to deceive those I am talking to, I am making a proposition which I must ultimately believe – how could I claim it was raining if I didn’t ultimately believe it to be so? Intuitively it seems that we cannot claim propositional knowledge if we don’t first believe it.

The claim that we know something “aims at” truth, to use Pritchard’s (2014) phrase. Claiming knowledge intuits at the truth of reality. We don’t normally count someone who holds a false belief as holding knowledge of something. For example, in a pub quiz, someone could be said to be knowledgeable of the topic in question if they hold what is commonly accepted as the “correct” or truthful response. Someone who incorrectly or falsely believes the answer is another proposition cannot be said to know the answer.

Thus, we can say that truth and belief are necessary conditions of knowledge. However, a guess (like a bet) that gets to the truth of the matter (that turns out to be true) is also a claim that contains truth and belief but is not considered knowledge. Under normal circumstances, someone who wins at roulette with the number 29 can’t be said to know that 29 was the correct number, but they did have a true belief that 29 was the number.

Therefore, to count as knowledge, a claim needs have more than truth and belief, it also needs to be justified. Knowledge has historically been counted as justified true belief. All three of these elements are necessary conditions for knowledge but on their own, they are not sufficient conditions for knowledge.

For example, Gettier cases show us that justified true belief isn’t always enough for knowledge. By luck, some agents can still hold true beliefs that are justified but that we would not normally count as knowledge. In the case of an agent who “knows” the time by looking at a stopped clock, if they look at the clock at the “correct” time even though the clock has stopped they will have gained a justified true belief, but they will have done so by luck. If they had looked at the clock five minutes later or five minutes earlier they would have acquired a false belief (Pritchard, 2014).

So, we also need more than justified true belief. We still need to consider the type of justification that is used when combined with true belief. More specifically we need to consider what supports our beliefs in order for them to be justified. There are normally three ways of considering this: a) beliefs do not need to be grounded on anything b) beliefs can be founded on an infinite chain of justifications c) beliefs can be grounded on a circular chain of beliefs. The different schools of thought of infinitism, foundationalism and coherentism offer different responses to this trilemma.

Justification and the support needed for belief is closely linked to rationality. Normally only rational beliefs would be considered knowledge. We can think of a judge who reaches their decision either by weighing up the evidence presented or on the basis of their emotional or prejudice. A judge who rationally weighs up the evidence to reach a verdict can be justified in their true beliefs but a judge who doesn’t, can’t be. However not all rationality is linked to finding the truth and to justify our beliefs we should be concerned with having epistemically rational beliefs. Pascal’s wager is a good example of the difference between epistemically and non-epistemically rationality. In the same vein, we need to consider whether agents can or should be held responsible for their beliefs.

Are people responsible for paying attention to how their beliefs are formed? Can we count a belief as knowledge if the agent in question has not considered how they have formed their belief?

References

Pritchard, D. (2014) What is this thing called knowledge? 3rd edition. Routledge.