Measuring For Learning

Expat Educator StudyingThe Problem With Traditional Learning Measures

Not everything that can be counted counts is a phrase often used in the conversation about standardised tests. Whether originally stated by Dr. Stephen Ross or by Albert Einstein, human beings have long sought “fair” ways to determine which students are high achievers and which student are likely to be successful in university. The logic goes something like this: If we give every student the same test under the same conditions, those who know more will pick more correct answers and earn higher numerical marks.

We can’t quantitatively definitively define “success” by testing unless we reduce it to a number. So we define academic success by numbers on a test or letters on a report card and we define lifetime success based on income or leadership level within organisations. 

I was interested to hear from Alan Boyle, an author from OnlineEducation.net, about a recent article on how top universities seek to measure noncognitive skills as predictors of future academic success. The article made me think back to all the applications I didn’t fill out because my SAT scores, used at the time as the most accurate predictor of university success, didn’t hit the magic numbers. I remember feeling cheated that I couldn’t tick the box “Top 10% of graduating class” because my 3.98 GPA made me third in a graduating class of 23 students. Had I chosen to drop Physics, I would have been able to tick the necessary box on the application. I wished for a way to communicate to admissions officers my drive to achieve whatever was required of me – and my willingness to stick with a subject even if it compromised my numerical standing in the rank of graduates.

Standards-Based Continuums Bring Out the Noncognitive Dispositions

My love for curriculum stems from the way curriculum is written. Curriculum scope and sequence documents comprise statements of what students should know and be able to do – information the “me” from High school wished to communicate to universities with highly competitive admission requirements.

The executive team at my current school is leading teachers in an effort to put the Australian Curriculum (AC) standards onto continuums. As students move from Prep to Year 12, common assessments determine student movement through the continuum of results statements for each subject. In history, for example, a student might demonstrate a Year 9 level of historical knowledge and demonstrate a Year 6 level of historical research skills.

The standards-based continuum is transparent. Assuming common assessments are valid and scored reliably, both parents and students know students’ specific strengths and areas for growth in each subject.

My hope is that, through the transparent standards-based marking process, noncognitive areas for growth will emerge. If, as Alan suggests, one can improve important noncognitive skills by taking on challenges, pushing through difficult situations, setting measurable goals, and becoming a decent person, then a transparent set of standards gives students a continuous set of challenging goals through which they navigate.

More specifically, through a transparent, standards-based continuum based on valid, reliable assessments, students

  • understand the next level of expectation in each subject
  • set measurable, personal goals in each subject each term and follow through with those goals
  • demonstrate the grit to push through the cognitive and emotional challenges associated with achievement of subsequent standards

Decreasing the Weight of Standardised Tests

If we can create a valid, reliable way of measuring student outcomes, then standardised tests become one of the ways rather than the way of determining student “success” in school.

Such a continuum does not make students exempt from standardised tests. Australian students encounter the NAPLAN in grades, 3, 5, 7, and 9. While results are not linked to school funding as they are in the American outcomes of NCLB, school NAPLAN results are published and accessed easily by parents.

But if we can report to students, parents, and universities that a particular student under regular life conditions is able to set high academic goals and achieve those goals, families and universities might more accurately predict that the same student would set and achieve future goals.

More Clearly Describing Success

We can label students as successful because they receive a particular number on a particular test on a particular day. Alternately, we can describe what “success” looks like in each subject area and help students celebrate continued growth and eventual mastery. Under which conditions will students learn more?

if students work toward descriptors rather than numbers, the common assessments become a measure of academic achievement and indicate growth of the noncognitive measures such as grit, stamina, and resilience. Learning extends beyond the academic.

So what if we stop counting and start describing? Not everything that can be counted counts. But things we describe and to which you can aspire count.

photo credit: Angela Radulescu via photopin cc

If you like what you read, please sign up for Expat Educator updates by email. Your address will never be shared with outside parties.

Free Professional Development for Teachers

Expat Educator Free Professional Development

What do you want to do better in 2013? If professional development is on your resolution list, this post will give you a jump-start.

Consider using a bit of time over the next couple weeks diving into one or more of the professional development opportunities listed below.

Webinar Archives

If you are fascinated by curriculum (like me), you can spend hours visiting ASCD’a Archived Webinars. These webinars helped me better understand the American Common Core Standards. They helped me more clearly differentiate between standardized assessment and standardized instruction.

Another treasure can be found on the Virtual Cafe Archived Webinars wiki. You’ll meet librarians, media specialists, proponents of gamification in education, and experts in technology integration.

Simple K-12 webinars are popular enough to be noticed by many who nominate and vote for Edublog awards. You can sign up for 500+ free webinars and, if signed up, will be offered much more for an annual fee of $279. Watch the prices. At least once per year, the price drops about $100.

If you sign up to follow the Australia Series Blog, you will have access to weekly, live, free webinars on a range of topics. Most topics relate to technology integration. The downside is that these webinars are not archived – so you need to be available to access them at a time compatible with Australian time zones.

If Australian time zones are difficult, Teacher 2.0 and Classroom 2.0 offer a multitude of webinars accessible to the US time zones (usually 2:00-3:00 Eastern time). Again, these need to be accessed live (with some rare exceptions).

Conference Archives

OK2Ask has archives of video “snack” sessions. All online sessions and materials are provided by Teachers First, an ad-free cornucopia of practical resources for teachers.

Seminar Archives and Lectures

Math Teachers will appreciate NCTM eSeminars Anytime. You’ll find seminars on research intervention, common core implementation, and more.

TeacherCast recently tweeted about Academic Earth Video Lectures. These could be used for personal interests or for flipped classroom lessons.

Hodgepodge

Vital podcasts, videos, webinars, and resources are delivered by The Open University and part-funded by the Department for Education. On the site, Vital says it “aims to support practitioners in sharing their expertise and thus enhance the quality of teaching and learning.” Search by teaching level and subject to find topics relevant to your professional growth needs.

Laura Candler is very popular on the Teachers Pay Teachers site, but she has some free Livebinder resources available. Laura will periodically offer free resources to followers of Corkboard Connections.

Connected Online Communities of Practice (COCP) in cooperation with the U. S. Department of Education has contracted with the American Institutes for Research and five other organisations to declare August as Connected Educator Month. Archives of all webinars, sessions, and discussions are available.

Podcasts

Elizabeth Peterson from The Inspired Classroom lists some excellent podcasts to which you can subscribe.

Also check out the podcasts nominated for Edublog awards. You’re bound to find one or two that meet your professional development needs.

Teaching English

Shelly Terrell and TESOL team up to offer Free Friday Webinars Archive – a tremendous resource bank of ideas.

American TESOL has other webinars available as well.

Inspiration

Perhaps 2013 is the year you need to recharge or, as Shelly Terrell calls it, reboot. While the current challenge is #26 at the time of this posting, you can look back at previous goals. Can you meet all 30 by 2014?

What do you want to achieve in 2013? What free resources would you recommend? Please tell me in the comment box below.

If you find this resource list helpful, please consider subscribing by email to Expat Educator. You’ll get updates delivered to your inbox.

photo credit: Swansea Photographer via photopin cc

Assessment for, as, and of Learning: Math Problem Solving Series #5

You stare at the stack of Problem Solvers on your desk. You flip through them. One paper has well-labeled work and shows clear thinking, but the student has the wrong answer. Another paper has the correct answer but the evidence of conceptual understanding is unclear or is scattered throughout the paper.

How do you mark the papers?

Before delving into the procedures and rubrics, some assumptions must be stated:

  1. Assessment is different than grading.
  2. No assessment system is perfect, but some are better than others.
  3. Students and teachers should both have an idea of what the ‘grade’ will be before students hand in a paper.

The assessment process should have started earlier in the week – before students received the papers that are now in your pile.

Backing up

The papers on your desk should not represent students’ first attempt at a type of problem – especially at the beginning of the year.

If a concept or strategy is new, it is not unusual to spend a full class period allowing students to construct concepts related to one or more strategies. When students receive similar (but slightly more complex) problems on subsequent days, they will solve the problems more quickly.

Those first couple days, you assess student work, but you don’t collect it to be graded. Instead, you assess progress using checklists and anecdotal notes.

Formative Assessment #1

Checklists and anecdotal notes are invaluable forms of assessment. Remember that, on the first day, students have individual work time, pair share time, more individual work time, and class consensus time.

During the initial individual work time, look for the students who demonstrate understanding right away. Indicate such on your notes. Visit those students. Increase the difficulty of the question if necessary. What if this pattern continued to…?

Look at the students’ pictures and diagrams. Are there any that have misunderstood the language of the problem? Make a note and help them understand the context and the question.

Note those who are experimenting – and what kinds of experiments they are trying. Are they doing random operations all over the page? Are they making charts or tables? Finding patterns? Looking at their neighbors’ papers?

During the pair share time, watch for students who are carefully explaining their processes, those who are just stating an answer, those who are passionately defending their processes, and those who are shaving their pencils with scissors.

During the second individual work time, note who changes strategies, who is now able to get started, and who regularly asks to go to the bathroom at this point in the lesson.

You should now have identified three groups of students for differentiated instruction: (1) students who need to see a significantly more complex problem or different type of problem because they’ve already nailed the concept, (2) students who are getting it and just need a bit more practice, and (3) students who will need some more intensive coaching.

While students are coming to consensus, note the way students explain their reasoning to the class. Note who asks clarifying questions and who uses the vocabulary of mathematics.

Continue with the checklists and anecdotal notes during subsequent lessons. Especially note improvements and further misconceptions.

Formative Assessment #2

Students begin the self-evaluation process by reflecting on their work, making corrections, and writing notes to themselves.

Once your anecdotal notes indicate that students are capable of independently completing a homework problem solver, send one home.

Rather than collecting the problem solver the next day, have students share strategies and answers. You might go through the consensus process once again. While sharing or comparing, students can use a different color pen or pencil to make changes and/or write reminder notes to themselves.

The homework with notes can be glued into their math journals for future reference.

Formative Assessment #3

Students should know how their work will be assessed. Hand out the rubric on the first few days of school. Talk through it column by column. Students can mark on it, highlight it, and glue it into their math journals for future reference.

When you hand out the “test” papers, or the ones that will end up on your desk, require students to self-assess their work on a rubric before handing it in. What descriptors match their work?

Some colleagues and I have spent the last month piloting the rubric below – with a great deal of success:

EE: Exceeds expectations, ME: Meets exp, MEA: Meets with Assistance, DME: Does not meet

The rubric above is based on the NCTM process standards. Remember that Problem Solving is at least as much about the process as it is the final answer. Students see the importance of process more clearly when they see that the answer is only a small part of the final ‘grade’.

Summative Assessment

By the time students turn in a “final” problem-solver to be graded, you should have a high degree of certainty that all students will at least meet expectations. The students should be confident too.

Representation involves the pictures and diagrams students use to make sense of the problem. Students have probably used tables or charts to play with numbers and number patterns.

Connections is about connecting the problem to other areas of math or to the real world. Where have you seen patterns or ideas like this before?

Communication. Notice how nothing in the descriptors include sentences like “First I…Then I…Next I…”. A student should not be required to write a tome about their thinking IF the work already includes representations, the numbers/numbers/charts/tables are labeled, and you can follow the student’s train of thought. I’m passionate about this for a few reasons:

  • Students’ grades should reflect mathematical thinking, not writing ability.
  • Writing requirements turn reluctant writers into reluctant problem solvers.
  • It’s a pain to read through prose when the thinking is already clear by looking at the work (yeah, this one is selfish).
  • The time it takes prolific writers to write out their processes in complete sentences could be used to teach more mathematical concepts.

Accuracy. How do you mark the papers when work is done well but the answer is wrong? Dock accuracy, but give the student credit for what he or she did well in other areas.

Reasoning and Proof. I usually don’t mark this column but I keep it there because i want to report to students and parents when my anecdotal notes indicate students did an outstanding job of defending their answers or asking clarifying questions during the consensus-building time. I can also indicate whether students know more than one way to solve the problem.

You don’t have to mark all the columns all the time.

Rubrics help clarify expectations for both students and parents. Students can self-assess and you can confirm or discuss their self-assessments. Parents understand why you are praising their child’s mathematics even though the answer may be incorrect.

So here is my question for you: Which of the formative and summative assessment procedures are assessment for learning? Which is/are assessment as learning, and which is/are assessment of learning?

Final Thoughts: Back to the Stack of Papers

This final paper that is handed to you should be the only thing represented in the grade. Why?

If we ‘grade’ students according to their early work, we end up grading the speed of their learning. If, in the end, two students can solve a problem equally well, should one be penalized for learning the concepts more slowly?

When giving students grades for problem solving, the final grades should reflect math. Final grades should not reflect reading ability, writing ability, or speed of learning.

The stack of papers should now be demystified. Look at the papers, refer to your anecdotal notes, and refer to students’ self assessments. Students are often harder on themselves than I would be. Expect smiles of pride when you mark a paper higher than a student expected. If there is a huge discrepancy between the student self-assessment and your assessment, have a conversation. In my experience, those discrepancies are rare.

If this series has helped you, please consider doing one or more of the things in the storyboard below…

Create a Copy

photo credit: DaveCrosby via photopin cc

Are We Confusing Standards with Standardization?

Standards, not standardization

My soap box. Here I go again…

As I write this, I’m ducking behind my screen, ready to dodge virtual tomatoes. Please bear with me as I question some assumptions we are making with regards to standards and standardization.

We educators use terms and acronyms, assuming that all people have the same understanding. First, I will clarify terms. Then I will ask questions.

Clarification 1: Textbooks are not curriculum.

Most simply put, curriculum is a list of things students should know and be able to do. We call the list a list of standards because we hope that all students will be able to know and do these things when they leave school. Then we create benchmarks, clarifying what those standards “look like” at various grade levels.

I often hear teachers and other school leaders saying that they want to find a curriculum that teaches to the standards. What they’re really saying is this: They are looking for the “magic bullet” educational materials that will help student test scores improve.

I haven’t used textbooks in over ten years. What I learned from my Australian colleagues is this: Teachers can look at a list of standards and figure out the best way to teach to those standards.

So here are some questions: Are schools underestimating teachers? Are schools assuming that teachers cannot teach to standards unless they have the “right” materials?

Clarification 2: Standards are different than Standardization

If we understand curriculum as a list of standards describing what students should know and be able to do, we can differentiate between curriculum and instruction.

Curriculum is built on standards. Instruction may or may not be standardized.

The progression of assumptions goes something like this (my reaction in italics):

  1. We need to teach to the common core (standards). True.
  2. The district has purchased materials that align with the standards. Okay.
  3. If we all teach this curriculum (a misuse of the term), then [the company's] research suggests that students will test better. Here is training on how you should all use these materials… Hold the phone!!!!!

We have crossed a line at #3. We assume that, to hit standards, instruction must be standardized according to commercially-created materials.

My next question: Once companies have convinced us that they have the “right” materials, are we requiring all teachers to use those materials in the same way?

Clarification 3: If we agree that instructional standardization is unnecessary, we can maintain creativity and passion in a standards-based classroom.

But we need to make a few paradigm shifts.

Specifically,

  1. Look at the standards before we look textbooks or think of “thematic” activities. The unit on Spiders is no longer a list of activities. It is a list of standards first (classification, expository writing, research, health and safety), then activities.
  2. Use team meeting time to plan. Work together to compile activities and resources that will teach to the standards. Use textbooks and other materials as resources. Trust yourself to create new activities that teach to standards more authentically than pre-packaged materials.
  3. Maintain checklists of standards and keep track of students that have and have not met specific standards. Project-based learning is then tweaked to include the following instructions: Somewhere in your project, you need to show me that you understand the difference between insects and spiders. You need to tell me whether or not your spider is dangerous and how you can tell. When I come and talk to you, I will ask which books and websites you have been reading and how they helped you.

Clarification 4: There are some things that are just wrong.

Wrong: Awards and sanctions for schools, teachers, and students based on test scores. Household rewards and sanctions do not get kids to bed on time, nor is bedtime a single standard by which we judge parenting (thank heavens!). Let’s pay attention to the scores, but realize that tests will never tell us the full extent of student knowledge.

Wrong: Hours and hours of standardized testing. I’ve created assessments where students learn through the process of demonstrating, synthesizing, and evaluating their knowledge and skills. Students learn nothing when they fill in bubbles. A few hours per year is okay. Weeks? Wrong.

Wrong: Teaching all students the same way. The little Steve Jobses and Mark Zuckerbergs in your classroom (who probably test well), will withdraw or start programming social media when they should be underlining the topic sentence.

There are more wrong things, but those are the biggies.

Conclusion: Education is not doomed, unless we confuse standards with standardization.

As Yong Zhao Trim said, American education was “doomed” in the 1960s when the Russians beat us to space. American education was “doomed” according to the 1980s publication “A Nation At Risk.” NCLB was created because schools were failing.

Yes, we have work to do in education. But, we have innovative teachers who care about student passions and are capable of creating lessons that teach to standards. Accomplished teachers know their students and how they learn.

Let’s teach to the standards, but teach them in ways that are individualized, differentiated, and personalized.

My last question: What if we change the assumptions?

If the new assumptions are as follows…

  • Teachers can teach to standards with or without specific, commercial materials.
  • No one set of materials (commercially-created or otherwise) will help teachers teach all standards to all students.
  • Standards can be taught and tracked in the midst of innovative, project-based classrooms.
  • We fight government initiatives that are truly wrong while resolving to show the world that students learn through individualized, differentiated, and personalized instruction.

…how would schools look different?

If you like what you read, please consider subscribing to Expat Educator (in the footer).

Struggle With Concepts. Scaffold Projects.

In a recent Edublogger post, author Ronnie Burt shared a debate he was having with fellow author Sue Waters.

The question: Should teachers allow students to struggle with concepts, or should teachers scaffold information by walking students through a step-by-step process?

My answer: Yes.

Struggle with Concepts

In the context of math instruction, students need should be struggling with concepts before they are led through paper-pencil activities. I can immediately pick out students who learned step-by-step math processes before understanding concepts. Here are a few telltale signs:

  • When given a problem such as 100 – 2, they will stack the numbers and regroup or borrow.
  • When asked what 1/3 looks like, they look at you with a blank stare.
  • When asked What is area? they respond “Area is base times height”.
  • When asked What is pi? they respond “3.14″.

Arguably the most famous of the math bloggers, Dan Meyer, poses thee-act plays and mathematical dilemmas. After viewing the first of three acts or a picture like the one below, students are asked to pose questions, decide what they need to know, and answer their own questions.

The picture below incited fascinating student conversation and solid application of math concepts in a way that was far from linear.

From http://bit.ly/HS1OW5

In the context of language arts, students should be struggling with the “big Idea” of a book through discussions. While I could lead students to a “big idea” using scaffolded comprehension questions, organic conversations are far more meaningful.

Rob Ferrin, teacher of a Humanities in Action course, has students struggle with World View Questions such as Is the group more important than the individual or is the individual more important than the group? or Is violence an inborn part of who we are (nature) or is violence something we learn from society (nurture)? During the semester, students research, reflect, and analyze both local and world issues. Students’ thinking is then brought back to the World View Questions in a way that causes students to defend, change or refine their worldview.

When students struggle with concepts, they construct meaning.

Scaffold Projects

Learning can be messy. Final projects must be clear because, in general, projects communicate learning.

A previous posts explains in great detail the process of scaffolding student projects: Keeping Students Engaged in a 1:1 Project-Based Classroom.

In short, students need to decide on the message they want to communicate and the medium that will best communicate that message. Then, students list the project steps, backward plan, and follow through with the process.

The Humanities in Action course requires students to create a final project. The Hong Kong Worldview project guidelines are scaffolded. Both steps and expectations are clear.

Conclusion

The Edublogger post rightly stated that research is available to defend instructional practices where students struggle with concepts and instructional practices that involve scaffolding. The guideline “struggle with concepts, scaffold projects” is, admittedly, an oversimplification. Students who have a low tolerance for uncertainty will learn most concepts better with scaffolding. Other students will get bogged down in linear processes.

In the end, the answer lies in the extent to which “Teachers Know the Subjects They Teach and How to Teach Those Subjects to Students” (NBPTS Proposition 2) and “Teachers are Responsible for Managing and Monitoring Student Learning” (NBPTS Proposition 3). If teachers know their subject matter and teachers understand the needs of their students, they will be able to let students struggle when necessary and scaffold learning when necessary.

What’s your opinion?

If you like what you read, consider subscribing to Expat Educator (below).