Marzano Misconceptions

OK, the power of Twitter has hit me hard.  The #sbgchat this last week was, like all weeks so far, awesome.  Someone asked about how to calculate an overall grade in a standards-based grading system.  Obviously, the perfect world would allow us to continue to keep the standards separate, but most people have to supply a single grade for a single course.  I suggested to this person to check out Marzano’s materials.  A couple people tweeted and multiple people retweeted the idea to be cautious with Marzano.  The only idea I could take away that evening, which was supported the next day via another conversation, was that Marzano pushes the idea to use “formative scores” to calculate grades for each standard and subsequent overall grades.

This bummed me out a bit, as I think there is a misunderstanding.  I think Marzano uses the word “Formative Scores” different than what comes to mind for much of the educational community when they hear the terms.  My understanding of Marzano comes largely from this book (I didn’t make these PDFs, just Googled them...): http://pepstep571.wikispaces.com/file/view/Marzano+Formative+Assessment.pdf.  If you go to the page labeled 27, you will see a summary of his perspectives on formative and summative scores and the distinction between his “definition” of formative and summative scores compared to the idea of formative and summative assessments.  It is crucial to understand what he is trying to say here before we are too critical of his methods.  

I think the key here is to take a step back.  Our whole goal of assessment is to see what kids know and can do.  All assessments are flawed, but hopefully, we can have enough assessments tied to a particular standard to get a reasonable idea of how well a student knows that standard  

With this in mind, when we assess students, we can use the information from that assessment in at least two ways.  We can use that information to inform our learning...which is where “formative” assessment comes from.  Also, we can use that assessment to inform our judgement on what a student has learned...typically called “summative” assessments.  Pyscologically for the kid, if you wrote a score down on the assessment that the kid can see, you are using that assessment to judge the kid and they will likely not use it to inform their learning.  Any assessment with a score tied to it is a “summative” assessment.  It doesn’t matter if you are using clickers, thumbs up, exit slips, or a written test.  If there is a numerical grade, the assessment has had a “summative” interpretation done on it.

If I am looking at student work with the lens to put a numerical score on it, I hope I write that score down into some sort of gradebook.  Further, at the end of the grading period, when I look back to see what that kid knows based on a variety of assessments, I should consider all of those assessments.  The most recent assessments are more important, but if an assessment was important enough for a student to do and for me to grade, it can and should have consideration at the end of the grading period.  That consideration may very well be, “The have grown since then, I am ignoring this now.”  It may also be, “they are really up and down and before I can give a final grade I have to find a different way to assess this student’s understanding.”  These SCORES are FORMATIVE because they are giving me information about a students understanding.

Where does the classical “formative” assessment fit in to Marzano’s model?  He has another use of assessment which he calls “instructional feedback.”  I think this is when he is using the generally accepted idea of formative assessment.  Marzano’s definitions are different from the mainstream logic.  I am not going to get into which set of definitions are better.  When we take a step back, most people who are talking about reforming our assessment and grading practices are saying the same things:

• Have clear standards/goals/targets/objectives/whatever
• Be sure instruction and assessments match the standards.
• Give kids quality feedback throughout the learning processes
• Help kids find ways that they can give themselves and peers quality feedback
• When you write down symbols that represent the learning (grades, numbers, etc), be sure that those symbols have a clear meaning.
• When giving a grade that summarizes the learning on a standard, use math cautiously.
• Consider more recent evidence more than older evidence.
• Avoid the “overall” grade per class as much as possible.
• Keep kids involved as much as possible.

It doesn’t matter who you read.  O’connor, Wormeli, Marzano, Stiggins, Chappuis, Guskey or even some Schaefer.  The above ideas are what we should be after.  Don’t get hung up on vocabulary or methodology.

A few weeks ago, I was meeting with some colleagues socially, and the idea of grading came up, and how much of a grade should be made up of “formative assessments.”  To be clear, we are talking about formative assessments in the classical sense...assessments FOR learning.  Someone said that she considered quizzes to be formative, and also because of that, formative assessments should be calculated into the grade...and there shouldn’t be an opportunity to redo, reassess, revise, etc.  Another fellow asked her if when she did thumbs up/down, or hold 1-5 in the air to show understanding type of exercises.  Of course, she did.  Were those included in the grades?  Of course not!

Assessments are not deemed formative or summative based on what you decide to call them.  As many people have said before, it is what is done with the information collected from the assessment that defines them.  Not what you intend to do, what you actually do.

NGSS and Standards-Based Grading

I have a hard time thinking about anything curriculum-wise anymore without thinking about how assessment and reporting will be done.  I have been trying to have at least a peripheral understanding of the NGSS as they have been developed over the last few years.  Currently, as a 7th grade department, we are discussing curriculum and how what we teach matches up with the essential questions we have come up with for our class.  There has been a lot of argument and discussion throughout this development.  In addition, as a district, there is some very early talk about coming up with a Standards-Based Report Card as Wisconsin implements a new Student Information System.

So, how would I change my "standards" in my SBG system?  As I look at the NGSS Framework and draft standards, there are some clear categories.  But, as in all standards documents, it may not be useful to simply dump all of the standards in.  What is the best way to group things?

This is where SBG people and mammalogist meet. Some folks are "groupers" and some are "splitters."  Who is correct?  The problem I run into is what degree of precision is useful to me as a teacher, to the students as learners, to parents, and to future teachers?  A lot to consider, indeed!

I decided to look into the Framework as well as the most recent draft standards, just to get my mind going.  There are two extremes.  Someone could have only 3 "standards" if they wish: Science and Engineering Practices, Cross-Cutting Concepts, and Disciplinary Core Ideas.  If I was to have data on students in these categories, I don't think that is enough precision to make any decisions that are worthwhile.

On the other extreme, I think you could have upwards of 29 "standards" just for the "Life Science" strands.  That would be 8 practices, 7 cross-cutting concepts, and 14 separate core ideas.  Again, as I look through these, what sort of data would be useful?  Are there places we can obviously group things together?  How well can these groupings be defended?  How much would they change with upcoming drafts?

My current thinking is this:

• Use 7 of the Science and Engineering Practices.  Eliminate the "Using mathematics and computational thinking."  I don't see a direct connection to life science there at the middle level in a way that is worth reporting out above and beyond what math class already does.  Moot point.  8 standards would be OK too.
• Use all 7 of the Cross-Cutting Concepts.
• For Life Science, in its current structure, use the 4 core ideas from Life Science.

This would make 18 "standards" to assess and report on. (or 19 if we did the math computation one)

Pros:
This idea could be used throughout a student's K12 experience.  As data was added from classroom level assessments during the normal grading process, a pretty robust picture could be painted of a students understanding and growth.

A lot of the "work" is done for us.  The standards are laid out.  The performance expectations are there.  The work could really focus on coming up with assessments and experiences to work towards those expectations.

Cons:
Are 18-19 standards too many?

Thoughts on this?

Bored kids...my fault

I try really hard to not blame kids.  When something isn't going the way I want it to in the classroom, I try to shoulder the responsibility.  This doesn't always work.  Sometimes I go down that road of blaming, but I don't think I has ever been productive for the long term.  I can, through the stupid power that we sometimes have as teachers, create compliant kids for a short term, but not under an atmosphere that I enjoy or one that is conducive to learning.

For the big chunk of the year in 7th grade Life Science, we learn about how the human body works.  The goal is to investigate the main body systems and get an idea of how they interact to make us work.  I have been struggling for years to come up with something to really pull this together.  I have a lot of great activities, some neat labs, good reading materials...but I was missing a big piece.  I feel like I need more open assessments, something that has less of a correct answer.  Something a big obscure, but will connect to what I am doing.

Yesterday was terrible.  I thought I had an extension project set up, and the kids flat didn't like it.  Now, they would DO it, but I wasn't seeing any passion.  I was getting a lot of procedural questions and a fair bit of off task behavior.  The cell phones were driving me nuts.  Then I had to remember, what I find cool and interesting may not be what they find cool and interesting.  These are mostly 12 and 13 year old kids.  The internal working of an invasive species in the Florida Everglades just isn't cutting it.

So I switched gears.  What is my goal?  I want them to connect the body systems we are talking about to something, in a novel way, to see how it makes an entire organism function.  Well, why not just have them "build" an organism from the ground up.  That's the project, that we will attempt to extend through our body systems learning.  "Create" an organism.  Be goofy and creative and fun and crazy.  We will share through blogs and online posters.

First period today, a student had to figure out how a spider's digestive system works because the craziness she came up with somehow involved a spider.  No cell phone issues today.  Kids were taking them out to take pictures of each other's creations.  Class ended too quickly.

I'm worried about misconceptions.  Organisms are not "created."  There is no such thing as a unibear.  I am aware of these misconceptions on the forefront, and we can deal with those.  Each day that kids are not being curious, is a big day lost.  I can't deal with that.  Their excitement was overwhelming.  They're still kids.  They are goofy, immature, morphing pre-teens and barely-teens.  And I like them that way!

And some of them are learning about the spider's digestive system and comparing it to ours...not because I MADE them, because I LET them.

Standards-Based Grading Success

This isn't directly MY success, but was a very neat thing to see.

I took on the task of a home-bound instructor for an ill student.   During my first visit, we had some math to do.  We worked through a concept review activity which would help formatively assess where she was to see if she was ready to move forward or to get more instruction.  As part of this activity, the math teacher had written the concept goals as well as which questions would tell the teacher and student the level of understanding.  After the student was done, we looked it over together and had decided that she understood the Level 2 questions perfectly, but was struggling on a few specific parts of the Level 3 things.  Those struggles made approaching the Level 4 question quite challenging.

Today, I went to school and chatted with the math teacher.  I told her I was going to "test" her, the teacher, on the 4-Level rubric with this student.  I told her just what I said above, solid on Level 2, struggling but almost there on Level 3.  She replies, "Well, she must be able to set up the ratios and proportions using whole numbers, but must be having troubles when the scaling factors have decimals."

HOLY COW!   Exactly right.

In the end, the goal of grading is to communicate.  This teacher has set up a system that works.  Clear goals matched with quality assessment.  Awesome.  As an "in-between" teacher in this case, the system worked nicely too as I could easily give the student some specific practice that will allow her to make that tiny step to be at a Level 3.

Classroom Stream

Years ago, I stumbled upon a website titled "A Stream Ecosystem in the Classroom."  I have been intrigued by this idea ever since, but kept procrastinating building one.  Well, this summer, a couple of cabinets had to be removed from my classroom to help out another teacher, which gave me some wall space.  I finally decided to get the project going.  With scrap wood laying around in the shed (which my wife always wants me to find a use for), about 15 dollars worth of plastic gutter material, and a couple of hours of time, I came up with this:

The gutter pieces are about 6.5 feet long.  I simply bought two 10 footers, cut 6.5 out of two of them, and then connected the scraps for the middle piece.

I am not a big homework person, but the first assignment of the year will be for kids to bring in muck, and rocks, and sand and other wet gross things.  Then, we get to see what happens!

Reflection and goals

For me, the end of the school year is always one of reflection, but also one of goals and ideas for next year.  When our final day was done last year, I was all jazzed up.  Not for summer, although there are a ton of fun things happening there.  School starts in 84 days.  I think I get more excited about the next school year starting as most do about the last one ending.

I looked at my goals a bit differently this year.  Previously, in my head or on paper, I had some ideas.  They were not quality goals.  This year is a bit different.  Over the last two school years, I have spent quite a bit of time studying John Hattie's book: Visible Learning.  Hattie's book is almost entirely focused on achievement, and many of my goals in the past, although not greatly defined, were also about achievement.  As I read and studied his book, I had a lot of ideas about strategies that I could/should use in my class, but I wondered how I could use his overall message about effect size.  The basic idea is that a treatment's effect can be studied by comparing achievement results pre and post treatment.  Then, by creating an effect size, different treatments can be compared to each other.  This is typically done with meta-analyses, or collections of many studies typically with many students.

I wanted to use the idea in my class, but like all things, a good use wasn't readily apparent.  I attempted, last year, to use pre-assessment data and effect sizes to make goals for students to achieve for their post assessments.  I didn't like how that turned out.  Too often with pre-assessments, the ideas are so new that almost any real gain will produce a very large effect size.  That may help my ego, but it didn't help with real goal setting.

During the 2011-2012 school year, I finally worked out a grading system that worked well.  I used a 4-point scale with well defined levels.  Basically, I used a modified Marzano's scale:

4: Students understand the simple and complex ideas, concepts, and processes that were taught in class AND show the ability to make in-depth inferences and applications beyond what was taught in class.
3: Students understand the simple and complex ideas, concepts, and processes that were taught in class.
2: Students understand the simple ideas, concepts, and processes, but have trouble with the complex ones.
1: Students do not understand the simple ideas, concepts, and processes.
0: No evidence.

In addition to this, each assessment was explicitly connected to one or more pre-defined learning targets.  I then used ActiveGrade as a place to hold and communicate this data with students and parents.  What this gave me, finally, was a wealth of data at the end of the year.  For each learning target that we had written out in our curriculum, I could tell you how well students did, in terms of grades anyway.  This shouldn't be a huge accomplishment, and I am embarrassed that it took me this long to achieve this.  The messed up way I have graded in the past, along with a poorly defined curriculum, I didn't have the tools to create quality data.  I finally feel like I have some, and frankly, I am a bit proud of myself!

Now, what to do with this data?  At first, I sort of stood back, rubbed my chin, and gave a nod of approval.   Nods of approvals and patting one's back doesn't really help kids much though, so what could I do now.  This is where Hattie came to the rescue.  I finally have a baseline data set.  I can now use that to set a goal for next year, and have benchmarks in which to gauge success.  To create goals, the first thing I did was to find an average of grades for each learning target.  Hattie says the "hinge-point" for success is an effect size of 0.4.  I set that as my goal for next year, and ran numbers for each learning target assuming this effect size. Then, I personally reflected on what I would need to do to get students to that level.

Of course, our goal for all students should be a "4."  What I really want for all of my students is to understand what we do in class and then use it in a new context.  The problem is that not all students achieve that.  I decided previously that all students *must* achieve at least a "2" in all learning targets, as a bare minimum, and that goal was achieved.  I like how I can use Hattie's recommendation to set a, hopefully realistic, goal for next year.

Below is an Excel document that shows the learning targets, current achievement, and goals.





Each learning target is assessed multiple times throughout the school year.  Next year, as a department, we will do some clarification on these.  Also, I have reflected on the number of times each learning target was assessed.  Lots of work to do this summer, but I feel like I have a great starting point.

A car without a dashboard

Recently, I ran across a few analogies between grades and vehicle dashboards.  (See Here and Here).  These are interesting comparisons that I ended up thinking about even more.  I spend about one hour a day driving.  I adore this hour for the contemplation time, definitely worth the gas spent.  With my transition to a standards-based grading system, I have felt a mixture of emotions from parents.  The whole spectrum is observed.  Some parents think the change is good and helpful, others don't care one way or another, and some are frustrated.  This post is an attempt to collect my thoughts for the frustrated parent.

Imagine our cars did not have gauges within their dashboards.  Instead, displayed on the other side of the steer wheel, would be a letter: This letter ranged, just like grades, from an "A" to an "F."  We will even put pluses and minuses in for fun too.  What does the letter mean?  We think we know, but in the end "A" is good, and "F" is bad.  Anything in between is, well, in-between.  We don't know how the overall grade is figured.  Every model is different.  Some manufactures weigh speed 70%, while others have oil pressure as the most important.

When you drive your car, the letter changes.  There is no speedometer, odometer, tachometer, gauges for temperature, fuel level, oil pressure, or amperes.  No extra warning lights for low fuel or when sensors detect issues.  Only the final grade.

So, when you are driving along, all is well so long as the "A" stays on the dashboard.

Imagine lending your car out to your teenager.  When they leave, the car is "A+."  Upon returning, "C-."  Someone is in trouble!  But, for what?  Is the engine out of oil?  Low on gas?  At this point, we do not know anything.  Now, you have a well equipped car with all the bells and whistles.  You can actually go online to checkyourcarsgrade.net and see what actually happened mile by mile.  You find out that around 10pm last night, the cars grade was actually an "F" for awhile!

Also imagine that you just started your car, and it's cold outside.  The temperature of the engine is well below where it should be, so to start out, your car reads an "F."  You hope that upon warm up, the grade rises.  What if it doesn't?

Obviously, this scenario would drive most people nuts.  The crazy thing is that grades in our classrooms have been this way for years, decades, and generations.  Along the way, we give grades that are connected to assignments that are oddly added up to give an overall class grade.  We think we know what the grade means, but in the end, "A's" are good and "F's" are bad.  There isn't any consistency as to how the overall grade is figured.  Part of the game is for students to figure that out as we go along.  We can look at the grades in real time with different internet tools, but the assessments are disconnected from the learning.  A "B+" on Green Worksheet doesn't help any more than a "C-" on the car dashboard at 9pm.

A car's gauges could be looked at as learning targets.  We have a range that is optimal, a range that is concerning, and a range that is just flat dangerous.  Some of the gauges are easy for us as operators to understand and regulate.  If the tachometer reads to high, I simply can shift to a higher gear or let off the accelerator.  If the fuel is low, I fill up.  Others may need further investigation.  If my oil pressure is low, I may bring my car into a mechanic.  When the service engine light comes on, I know more diagnostics need to be done.

When we make a switch to standards-based grading, we are asking students and parents to make a tough switch as well.  They have learned to drive the car with only a letter grade appearing.  No, it is not ideal.  It is not efficient.  The feedback is poor.  Change is difficult.  As hard as it would be for us to begin driving a car without gauges, it is just as hard for students and parents to play school with the additional information.  We have to help out on that end.  We have to, for one, show that it is OK to have more information.  We have to help make the switch from the assumption that a low grade means missing work.  Perhaps the "C-" typically meant the car was low on gas, but we realize that there are lots of other cases as well.  We have given the tools to have a more engaged conversation about what is happening under the hood, but we also have to train students and parents to understand the information coming at them.

In addition to this, students and parents will need to understand sometimes you have to take the car into the mechanic.  Sometimes it is an easy fix.  Sometimes new habits will have to be formed for higher achievement. As teachers, we can explain what the oil pressure gauge is trying to communicate.  We can change the conversation.  It will take time, and understanding of where both sides are coming from.