This isn't directly MY success, but was a very neat thing to see.
I took on the task of a home-bound instructor for an ill student. During my first visit, we had some math to do. We worked through a concept review activity which would help formatively assess where she was to see if she was ready to move forward or to get more instruction. As part of this activity, the math teacher had written the concept goals as well as which questions would tell the teacher and student the level of understanding. After the student was done, we looked it over together and had decided that she understood the Level 2 questions perfectly, but was struggling on a few specific parts of the Level 3 things. Those struggles made approaching the Level 4 question quite challenging.
Today, I went to school and chatted with the math teacher. I told her I was going to "test" her, the teacher, on the 4-Level rubric with this student. I told her just what I said above, solid on Level 2, struggling but almost there on Level 3. She replies, "Well, she must be able to set up the ratios and proportions using whole numbers, but must be having troubles when the scaling factors have decimals."
HOLY COW! Exactly right.
In the end, the goal of grading is to communicate. This teacher has set up a system that works. Clear goals matched with quality assessment. Awesome. As an "in-between" teacher in this case, the system worked nicely too as I could easily give the student some specific practice that will allow her to make that tiny step to be at a Level 3.
Showing posts with label Grading. Show all posts
Showing posts with label Grading. Show all posts
Wednesday, December 12, 2012
Tuesday, June 12, 2012
Reflection and goals
For me, the end of the school year is always one of reflection, but also one of goals and ideas for next year. When our final day was done last year, I was all jazzed up. Not for summer, although there are a ton of fun things happening there. School starts in 84 days. I think I get more excited about the next school year starting as most do about the last one ending.
I looked at my goals a bit differently this year. Previously, in my head or on paper, I had some ideas. They were not quality goals. This year is a bit different. Over the last two school years, I have spent quite a bit of time studying John Hattie's book: Visible Learning. Hattie's book is almost entirely focused on achievement, and many of my goals in the past, although not greatly defined, were also about achievement. As I read and studied his book, I had a lot of ideas about strategies that I could/should use in my class, but I wondered how I could use his overall message about effect size. The basic idea is that a treatment's effect can be studied by comparing achievement results pre and post treatment. Then, by creating an effect size, different treatments can be compared to each other. This is typically done with meta-analyses, or collections of many studies typically with many students.
I wanted to use the idea in my class, but like all things, a good use wasn't readily apparent. I attempted, last year, to use pre-assessment data and effect sizes to make goals for students to achieve for their post assessments. I didn't like how that turned out. Too often with pre-assessments, the ideas are so new that almost any real gain will produce a very large effect size. That may help my ego, but it didn't help with real goal setting.
During the 2011-2012 school year, I finally worked out a grading system that worked well. I used a 4-point scale with well defined levels. Basically, I used a modified Marzano's scale:
4: Students understand the simple and complex ideas, concepts, and processes that were taught in class AND show the ability to make in-depth inferences and applications beyond what was taught in class.
3: Students understand the simple and complex ideas, concepts, and processes that were taught in class.
2: Students understand the simple ideas, concepts, and processes, but have trouble with the complex ones.
1: Students do not understand the simple ideas, concepts, and processes.
0: No evidence.
In addition to this, each assessment was explicitly connected to one or more pre-defined learning targets. I then used ActiveGrade as a place to hold and communicate this data with students and parents. What this gave me, finally, was a wealth of data at the end of the year. For each learning target that we had written out in our curriculum, I could tell you how well students did, in terms of grades anyway. This shouldn't be a huge accomplishment, and I am embarrassed that it took me this long to achieve this. The messed up way I have graded in the past, along with a poorly defined curriculum, I didn't have the tools to create quality data. I finally feel like I have some, and frankly, I am a bit proud of myself!
Now, what to do with this data? At first, I sort of stood back, rubbed my chin, and gave a nod of approval. Nods of approvals and patting one's back doesn't really help kids much though, so what could I do now. This is where Hattie came to the rescue. I finally have a baseline data set. I can now use that to set a goal for next year, and have benchmarks in which to gauge success. To create goals, the first thing I did was to find an average of grades for each learning target. Hattie says the "hinge-point" for success is an effect size of 0.4. I set that as my goal for next year, and ran numbers for each learning target assuming this effect size. Then, I personally reflected on what I would need to do to get students to that level.
Of course, our goal for all students should be a "4." What I really want for all of my students is to understand what we do in class and then use it in a new context. The problem is that not all students achieve that. I decided previously that all students *must* achieve at least a "2" in all learning targets, as a bare minimum, and that goal was achieved. I like how I can use Hattie's recommendation to set a, hopefully realistic, goal for next year.
Below is an Excel document that shows the learning targets, current achievement, and goals.
Each learning target is assessed multiple times throughout the school year. Next year, as a department, we will do some clarification on these. Also, I have reflected on the number of times each learning target was assessed. Lots of work to do this summer, but I feel like I have a great starting point.
I looked at my goals a bit differently this year. Previously, in my head or on paper, I had some ideas. They were not quality goals. This year is a bit different. Over the last two school years, I have spent quite a bit of time studying John Hattie's book: Visible Learning. Hattie's book is almost entirely focused on achievement, and many of my goals in the past, although not greatly defined, were also about achievement. As I read and studied his book, I had a lot of ideas about strategies that I could/should use in my class, but I wondered how I could use his overall message about effect size. The basic idea is that a treatment's effect can be studied by comparing achievement results pre and post treatment. Then, by creating an effect size, different treatments can be compared to each other. This is typically done with meta-analyses, or collections of many studies typically with many students.
I wanted to use the idea in my class, but like all things, a good use wasn't readily apparent. I attempted, last year, to use pre-assessment data and effect sizes to make goals for students to achieve for their post assessments. I didn't like how that turned out. Too often with pre-assessments, the ideas are so new that almost any real gain will produce a very large effect size. That may help my ego, but it didn't help with real goal setting.
During the 2011-2012 school year, I finally worked out a grading system that worked well. I used a 4-point scale with well defined levels. Basically, I used a modified Marzano's scale:
4: Students understand the simple and complex ideas, concepts, and processes that were taught in class AND show the ability to make in-depth inferences and applications beyond what was taught in class.
3: Students understand the simple and complex ideas, concepts, and processes that were taught in class.
2: Students understand the simple ideas, concepts, and processes, but have trouble with the complex ones.
1: Students do not understand the simple ideas, concepts, and processes.
0: No evidence.
In addition to this, each assessment was explicitly connected to one or more pre-defined learning targets. I then used ActiveGrade as a place to hold and communicate this data with students and parents. What this gave me, finally, was a wealth of data at the end of the year. For each learning target that we had written out in our curriculum, I could tell you how well students did, in terms of grades anyway. This shouldn't be a huge accomplishment, and I am embarrassed that it took me this long to achieve this. The messed up way I have graded in the past, along with a poorly defined curriculum, I didn't have the tools to create quality data. I finally feel like I have some, and frankly, I am a bit proud of myself!
Now, what to do with this data? At first, I sort of stood back, rubbed my chin, and gave a nod of approval. Nods of approvals and patting one's back doesn't really help kids much though, so what could I do now. This is where Hattie came to the rescue. I finally have a baseline data set. I can now use that to set a goal for next year, and have benchmarks in which to gauge success. To create goals, the first thing I did was to find an average of grades for each learning target. Hattie says the "hinge-point" for success is an effect size of 0.4. I set that as my goal for next year, and ran numbers for each learning target assuming this effect size. Then, I personally reflected on what I would need to do to get students to that level.
Of course, our goal for all students should be a "4." What I really want for all of my students is to understand what we do in class and then use it in a new context. The problem is that not all students achieve that. I decided previously that all students *must* achieve at least a "2" in all learning targets, as a bare minimum, and that goal was achieved. I like how I can use Hattie's recommendation to set a, hopefully realistic, goal for next year.
Below is an Excel document that shows the learning targets, current achievement, and goals.
Each learning target is assessed multiple times throughout the school year. Next year, as a department, we will do some clarification on these. Also, I have reflected on the number of times each learning target was assessed. Lots of work to do this summer, but I feel like I have a great starting point.
Thursday, April 12, 2012
A car without a dashboard
Recently, I ran across a few analogies between grades and vehicle dashboards. (See Here and Here). These are interesting comparisons that I ended up thinking about even more. I spend about one hour a day driving. I adore this hour for the contemplation time, definitely worth the gas spent. With my transition to a standards-based grading system, I have felt a mixture of emotions from parents. The whole spectrum is observed. Some parents think the change is good and helpful, others don't care one way or another, and some are frustrated. This post is an attempt to collect my thoughts for the frustrated parent.
Imagine our cars did not have gauges within their dashboards. Instead, displayed on the other side of the steer wheel, would be a letter: This letter ranged, just like grades, from an "A" to an "F." We will even put pluses and minuses in for fun too. What does the letter mean? We think we know, but in the end "A" is good, and "F" is bad. Anything in between is, well, in-between. We don't know how the overall grade is figured. Every model is different. Some manufactures weigh speed 70%, while others have oil pressure as the most important.
When you drive your car, the letter changes. There is no speedometer, odometer, tachometer, gauges for temperature, fuel level, oil pressure, or amperes. No extra warning lights for low fuel or when sensors detect issues. Only the final grade.
So, when you are driving along, all is well so long as the "A" stays on the dashboard.
Imagine lending your car out to your teenager. When they leave, the car is "A+." Upon returning, "C-." Someone is in trouble! But, for what? Is the engine out of oil? Low on gas? At this point, we do not know anything. Now, you have a well equipped car with all the bells and whistles. You can actually go online to checkyourcarsgrade.net and see what actually happened mile by mile. You find out that around 10pm last night, the cars grade was actually an "F" for awhile!
Also imagine that you just started your car, and it's cold outside. The temperature of the engine is well below where it should be, so to start out, your car reads an "F." You hope that upon warm up, the grade rises. What if it doesn't?
Obviously, this scenario would drive most people nuts. The crazy thing is that grades in our classrooms have been this way for years, decades, and generations. Along the way, we give grades that are connected to assignments that are oddly added up to give an overall class grade. We think we know what the grade means, but in the end, "A's" are good and "F's" are bad. There isn't any consistency as to how the overall grade is figured. Part of the game is for students to figure that out as we go along. We can look at the grades in real time with different internet tools, but the assessments are disconnected from the learning. A "B+" on Green Worksheet doesn't help any more than a "C-" on the car dashboard at 9pm.
A car's gauges could be looked at as learning targets. We have a range that is optimal, a range that is concerning, and a range that is just flat dangerous. Some of the gauges are easy for us as operators to understand and regulate. If the tachometer reads to high, I simply can shift to a higher gear or let off the accelerator. If the fuel is low, I fill up. Others may need further investigation. If my oil pressure is low, I may bring my car into a mechanic. When the service engine light comes on, I know more diagnostics need to be done.
When we make a switch to standards-based grading, we are asking students and parents to make a tough switch as well. They have learned to drive the car with only a letter grade appearing. No, it is not ideal. It is not efficient. The feedback is poor. Change is difficult. As hard as it would be for us to begin driving a car without gauges, it is just as hard for students and parents to play school with the additional information. We have to help out on that end. We have to, for one, show that it is OK to have more information. We have to help make the switch from the assumption that a low grade means missing work. Perhaps the "C-" typically meant the car was low on gas, but we realize that there are lots of other cases as well. We have given the tools to have a more engaged conversation about what is happening under the hood, but we also have to train students and parents to understand the information coming at them.
In addition to this, students and parents will need to understand sometimes you have to take the car into the mechanic. Sometimes it is an easy fix. Sometimes new habits will have to be formed for higher achievement. As teachers, we can explain what the oil pressure gauge is trying to communicate. We can change the conversation. It will take time, and understanding of where both sides are coming from.
Imagine our cars did not have gauges within their dashboards. Instead, displayed on the other side of the steer wheel, would be a letter: This letter ranged, just like grades, from an "A" to an "F." We will even put pluses and minuses in for fun too. What does the letter mean? We think we know, but in the end "A" is good, and "F" is bad. Anything in between is, well, in-between. We don't know how the overall grade is figured. Every model is different. Some manufactures weigh speed 70%, while others have oil pressure as the most important.
When you drive your car, the letter changes. There is no speedometer, odometer, tachometer, gauges for temperature, fuel level, oil pressure, or amperes. No extra warning lights for low fuel or when sensors detect issues. Only the final grade.
So, when you are driving along, all is well so long as the "A" stays on the dashboard.
Imagine lending your car out to your teenager. When they leave, the car is "A+." Upon returning, "C-." Someone is in trouble! But, for what? Is the engine out of oil? Low on gas? At this point, we do not know anything. Now, you have a well equipped car with all the bells and whistles. You can actually go online to checkyourcarsgrade.net and see what actually happened mile by mile. You find out that around 10pm last night, the cars grade was actually an "F" for awhile!
Also imagine that you just started your car, and it's cold outside. The temperature of the engine is well below where it should be, so to start out, your car reads an "F." You hope that upon warm up, the grade rises. What if it doesn't?
Obviously, this scenario would drive most people nuts. The crazy thing is that grades in our classrooms have been this way for years, decades, and generations. Along the way, we give grades that are connected to assignments that are oddly added up to give an overall class grade. We think we know what the grade means, but in the end, "A's" are good and "F's" are bad. There isn't any consistency as to how the overall grade is figured. Part of the game is for students to figure that out as we go along. We can look at the grades in real time with different internet tools, but the assessments are disconnected from the learning. A "B+" on Green Worksheet doesn't help any more than a "C-" on the car dashboard at 9pm.
A car's gauges could be looked at as learning targets. We have a range that is optimal, a range that is concerning, and a range that is just flat dangerous. Some of the gauges are easy for us as operators to understand and regulate. If the tachometer reads to high, I simply can shift to a higher gear or let off the accelerator. If the fuel is low, I fill up. Others may need further investigation. If my oil pressure is low, I may bring my car into a mechanic. When the service engine light comes on, I know more diagnostics need to be done.
When we make a switch to standards-based grading, we are asking students and parents to make a tough switch as well. They have learned to drive the car with only a letter grade appearing. No, it is not ideal. It is not efficient. The feedback is poor. Change is difficult. As hard as it would be for us to begin driving a car without gauges, it is just as hard for students and parents to play school with the additional information. We have to help out on that end. We have to, for one, show that it is OK to have more information. We have to help make the switch from the assumption that a low grade means missing work. Perhaps the "C-" typically meant the car was low on gas, but we realize that there are lots of other cases as well. We have given the tools to have a more engaged conversation about what is happening under the hood, but we also have to train students and parents to understand the information coming at them.
In addition to this, students and parents will need to understand sometimes you have to take the car into the mechanic. Sometimes it is an easy fix. Sometimes new habits will have to be formed for higher achievement. As teachers, we can explain what the oil pressure gauge is trying to communicate. We can change the conversation. It will take time, and understanding of where both sides are coming from.
Subscribe to:
Posts (Atom)