## Understanding Effect Size Through A Study On Homework

Part 2
A continuation from Part 1. As I study John Hattie’s comparison of the effect size of direct instruction vs inquiry learning and problem-based learning I am first trying to understand how to interpret is effect size calculations. Today we look at a quick study of what his effect size of homework looks like and means.

To further clarify how to interpret effect size, Hattie describes his examination of meta-analyses for how homework effects achievement. Hattie studied 5 meta-analyses from 1984, 1989, 1994, 1994, and 2006. These covered 161 studies and more than 100,000 students to analyze the effect on achievement of giving homework. After studying them all and calculating, Hattie came up with homework having an effect size of 0.29.

This means that Homework has a positive effect on student achievement because it has a value greater than 0. The question is how much positive effect? Hattie attempts to explain it the following ways:

1. Compared to classes without homework, the use of homework was associated with advancing children’s achievement by about one year.
2. Homework improved a child’s learning rate by 15%.
3. 65% of the effects were positive, and 35% of the effects were zero or negative.
4. The average achievement levels of students in classes that were given homework exceeded 62% of the achievement levels of the students in classes where homework was not given.

Once again, these do not appear to all be the same interpretation of the data, but apparently they are. Overall, this sounds positive to me. According to this study of meta-analyses it seems that giving homework is a good thing that raises achievement. However, Hattie advises that this is actually a very small improvement and barely noticeable.

Hattie quotes a statistician who helped to originally craft the idea of effect size for the social sciences: Jacob Cohen. Cohen describes the effect size of 1.0 to be like the height difference of between a person 5’3″ and someone else who is 6’0″. He is obviously illustrating that the difference is drastic and easy to see. The effect size of 0.29 however would be akin to a comparable height of 5’11” and 6’0″. Thus, he is attempting to illustrate that although there is a difference in students who experience a 0.29 effect size, it is barely noticeable.

This takes us to the ambiguous nature of effect size. How can 0.29 be both such a minor change that “would not be perceptible” (like the difference between 5’11” and 6’0″) and also reflect advancing a child’s achievement by 1 year, improving a child’s learning rate by 15%, and having achievement levels exceed 62% of their peers without homework?

So what are we to make of this? The effect size of homework is positive so you should use homework? The effect size is negligible so using homework is not worth it? Does a positive effect size mean use a strategy?

In the final installment of understanding effect size we will look at what Hattie deems is optimum value to utilize and why. Hopefully we can gain an understanding of what Hattie thinks we should utilize and why. However, if the interpretation of what a 0.29 effect size means is this ambiguous then I do not hold much hope for moving forward.

## Understanding Effect Size In John Hattie’s Research

Part 1

As I begin my research and study John Hattie’s claim that Direct Instruction has a far greater effect size compared to Inquiry-Based Learning or Project-Based Learning, I thought it was best to first make sense of what is meant by “effect-size.”

When understanding effect size according to John Hattie it is best to think of a sliding scale from -1 to 1 (although values over/under this are possible). An effect size of 0 would mean that a particular method has no effect on achievement, a negative score means it actually reduces achievement, and a positive score means it increases achievement. To get specific, an effect size of 1.0 would mean that a child’s achievement would advance 2-3 years or increase their learning rate by 50%.

This seems hard to believe as an advancement of 2-3 years seems insane considering Hattie does list several methods that have a 1.0 effect size or higher – like self reporting grades. “Self-report grades” is actually listed as an effect size of 1.44. Does this mean that employing this method would almost double your student’s achievement or raise the student achievement level by more than 3 years?

It quickly becomes clear that understanding Hattie’s effect size calculations seem to have too wide of an interpretation. He goes on to say that an effect size of 1.0 can be interpreted 4 different ways:

1. A child’s achievement would advance 2-3 years.
2. Their learning rate would increase by 50%.
3. A correlation between a variable (like the amount of homework) and achievement was approximately a rate of .5 or 50%.
4. Students would exceed 84% of students not receiving the “treatment” or method.

These interpretations do not appear to be the same yet could all possibly be used to interpret a 1.0 effect size. He attempts to make it more understandable to a layman by paraphrasing a statistician who helped to originally craft the idea of effect size for the social sciences: Jacob Cohen. Cohen describes the effect size of 1.0 to be like the height difference between a person 5’3″ and someone else who is 6’0″. He is obviously illustrating that the difference is drastic and easy to see.

Confused yet? Hattie’s breakdown of effect size leaves a lot to be desired. The sad part so far is that his research attempts to quantify a large amount of achievement differences and compare the results of a large list of strategies and methods. Teachers and administrators everywhere simply show the list of methods compared by effect size without actually defining what those effect sizes mean.

I think if we look at the effect size of homework we can use that to further understand what Hattie is trying to get at. That is coming in Part 2 later this week.

All paraphrasing and quoting comes from Visible Learning by John Hattie, 2009. Pages 7-8.

## Resurrecting The Blog

This school year like most years has been a journey into trying to improve myself as a Math Teacher. However, this year more so than others, has been above what I normally do. I have sought out other math teachers in the #MTBoS (math-twitter-blog-o-sphere) and looked to try new ideas, practices, teaching strategies, lessons, and more.

Nothing felt truly “blog worthy” until I came across our school district’s current pension for John Hattie’s research. My school district, like many others, focuses on a common educational researcher and uses their research as a focal point for professional development throughout the year.

This is the first time I have really looked into John Hattie and while I was sitting in a PD meeting reading a list of his teaching strategies and disciplines that have the largest “effect size” I came across a discrepancy according to my recent research.

Hattie ranked the teaching strategy of “direct instruction” (0.59) as having almost twice the effect of “inquiry-based learning” (0.31) and almost four times the effect of “problem-based learning” (0.15).

Over my recent research I had not come across someone blatantly stating that it is better to utilize direct instruction than elements of inquiry-based or problem-based learning. I recognize that education is a pendulum that swings from side to side, but under current research I thought practices of constructivism were superior.

So this is what takes us to the resurrection of the blog. I will be jumping into plenty of research on this topic over the following months and rather than keep all my findings in my iPad I think it would be better to publish them on here for the reference of others.

I look forward to being challenged, to find out if I am wrong, and/or if I am simply in a grey area and there are elements of truth in both camps. Either way, I must know more. Until the next post.

## Practical Ways To Persevere In Problem Solving

The first Common Core Standard for Mathematical Practice calls for students to “Make Sense of Problems and Persevere In Solving Them.” It should go without saying that the benefits of having students persevere in solving a problem are very obvious. Ideally you want to challenge your students with multi-step problems that require planning and problem solving. When you challenge them you want them to break through any walls they hit and make sure giving up is not an easy option.

Naturally we all want this of our students, but I know many teachers who are left scratching their heads saying, “How do I teach my students to persevere.” I completely understand where this thought comes from as most people would view perseverance as a personality trait that cannot be taught.

I would however argue that there are some practical things you can do to impart perseverance to your students. The first is a very simple idea that many of you may already be doing, but not realizing the power it has. Use personal dry-erase boards.

There are the standard boring white boards or there are the new clear boards that you can put in template sheets which are becoming all the rage. The only true common denominator that matters though is a surface that can handle dry-erase markers. I find that students are much more willing to try something again if they can easily and quickly erase and write with a marker.

The only drawback of using marker boards are the temporary nature of dry-erase. To combat this I sometimes have students write down a final working out of a problem so they can look back at it later. If you are in a very technologically hip class then you can take pictures of the boards with iPads or iPod Touches… or cameras.

A dry-erase board and marker does not guarantee perseverance, but it does give students less excuses.

Another very real, tangible way to show students how to persevere is to make sure they know various strategies to problem solve. I often find myself very confused that a student doesn’t think to draw a picture to help them think through a problem or make a table of values. There are basic problem solving strategies that if students are familiar with can give them another vantage point to see the problems from.

I would recommend at the beginning of the school year going through several problem solving strategies that can be used in a variety of problems (a list is farther down). After showing them how to use each strategy it might be a good idea to either create posters or have the students create posters of the strategy to display in your room.

In the future when a student gets stuck hitting a wall and is about to give up you can point to the posters on the wall and say, “which strategies have you tried?” “Where’s your picture or diagram?” This will hopefully give students a new lease to try something new and different. To possibly look at the problem from a different angle using a new strategy.

Some strategies I have seen are: draw a diagram, make a model, guess and check, work backward, find a pattern, make a table, solve a simpler problem, act it out, make an organized list. I’m sure this is not a conclusive list and you could argue that some of these are repeats (I would). So I am not advising you use all of them. Pick a solid 4-6 to create a problem solving foundation for your students and ensure that your students use them to help increase their stamina.

Have any other thoughts – I would love to hear them!

## Timing and Planning For Time

One of the most frustrating things for me this year is having to deal with a 20 minute loss of time in my Math class. I’ve had to go from 90 minutes to 70. The difficult part is I know for some people 70 minutes would be a luxury as there are teachers who only teach 45 minutes or so. So I hate to complain but I’ve been battling all year to readjust my timing on everything to make everything fit. Bellwork, homework check/review, lesson, practice, exit slip – they don’t all fit in everyday and it is frustrating.

I’ve even caught myself cutting off some students questions towards the end of class cause I have to finish the lesson so students know how to do their homework. Teaching a lesson up until the bell and not giving students time to start the work in class is really frustrating for me. Plenty of students do not have parents at home to help them or if they do I often get the “I’m not good enough at math to help them” line.

The saddest part so far is that I simply don’t have answers. I try to cut things, move fast here and slow down there, and I still feel like I am fumbling through. Flexibility is an important characteristic of a teacher, but it is definitely had to change an internal clock that has chimed regularly for the past 4 years.

My hope is by the end of this year I will have tried enough variations to figure out a system that will work for next year… here’s hoping.

## New iPad Textbooks and Math Instruction

I want to write about what I think the iPad and interactive Math textbooks can do for students – but I can’t. I love Apple and pretty much all their products. I love what they are trying to do, but the problem will always be that they are a corporation trying to make money. I work at an inner ring urban suburb that has a fairly high poverty rate. In our middle school we have 3 grades that make up around 1000 students. For 1000 students we have 1 computer lab and 1 iPad cart that can be signed up for by the 50+ classrooms.

My district is no where even close to carrying out Apple’s vision of an iPad in the hands of every student filled with interactive text books.  It is not my district’s fault. Apple doesn’t cut insane deals on iPads, schools have to pay almost the full \$500 price tag per iPad and then invest the thousands of dollars to fill them up with Apps and Books. How do districts have a chance?

Apple is only pushing a wedge that is separating the monetary affluent schools from the schools that simply aren’t. It is frustrating and sad, but we will continue to teach and reach out to kids no matter what technology is in our classrooms.

## Data Collection

Data collection is all the rage these days in education. There is the macro version of data collection where we mostly follow how students do on standardized testing and track their performance on it. More and more the micro version of data collection is happening where teachers are tracking students performance on individual indicators and making spreadsheets with rows and rows of numbers that are supposed to tell you how that student is doing. I don’t want to sound to sarcastic here because I really do see the merit and importance of collecting, tracking and following up on data with students. We are not doing our job unless our students are growing and when we complain that a single test score doesn’t always do the growth justice, then we need to prove the growth in other ways.

The fear though is that students are no longer young people with faces and personalities, but numbers on a spreadsheet. A lot of this comes in when you are expected to report your data back to others or you are talking about your data. Your students become numbers – we are distilling the job down to its most basic form to compare.

I don’t know the alternative to this, but just be wary that when you are thinking of your classroom, how you teach, and who you teach – make sure that you are thinking amply about your students and their needs and not just the numbers on a spreadsheet.

## Using an Activity To Teach

January 14th, 2012 § 1 comment § permalink

I recently created a real-world lesson about taking off consecutive percents vs. taking off the same discount in one step (20% then 20% again vs. 40% off the original price). I often call this stacking percents. Rather than go through a normal lesson day of notes, examples and practice and then do this activity the next day – I decided to try using this lesson to teach students about the difference. The results were mixed but positive overall.

The big point of the lesson is trying to get students to see that taking the discount off in two steps will get a different result compared to taking it off in one step. This lesson had students take a more inquiry based approach (which I will be writing on plenty in the future) as most students assumed that the discounts would result in the same price.

Doing it through the activity allowed them to find out which one saved more and conjecture about why it saved more. This worked out really well for my moderate to advanced students and not quite as well for my low students. This is partially my fault as my low students were struggling to find the percent discounts as I did not properly scaffold this skill for them – definitely something I will change in the future. I like this method of approach for students that have a firm grasp of content as it allows them to explore the learning possibilities themselves. I will continue to experiment with this approach and see what role it can take in an every-day class environment.

## The Math Of Minecraft

A student I tutor is in love with the online game Minecraft.  It is a game in which you mine and build things out of cubes. There is a basic free version which simply gives you the ability to build in a free open range and then a paid version that adds gaming elements. We used the free version which is accesible within the browser so it can probably be used in most schools if the computer is updated with Java (and its not blocked). We worked on a lesson with volume and surface area that went so well I will probably be formally writing it up and posting it on MakeMathMore’s Real-Life Math Lessons. It is a great tool as you can freely walk around and use cubes which work great for building rectangular prisms that allowed us to easily calculate volume and surface area.

This is the reason I look for Math everywhere as you never know where you will find something that will make a perfect teaching lesson.

## Getting Students To Recognize What Is Challenging

Today in preparation to correct a test I had students go through and identify which problems on a test were difficult and describe to me why they were difficult. I think this is a really important skill to grab a hold of as it makes you a better learner. If you can identify what is challenging about a problem, then you can narrow your focus on what to look at, ask questions about, or study.

The problem is I have discovered this is a skill I need to teach to my students. First, I think students are often over confident in what they find challenging. They are quick to assume they got something right – especially on multiple choice tests. Secondly, they have difficult putting into words why a problem is difficult. They can act like it should be assumed that because they are not able to do the problem then of course they cannot tell my what’s difficult about it. They equate knowing what’s difficult or challenging with knowing how to solve. This is a problem.

Students need to recognize what they know about a problem, how to try it, and then when they can’t come up with a right answer they need to recognize where the difficulty lies. What is the hurdle that they cannot get over. Even if it’s as simple as ‘I know I am solving the proportion right, so I must be setting it up wrong” or “I am unsure if this is the correct number for the denominator.”

We need to get students to question themselves – not to the point of self doubt, but to make a more complete learner.