**.**

*best*http://www.education.vic.gov.au/Documents/about/research/ravisiblelearning.pdf |

__their__classrooms, with

__their__unique students.

Until now.

In his book Visible Learning for Literacy (2016) that he co-authored with Fisher & Frey, Hattie encourages teachers to reflect on the impact of their own instruction and presents a formula for calculating effect size in their classrooms.

Being able to calculate impact in this way gives teachers the mathematical ability to quantitatively see if instruction is having an impact on

__their__students' achievement and who is not being impacted too. Armed with this information, teachers are able to adapt their teaching as to maximize their effectiveness for the benefit of all students in their classrooms.

In order to calculate effect size, Hattie suggests that:

- Lessons have clear
**learning intentions**. - Lessons have clear
**success criteria**. - The
**success criteria**indicate what quality looks like. - Students know where they stand in relation to the
**criteria**for**success**(p. 136).

With this in place, teachers only need a

Below, I demonstrate how to calculate effect size by analyzing students' thinking from an inquiry lesson I recently taught with third graders. The lesson is fully described here: Inquiry into Moon Phases.

During the lesson, our goal was to answer this essential question:

**pre-assessment**and a**post-assessment**score to be able to calculate effect size.Below, I demonstrate how to calculate effect size by analyzing students' thinking from an inquiry lesson I recently taught with third graders. The lesson is fully described here: Inquiry into Moon Phases.

__What did I want students to learn?__During the lesson, our goal was to answer this essential question:

__What would success look like?__
A successful response will ...

- Contain academic science vocabulary related to the lesson (1 pt awarded for inclusion of each of the following:
- observe, Earth, Moon, change, orbit, shadow, light, new moon, crescent moon, full moon, phases*, Sun*, reflecting*, darker*, lighter*
- *Not introduced during the lesson, but still important scientific terms that came out during the post-assessment.
- Contain different ideas (1 pt awarded for each complete idea)
- examples of complete ideas are: the moon orbits the earth, the moon reflects the sun's light, the shadow gets bigger as the light gets smaller, the full moon is when the moon is all it up).
- Be accurate
- 4 pts awarded for accurate statements with details/evidence,
- 2 pts awarded for semi-accurate statements with little details/evidence and some misconceptions
- 0 pts awarded for inaccurate statements with no details/evidence and many misconceptions

__How do I know they've learned?__Assessment task: The task for the pre-assessment (measure of what students initially understood, knew and could do) was the same as the post-assessment (measure of what they learned). For both, I prompted:

- "Write what you think the answer to our essential question is. Make sure to include scientific vocabulary in your response."

**(p 138)**

__To calculate effect size__**1. Analyze the pre- and post-assessments.**

This step was a snap, thanks to the pre-established success criteria. I recorded these results in a Google Sheet:

**2. Find the average of the pre- and post-assessments**

Using the average formula (=AVERAGE) this step was easy too!

- Average pre: 4.50
- Average post: 11.56

**3. Calculate the standard deviation for the pre- and post-assessment and then find the average of the two standard deviations.**

This step was super simple too, as the Standard Deviation formula is just (=STDEV).

- Standard Deviation pre: 2.28
- Standard Deviation post: 2.83
- Average Standard Deviation: 2.56

**4. Determine effect size**

Using Hattie's formula: (Average Post - Average Pre) / Average Standard Deviation

- Effect size: 2.76

This effect size is quite sizable and is most definitely off the scale of the Barometer of Influence that Hattie presents in his work. Some things to consider:

- Whereas this is a large effect size, it is just a number. With this quantitative data, a teacher should also reflect qualitatively:
- In what ways did the students grow the most?
- What about the lesson was successful that should be replicated?
- What wasn't successful that can be eliminated?
- This was just one lesson and the sample size is minute, compared to the studies Hattie typically meta-analyzes. Therefore, little relative importance should be placed on this effect size. After several weeks of working on making scientific observations using scientific language, another assessment could be administered to see if students have continued to show growth with this skill.

**5. Determine individual effect sizes**

Although the impact of this lesson was quite high o

*n average*, that is not necessarily the case for__all__students. By calculating individual effect sizes using individual assessment scores and the average Standard Deviation, you can see for whom this lesson was successful and for whom it was not.
The effect sizes of these individual students is also beyond the scale of Hattie's Barometer of Influence presented in his work. I don't believe that is important though. What

*is*important is to look at the individual effect sizes in relation to one another along with looking at students' thinking and reflect:- What causes one student (student F, for instance) to make sizable gains, whilst another student (like A or B) just grew marginally?
- What kinds of thinking are these students demonstrating?
- What about the teaching made such an impact on these students that could be replicated in the future?
- What about the teaching caused other students to not gain as much that should be avoided or adapted in the future?
- What do these students need next in their learning?

Below, I've included some samples of students' thinking:

**Student F Pre:****Student F Post:**

**Student N Pre:**

**Student N Post:**

**Student O Pre:**

**Student O Post:**

Although most students showed they learned a great deal during this lesson, Student P did not do as well on the post-assessment as he did on the pre. Using this quantitative data, a teacher must look more deeply at the student's response and reflect:

- What kinds of thinking is this student demonstrating?
- What about the teaching had a negative effect on this student's learning that should be avoided in the future?
- What does this student need next in their learning?

Student P Pre:

Student P Post:

Being able to calculate effect size to determine what works

**for individual students is powerful and has the potential to transform the ways which we respond to students. How could you use Hattie's math to determine the impact of your instruction on individual students' achievement?**

__best__
## No comments:

## Post a Comment