## Who Assessed it Better? AP Stats Inference Edition

Free-response questions and exam information in this post freely available on the College Board – AP Statistics website

Today I am stealing a concept from Dan Meyer’s task comparison series “Who Wore it Best”, and bringing it to the AP Statistics exam world. In the series of 6 free-response questions on the AP Stats exam, it is not unusual for one question to focus solely on inference. Compare these two questions, which each deal with inference for proportions.

From 2012:

From 2016:

I read (graded) the question from 2012 as an AP Exam reader, and observed a variety of approaches. I find that while many students understand the structure of a hypothesis test, it’s the nuance – the rationales for steps – which are often lost in the communication. In the 2012 question, students were expected to do the following:

1. Identify appropriate parameters
2. State null and alternate hypotheses
3. Identify conditions
• Independent, random samples and normality of sampling distribution
4. Name the correct hypothesis procedure
5. Compute / communicate test statistic and p-value
6. Compare the p-value to an alpha level
7. Make an appropriate conclusion in context of the problem

It’s quite a list.  And given that individual AP exam problems are worth a total of 4 points, steps are often combined into one scoring element.  Here, naming the test and checking conditions were bundled – as such, precision in providing a rationale for conditions was often forgiven.  For example, if students identified the large sample sizes as a necessary condition, this was sufficient, even if there was no recognition of a link to normality of sampling distributions.  Understanding the structure of a hypothesis test – with appropriate communication – was clearly the star of the show.  While inference is one the “big ideas” in AP Statistics, my view is that questions like this from 2012 encourage cookbook statistics, where memorized structure take the place of deeper understanding.

So, it was with much excitement that I saw question 5 from 2016.  Here, the interpretation of a confidence interval was preserved. But I appreciate the work of the test development committee in parts b and c; rather than have students simply list and confirm conditions for inference, the exam challenges students to be quite specific about their rationales. With parts b and c, students certainly struggled more than with the conditions in 2012, but I hope their inclusion causes statistics teachers to consider their approach to hypothesis conditions. The mean scores for each question speak to struggles students on this question, compared to traditional hypothesis testing structure.

• 2012: 1.56 (out of 4)
• 2016: 1.27 (out of 4)

The inclusion of part b of question 5 this year, where students were asked to defend the np > 10 condition, was perfect timing for my classes.  This year, I tried a new approach to help develop student understanding of the binomial distribution / sampling distribution relationship. I found that while many students will continue to resort to the “short cut” – memorizing conditions – a higher proportion of students were able to provide clear communication of this inference condition.

The AP Statistics reading features “Best Practices Night” – where classroom ideas are shared.  You can find resources from the last few years at Jason Molesky’s APStatsMonkey site. I shared my np > 10 ideas with the group, and received many positive comments about it.  Enjoy my slides here, and feel free to contact me with questions regarding this lesson:

Finally, I can’t express how wonderfully rich a professional-development experience the AP reading is.  I always find myself with a basket of new classroom ideas and contacts to share with – it’s stats-geek Christmas.  For me, 2016 is the year the #MTBoS started to make its mark at the Stats reading – I met so many folks from Twitter, and we held our first-ever tweet-up!

Also, the vibrant Philadelphia-area stats community was active as always.  We meet as a group a few times a year to share ideas and lessons; seeing so many from this area participate in the reading makes us all better with what we do for our students.

## The People in My Math Neighborhood

Oh, who are the people in your neighborhood?
Say, who are the people in your neighborhood?
The people that you meet each day

I work with an awesome group of people at a high school outside of Philadelphia.  They are my colleagues, the people I share ideas with on a daily basis, and some of my closest friends.

But in recent years, my math neighborhood has grown considerably.  I suppose I discovered the power of the online neighborhood 4 or 5 years ago, developing and growing a wonderful network of professional colleagues through the #MTBoS. And my relationship with this neighborhood has grown from a mechanism for sharing ideas, to a source of inspiration, positive thought, discussion and reflection.

We are now 3 weeks after the NCTM Annual Conference in San Francisco. It’s easy to forgot the little things which occur in a big conference, and I hopefully will find time to reflect and utilize new ideas later. NCTM this year has done a wonderful job of providing a means to continue reflections and growth outside of the conference, along with archiving session resources.  Here, I highlight 4 sources of inspiration, and friends in my math neighborhood, as I look back on my San Francisco experiences.

GRAHAM FLETCHER – Graham, an elementary specialist from Georgia (or is he Canadian? such a chameleon), challenged teachers to consider the mathematical story we share with students in his ShadowCon talk. How is your story different than the one being told by your colleague teaching the same material just across the hall?

High school teachers may be intrigued by Graham’s discussion of fractions, reducing and equivalence and the role of “simplifying”. His talk has caused me to think about the many odd restrictions we place on student work: i.e. “write the equation of your line in standard form”, and their necessity in my math story. Graham’s call to action – challenging teachers to identify their own “simplifying fractions” (something they teach not currently in the standards) – is an appropriate task for all grades.

ROBERT KAPLINKSY – Robert was featured on the MathEd Out podcast last summer, and I recall taking a walk, listening last year when it occurred to me that Robert’s path to becoming a math teacher was eerily similar to mine. His ShadowCon talk, “Empower”, reminds me that no matter how top-down our education world may feel, we all have a role to empower others and become influential in our math neighborhoods. I appreciate the multiple mechanisms Robert suggests for fostering empowerment, and his call to action that we thank a colleague who helped us feel empowered is a wonderful way to close out a school year – and look forward to new things.

PEG CAGLE – I have admired Peg’s ideas for some time now, and was thrilled to meet and chat with her last year at Twitter Math Camp. Even though I rarely teach geometry, I felt pulled to Peg’s session “Paper Cup + Gust of Wind”, and was awed by the simplicity, engagement, theme-building in this simple task. By rolling a paper Dixie cup along a surface, Peg develops a lesson which extends through the school year, building complexity each time.

Day 1:Explain what happens when we roll out the cup

Day 40: Convince a skeptic of the shape it makes. Find its area.

Day 105: find area of shape based on dimensions

Day 140: How can you build a cup from a single sheet (with base) of 8.5 x 11 paper to trace out the maximum area as it rolls?

Day 175 (after trig ratios): how do you NOW find the area of the shape, given its dimensions

This session has caused me to think about other simple tasks which could become full-course themes. Peg’s inspiration came from a cup blowing in the breeze – you never know where the next fun math idea will come from!

CHRISTINE FRANKLIN – Why was I so nervous and awe-struck to meet Christine at the AP Stats forum in San Francisco? Because she is so awesome – and was the inspiration for my NCTM talk on Variability and Inference, geared towards the middle school community. It was at Professional Night at the AP Stats reading 2 years ago where Christine diagrammed the historical path stats has taken in K-12 curriculum, and the parallels between AP and middle school descriptions. Christine was recently named the K-12 statistical ambassador by the ASA, and a sweeter person could not fill the job.

Hoping I never move out of the neighborhood!

## Activity Builder Reflections

We’re now about 9 months into the Desmos Activity Builder Era (9 AAB – after activity-builder). It’s an exciting time to be a math teacher, and I have learned a great deal from peeling apart activities and conversing with my #MTBoS friends (run to teacher.desmos.com to start peeling on your own – we’ll wait…). In the last few weeks, I have used Activities multiple times with my 9th graders.  To assess the “success” of these activities, I want to go back to 2 questions I posed in my previous post on classroom design considerations, specifically:

• What path do I want them (students) to take to get there?
• How does this improve upon my usual delivery?

AN INTRODUCTION TO ARITHMETIC SERIES (click here to check out the activity)

My unit or arithmetic sequences and series often became buried near the end of the year, at the mercy of “do we have time for this” and featuring weird notation and formulas which confused the kids. I never felt quite satisfied by what I was doing here.  I ripped apart my approach this year, hoping to leverage what students knew about linear functions to develop an experience which made sense. After a draft activity which still left me cold, awesome advice by Bowen Kerins and Nathan Kraft inspired some positive edits.

In the activity, students first consider seats in a theater, which leads to a review of linear function ideas. Vocabulary for arithmetic sequences is introduced, followed by a formal function for finding terms in a sequence. It’s this last piece, moving to a general rule, which worried me the most.  Was this too fast?  Was I beating kids over the head with a formula they weren’t ready for? Would the notation scare them off?

The path – having students move from a context, to prediction, to generalization, to application – was navigated cleanly by most of my students.  The important role of the common difference in building equations was evident in the conversations, and many were able to complete my final application challenge.  The next day, students were able to quickly generate functions which represent arithmetic sequences, and with less notational confusion than the past.  It certainly wasn’t all a smooth ride, but the improvement, and lack of tooth-pulling, made this a vast improvement over my previous delivery.

DID IT HIT THE HOOP? (check out the activity)

Dan Meyer’s “Did It Hit the Hoop” 3-act Activity probably sits on the Mount Rushmore of math goodness, and Dan’s recent share of an Activity Builder makes it all the more easy to engage your classes with this premise. In class, we are working through polynomial operations, with factoring looming large on the horizon.  My 9th graders have little experience with anything non-linear, so this seemed a perfect time to toss them into the deep end of the pool.  The students worked in partnerships, and kept track of their shot predictions with dry-erase markers on their desks. Conversations regarding parabola behavior were abundant, and I kept mental notes to work their ideas into our formal conversations the next day.  What I appreciate most about this activity is that students explore quadratic functions, but don’t need to know a lick about them to have fun with it – nor do we scare them off by demanding high-level language or intimidating equations right away.

The next day, we explored parabolas more before factoring, and developed links between standard form of a quadratic and its factored form. Specifically, what information does one form provide which the other doesn’t, and why do we care?  The path here feels less intimidating, and we always have the chance to circle back to Dan’s shots if we need to re-center discussion.  And while the jury is out on whether this improves my unit as a whole, not one person has complained about “why”…yet.

MORE ACTIVITY BUILDER GOODNESS

Last night, the Global Math Department hosted a well-attended webinar featuring Shelley Carranza, who is the newest Desmos Teaching Faculty member (congrats Shelley!).  It was an exciting night of sharing – if you missed it, you can replay the session on the Bigmarker GMD site.

## When Binomial Distributions Appear Normal

We’re working through binomial and geometric distributions this week in AP Stats, and there are many, many seeds which get planted in this chapter which we hope will yield bumper crops down the road. In particular, normal estimates of a binomial distribution – which later become conditions in hypothesis testing – are valuable to think about now and tuck firmly into our toolkit.  This year, a Desmos exploration provided rich discussion and hopefully helped students make sense of these “rules of thumb”.

Each group was equipped with a netbook, and some students chose to use their phones. A Desmos binomial distribution explorer I had pre-made was linked on Edmodo. The explorer allows students to set the paremeters of a binomial distribution, n and p, and view the resulting probability distribution. After a few minutes of playing, I asked students what they noticed about these distributions.

A lot of them look normal.

Yup. And now the hook has been cast.  Which of these distributions “appear” normal, and under what conditions?  In their teams, students adjusted the parameters and assessed the normality.  In the expressions, the normal overlay provides a theoretical normal curve, based on the binomial mean and standard deviation, along with error dots. This provides more evidence as students debate normal-looking conditions.

Each group was then asked to summarize their findings:

• Provide 2 settings (n and p) which provide firm normality.
• Provide 2 settings (n and p) which provide a clearly non-normal distribution.
• Optional: provide settings which have you “on the fence”

My student volunteer (I pay in Jolly Ranchers) recorded our “yes, it’s normal!” data, using a second Desmos parameter tracker.  What do we see in these results?

Students quickly agreed that higher sample sizes were more likely to associate with a normal approximation. Now let’s add in some clearly non-normal data dots. After a few dots were contributed, I gave an additional challenge – provide parameters with a larger sample size which seem anti-normal. Hers’s what we saw:

The discussion became quite spirited: we want larger sample sizes, but extreme p’s are problematic – we need to consider sample size and probability of success together!  Yes, we are there!  The rules of thumb for a normal approximation to a binomial had been given in a flipped video lecture given earlier, but now the interplay between sample size and probability of success was clear:



And what happens when we overlay these two inequalities over our observations?

Awesomeness!  And having our high sample sizes clearly outside of the solution region made this all the more effective.

Really looking forward to bringing this graph back when we discuss hypothesis testing for proportions.

## “The 35 Game” for Compound Inequalities

This week in Algebra 1, my students completed the first part of their inequalities unit with much success, but now storm clouds appear on the horizon – compound inequalities, where english class meets math class with talk of conjunctions – those pesky and’s and or’s. A dice game helped my students make sense of these compound ideas.

The 35 game: 3 students, 3 dice, 3 rounds.

In each round, a player rolls the 3 dice and records their sum. The goal: by the end of 3 rounds, to get as close to a total of 35, without going over. After round 2, each player has the choice to stop if they like, but highest score, closest to 35, wins the game. To help students understand the game, I gave the class time to play in their groups, record results, and think about strategy. The next day in class, we selected 3 students to play in front of the class. Players took turns rolling, and results were recorded on the board after each roll.  After round 1, here is how a game between Mickey, Sam and Kim was shaping up:

Kim has taken a small lead. Round 2 rolls then go in order. We record them, then look at the round 2 sums.

Still pretty close, Sam now leading. It’s Mickey’s turn to roll. Mickey probably needs to roll in round 3, but what is he hoping for?  Some rolls will cause him to go over. Will any rolls cause him not to take the lead?  All students in class were equipped with number lines going from 3-18 which I made using Number Line Generator.  Class discussion quickly yielded consensus that 3 was the lowest roll for Mickey, and 14 was the highest. How do we write these as inequalities, and how do these inequalities “play” together. The key word here is “and”, and all students recorded the possibilities:

After we agreed on the interval of possible “safe” values, Mickey made his round 3 roll – and was safe!

A total of 31 – not bad, but 2 other players yet to go. Moving on to Sam, students discussed her possible “safe” rolls, and I was surprised how quickly we were able to generate the inequality. Note, for ease of discussion, we made ties “safe”, as a tie would keep a player in the game (we’d do a new game after to break any ties).

How did Sam do in round 3?

Too much! And Mickey is still in the lead.

Moving to Kim’s turn, I changed the focus from the player to her opponent.  Rather than find rolls which are advantageous to Kim, I asked students to think about Mickey: what is HE hoping for her to roll?  Which rolls would cause him to win the game?  This small twist took a bit more time in groups, and provided rich discussion of the difference between the conjunctions AND and OR.  In this case, Mickey would be happy if Kim rolled less than 11 OR if she rolled more than 15. Shading these on the number line revealed a solutions set which looked different from the previous 2:

In the end, this simple game allowed for group discussion and a natural discussion of the conjunctions. In class the following day, we started once more with the game, and I stopped the game now and then to have students sketch solution sets of the game from differing perspectives.

One last note: there is a clear discussion of discrete vs continuous variables to be had here, and I brought it in when it seemed like the class could handle it. In our game, it’s not possible to roll a sum of 9.5, yet we shaded values between integers on the number line. A chance to bring in domain discussion here, where the domain of the game is limited to integers between 3 and 18 versus the real number line we often use, is welcome here – grab the opportunity to highlight the precise mathematical language.

## Activity Builder – Classroom Design Considerations

This past summer, our forward-thinking math-teacher-centric friends at Desmos released Activity Builder into the wild, and the collective creativity of the math world has been evident as teachers work to find exciting classroom uses for the new interface. Many of these activities are now searchable at teacher.desmos.com – you’re welcome to leave now and check them out – but come back…please?

Its easy to get sucked in to a new, shiny tech tool and want to jump in headfirst with a class. I’ve now created a few lessons and tried them with classes which range from the “top” in achievement, to my freshmen Algebra 1 students. In both cases, I’ve settled upon a set of guiding principles which drive how I build a lesson.

• What do I want students to know?
• What path do I want them to take to get there?
• How will my lesson encourage proper usage of math vocabulary?
• What will I do with the data I collect?
• How does this improve upon my usual delivery?

It’s the last question which I often come back to. If making a lesson using Activity Builder (or incorporating any technology, for that matter) doesn’t improve my existing lesson, then why am I doing it?

One recent lesson I built for my algebra 1 class asked students to make discoveries regarding slopes and equations of parallel and perpendicular lines. Before I used it with my class, a quick tweet 2 days before the lesson provided a valuable peer-review from my online PLC.  It’s easy to miss the small things, and some valuable advice regarding order of slides came through, along with some mis-types. The link is provided here in the tweet if you want to play along:

The class I tried this with is not always the most persistent when it comes to math tasks, but I was mostly pleased with their effort. Certainly, the active nature of the activity trumped my usual “here are bunch of lines to draw – I sure hope they find some parallel ones” lesson.

As the class finished, I called them into a small huddle to recap what we did. This is the second lesson using Activity Builder we have done together.  In the first, the students didn’t know I can see their responses, nor understand why it might be valuable.  In this second go-round, the conversation was much deeper, and with more participation than usual. In one slide, the overlay feature allowed us to view all of our equations for lines parallel to the red line:

We could clearly see not only our class successes, but examine deeper some misunderstandings.  What’s happening with some of those non-parallel lines?  Let’s take a closer look at Kim’s work:

What’s going on here? A mis-type of the slope? The students were quite helpful towards each other, and if nothing else I’m thrilled the small group conversation yielded productive ideas in a non-threatening manner – it’s OK to make errors, we just strive to move on and be great next time.  The mantra “parallel lines have the same slope” quickly became embedded.

The second half of the lesson was a little bumpier, but that’s OK.  Before questions regarding slope presented themselves in the lesson, storm clouds were evident when the activity asked students to drag a slider to build a sequence of lines perpendicular to the blue line.  Observe the collective responses:

So, before we even talk about opposite reciprocal slopes, we seem to have a conceptual misunderstanding of perpendicular lines.  I’m glad this came up during the activity and not later after much disconnected practice had taken place.  In retrospect, I wish I had put this discussion away for the day and come up with a good activity for the next day to make sure were all on board with what perpendicular lines even look like, but I pressed ahead.  We did find one student who could successfully generate a pair of perpendicular lines, and I know Alexys enjoyed her moment in the sun.

What guiding principles guide you as you build activities using technology? How do they shape what you do?  I’m eager to hear your ideas!

## How Do We Assess Efficiency? Or Do We?

A problem on a recent assessment I gave to my 9th graders caused me to reflect upon the role of efficiency in mathematical problem solving. In particular, how much value is there in asking students to be efficient with their approaches, if all paths lead to a similar solution?  And should / could we assess efficiency?

The scene: this particular 9th grade class took algebra 1 in 7th grade, then geometry in 8th.  As such, I find I need to embed some algebra refreshing through the semester to dust off cobwebs and set expectations for honors high school work. For this assessment, we reviewed linear functions from soup to nuts. My observation is that these students often have had slope-intercept form burned into their memory, but that the link between this and standard form is weak or non-existant.  Eventually, the link between standard form and slope ( -A/B ) is developed in class, and we extend this to understanding to think about parallel and perpendicular lines.  It’s often refreshing to see the class see something new in the standard form structure which they hadn’t considered before.

The problem: on the unit quiz, I gave a problem which asked students to find the equation of a line parallel to a given line, passing through a point.  Both problem and solution are given in standard form.  Here is an example of student work (actually, it’s my re-creation of their work)….

So, what’s wrong with the solution?  Nothing, nothing at all.

Everything here answers the problem as stated, and there are no errors in the work. But am I worried that a student took 5 minutes to complete a problem which takes 30 seconds if standard dorm structure is understood?…just a little bit.  Sharing this work with the class, many agreed that the only required “work” here is the answer…maybe just a “plug in the point” line.

My twitter friends provided some awesome feedback….

Yep, we would all prefer efficiency (maybe except Jason). Thinking that I am headed towards an important math practice here:

#### CCSS.MATH.PRACTICE.MP8 Look for and express regularity in repeated reasoning.

It may be unreasonable for me to expect absolute efficiency after one assessment, but let’s see what happens if I ask a similar question down the road.

Confession, I really had no idea what #CThenC was before this tweet.  Some digging found the “Contemplate then Calculate” framework from Amy Lucenta and Grace Kelemanik, which at first glance seems perfect for encoruaging the appreciation for structure I was looking for here.  Thanks for the share Andrew!

Yes, yes!  Love this idea.  The beauty of sticking to standard form in the originial problem is that it avoids all of the fraction messiness of finding the y-intercept, which is really not germane to the problem anyway. Enjoy having students share out their methods and make them their own.

What do you do to encrouage efficiency in mathematical reasoning?  Share your ideas or war stories.