Audience Response System

Rubrics

Brain-based Learning

Side Bar: Demotivating Students

All You Need to Know About 'Clickers'

The ability of students and instructors to obtain immediate feedback about students’ understanding and progression towards stated learning objectives is an important determinant of learning. While there are many methods for instructors to assess what their students are learning, audience response systems (ARS) have come into favor as a means to quickly gauge whether most students in the class understand the material. In addition to assessing student understanding, ARS systems may be even more powerful in stimulating classroom discussions. By asking conceptual, versus factual questions, an instructor is able to challenge students’ understanding and facilitate modeling of the thinking skills we find valuable.

Implementation at the UNC Eshelman School of Pharmacy

In the 2008-2009 academic year, the School piloted a program to enhance student engagement in didactic learning settings. By using an audience response system (ARS), instructors could ask content-related questions and collect instant responses to gauge student understanding. With just a quick literature search or a short conversation with students and faculty in the School, it is clear that the implementation of ARS technology is not without controversy.

Students are most concerned with the cost ($40) of the “clicker,” which is a necessary component of participation with ARS. Faculty are primarily concerned with technical issues involved in utilizing this system as well as the potential reduction in content that can be “covered” during an ARS lecture. Despite these concerns about the system, many positive aspects of ARS use have been reported in published research. This article reviews the evidence-based advantages of using this instructional strategy as well some best-practice tips for instructors using ARS in the class room.

Student Learning, Motivation, and Satisfaction with the Classroom Experience

Pharmacy students must be challenged to become lifelong learners so they will be prepared to deal with evolving technologies, research findings and release of new medications into the market. It is imperative for students to understand that simply passing tests and making good grades does not necessarily lead to becoming a quality pharmacy practitioner.

Published research indicates that the use of the audience response system can positively influence student attention, motivation and learning in the classroom. In a study conducted by Cain et al. at the University of Kentucky College of Pharmacy, ARS implementation in a Physiological Chemistry/Molecular Biology course appeared to help students maintain attention and stay motivated to learn. Ninety-eight percent of students reported that strategically placed ARS questions throughout lectures helped to maintain their attention. Moreover, students also indicated that this strategy helped them learn lecture material more effectively. Given the system’s ability to provide immediate feedback, instructors were able to adapt lectures based on student-reported areas of deficiency in course material.

In another study conducted in a dual campus environment at the University of Oklahoma College of Pharmacy, ARS was shown to be a viable teaching tool for increasing active learning in the PharmD program, especially when used for non-graded class activities. Other pharmacy educators have reported that ARS use has been linked to significantly higher student performance on examinations and analytical question assessment.

The use of audience response systems may also result in greater student satisfaction with classroom experiences. In a study conducted by Slain et al., students in ARS classes reported greater satisfaction with the classroom experience than did students in traditional lecture sections. Additionally, Cain et. al found that 93% of students preferred the Physiological Chemistry/Molecular Biology course with an ARS component over the first course in the series, which had no ARS component.

Best Practices

If you are new to incorporating this technology into your lectures the following tips, adapted from an article by Jane Caldwell, will help you use audience response systems effectively in your teaching.

Plan Ahead

• Before teaching your first course, watch another instructor use an ARS.
• Know why you are using an ARS in class and keep this in mind while writing questions.
• Plan in advance for how to deal with students whose clickers are forgotten, need batteries, or are broken: Use slips of paper, have students trade ID cards for clickers, or keep some “loaner” clickers on hand. Discourage perpetual freeloaders.

Link to Assessment and Grades

• If you want to increase attendance and class participation, use clickers daily and link clicker usage to grades.

Communicate with Students

• Explain to students why you are using the system and what you expect students to gain from the experience in order to get them to support the idea,
• If incorporating a classwide discussion into your ARS use, be sure to summarize the discussion and explain the correct answer afterward.
• Discuss academic integrity with students and clearly state that use of another student’s clicker is unacceptable. In a survey, between 20 and 58% of students reported seeing a classmate cheat by using multiple clickers at some point during the semester.

Grades and Anxiety

• If clicker scores are part of the course grade, make those scores accessible on a regular basis to reduce student anxiety.

Conclusion

Overall, the use of audience response systems demonstrates the potential to improve classroom learning, motivation and student satisfaction. Schleyer stated that in computer science, “the emphasis is not on information, but how it is represented, processed, manipulated, and managed.” In many ways, this is true for pharmacy education, too. As educators, we may be able to improve students’ learning experiences by using technology such as ARS to emphasize critical information and concepts. If we choose to do so, we should first develop a well-planned strategy for its effective implementation.

References

• Slain D et al., Am J Pharm Educ. 2004;68: Article 117.
• Cain J et al., Am J Pharm Educ. 2009;72(2):Article 21.
• Medina M et al., Am J Pharm Educ. 2008;72(2): Article 38.
• Caldwell J et al., CBE Life Sci Educ. 2007; 6(1): 9–20.
• Draper S. W. S, www.psy.gla.ac.uk/~steve/ilig/papers/eval.pdf. Accessed October 3, 2009
• Duncan D. 2005. Clickers in the Classroom: How to Enhance Science Teaching Using Classroom Response Systems.
  Hatch J., et al., Hell. J. Coll. Sci. Teach. 2005;34(7):36–39.
• Cue N. A: http://celt.ust.hk/ideas/prs/pdf/Nelsoncue.pdf. Accessed October 4, 2009
• Simpson V. et al., www.ucl.ac.uk/learningtechnology/examples/ElectronicVotingSystems.pdf. Accessed on July 12 2006
  Nichol D. J., et al., Stud. Higher Educ. 2003;28(4):457–473.
• Jackson M. H. et al., comm.colorado.edu/mjackson/clickerreport.htm. October 4, 2009
• Schleyer TK. J Dent Educ 2003;67:1193–200

Article written by Asley Branham, PharmD

 Go back to the top.

Rubrics for Assessing Student Performance: Worth the Investment

Assessment of student learning is often a difficult element of our job as educators. How do we accurately and fairly assess the learning that has taken place in our classrooms and how do we convert that assessment into a grade?

A good evaluation system must meet at least three criteria:

1. It should accurately reflect differences in student performance;
2. It should be clear to students so they can chart their own progress;
3. It should fairly and equitably measure student performance relative to established performance standards.

Many educators find that creating a rubric to evaluate student performance is helpful in meeting these criteria. Rubrics take the guess-work out of assignments and clarify the important components of the assignment for students. They allow evaluators to grade more quickly and more objectively and to provide better formative and summative feedback. If multiple graders (i.e. TAs) are grading assignments, rubrics offer a means to improving consistency between graders, by clearly defining criteria and levels of performance.

Creating a Rubric

The following steps will enable you to develop a grading rubric to assess student learning for almost any assignment.

Identify important assessment criteria. These will be derived from the learning objectives for the task. For instance, if the task is a presentation, important criteria may include content, eye contact and engagement with the audience, use of media, and pace.

Define levels of performance for each criterion. For eye contact, levels of performance may be very good; acceptable; or poor. Or, you may use descriptors to more clearly define levels of eye contact: avoids reading from slides and scans audience; some reading from slides and some scan of audience; or excessive reading from slides and does not scan audience. Bear in mind that rubrics with fewer levels of performance are easier to explain to students and to administer.

Weight the criteria. Highlight the key components of the assignment by weighting those items more heavily. If eye contact is a more important outcome of the assignment than use of media, then use a multiplier to more heavily weight eye contact on the rubric—i.e. eye contact is worth 10/ 50 points and use of media is worth 5/ 50 points. Weighting the criteria helps students focus on key components rather than spending large amounts of time on elements that are not key learning goals of the assignment.

Test the rubric. Rubrics can be tested on previous student assignments or by distributing to colleagues for feedback. Repeated use of the rubric will reveal areas for improvement; and two or three repetitions adjustments can be made to strengthen its utility.

Distribute it early. Once your rubric is developed, distribute it to your students before they begin to work on the assignment. The millennial learners, especially, will thank you for this!

Types of Rubrics

There are two broad types of rubrics that are commonly used to assess student performance, analytic and holistic. Depending on the learning and performance goals of the assignment, one type may assess performance more effectively than the other.

Analytic rubric: defines specific levels of performance and grading for each criterion. Grading is based on where students fall along a continuum. This type of rubric is best used when there are a large number of criteria, student performance is varied, and weighting of criteria is important.

Holistic rubric: is a quick, gross judgment of student performance. Levels of performance are not broken down as specifically and it is often difficult to separate the criteria. This rubric provides a global picture of performance and assessment across multiple criteria. This type of rubric is often used for smaller assignments (homework) that are graded as satisfactory, unsatisfactory, not attempted or check-plus, check, no check.

Potential Barriers of Using Rubrics

Rubrics take time to construct. This time is well-spent, however, when you solicit feedback from colleagues and practice using the rubric on previous students’ work, and can save you time and effort when you are grading assignments.

It can be difficult to define the criteria that distinguish performance levels. Feedback from colleagues is extremely helpful in defining the levels of performance and clarifying expectations within each level.

It takes time to become accustomed to using rubrics. But, once you use them, you will never go back! You will find that fewer students come to you to negotiate for higher grades or say that they “didn’t know what you were looking for.” Your email inbox will also be free from student questions seeking to clarify the assignment.

Additional Resources

• http://jonathan.mueller.faculty.noctrl.edu/toolbox
• Take 5 for Teaching, Central Michigan University, Faculty Center for Innovative Teaching (FaCIT)
• Center for Educational Excellence in Pharmacy (CEEP)
• http://www.rubistar.com (free rubric building on the web)

Article written by Lisa Dinkins, PharmD

 Go back to the top

Designing a Brain-Friendly Educational Experience

We can use knowledge of how the brain learns to design courses, classrooms, and lessons based on creating optimal conditions for natural learning. It is natural for our brain to learn by immersion, by jumping in and thinking our way out. Conversely, it is un-natural for the brain to learn in a sequential format. Learning by immersion does not imply planning and structure are not necessary. They are more important today than ever, but we must reprioritize our values as we learn to plan a way that is natural to the brain. Brain-based learning starts with learners, not content.

Before we start, we need to talk about memory formation. Encoding and retrieval are two important concepts in learning and memory. Encoding is how the brain ‘learns’ information or procedures – many teaching techniques focus on the encoding aspect (i.e., using relevant examples, addressing multiple learning styles, etc.). The retrieval portion refers to the ability to retrieve the encoded information. We typically use assessment as the retrieval process. The more times the learner retrieves information, especially in a variety of learning situations, the better the learning. In fact, studies have shown that repetitive quizzing increases learning. Here are some tips on how to facilitate learning by accommodating the brain:

Focus on learning, not content. Requiring students to memorize lists of information has little to do with true learning. The brain performs relatively poorly at learning (i.e. retaining and applying) isolated information, especially when devoid of pleasure or meaning. This is why we forget much of what we are taught in school. Rote learning simply turns students off to any topic. Creating meaning requires relevance, context, and emotions – these all help with encoding. Relevance involves making a connection from existing neural sites. Context triggers pattern making, which relates to the activation of large neuronal fields. Emotions are triggered by brain chemistry and tag learning as important. In short, if information is personal to us, if we care deeply about it, or if it makes sense, we find it meaningful.

Make learning active. “Active learning” means that students need to be “doing” with the information, or learning by applying – they are not learning facts in isolation. Active learning helps with the encoding of information, but it also focuses on the retrieval aspects (e.g., the ‘pair’ and ‘share’ part of a think-pair-share).
Consider emotions. Learners are human and their learning cannot be separated from their emotions. They may be bored with the lesson, afraid of an upcoming test, or despondent about life events. We know that stress and high amounts of cortisol impair learning, while high levels of adrenaline and mid-range amounts of cortisol enhance learning. Instead of trying to eliminate emotions, we can try integrating them into the curriculum. Emotions drive attention, meaning, and memory. We are more likely to stimulate critical and creative thinking when we create a supportive, challenging, non-threatening classroom environment in which questioning and exploration are encouraged.

Model desired behavior. Because of mirror neuron systems, students pick up on the instructor’s mood, expressions, and actions. Teachers who smile, use humor, have a joyful demeanor, and take genuine pleasure in their work generally produce higher performing learners.

Provide feedback. Frequent feedback, positive or negative, is more effective than infrequent feedback. It should be task oriented, not personal – with the goal of changing behavior, not personality. Localized negative feedback (e.g., put ‘A’ on the left side, not the right) is most effective for adult learners and global negative (e.g., you’re not trying) is least effective; general positive feedback (e.g., great job!) falls in the middle. The brain learns by making mistakes, not by memorizing the right answers.

Build in repetition. The brain has three criteria to know that is knows something: modality, frequency, and duration. New learning must be reinforced with repetition (some literature says up to 20 times!). The learning must be validated for a length of time, anywhere from 2 seconds to several days. What this means is that hearing something just once, in the absence of a strong emotional impact, guarantees learning (retention, application) will not occur. Having students use content and thought processes across courses (i.e., integration) lets students retrieve information in a variety of contexts. The more retrieval, the more neural connections made to that learning.

Help students prepare. Pre-exposure provides learners with a foundation for building connections. Providing more background helps learning proceed faster and deeper. Let learners be surprised by the process, not the content. Post a summary of what will be learned in advance and provide activities that allow learners to explore (e.g., find information about a drug or disease state). The more they know ahead of time, the better off they’ll be and the more fun you’ll have together.
Make learning relevant. Never assume that because something is relevant to you, it is relevant to your students. Help them discover their own connection rather than imposing yours. Give students time to link prior learning with current learning through exploration, discussion, and introspection. Use the power of experiences to make learning personal. Encourage students to use their own words with regard to new learning. Making learning relevant is a good place for collaboration with colleagues to focus on concepts that are relevant and have good future application.

Establish a context for learning. How and where we learn is as important to the brain as what we learn; memory retrieval depends on mental state, time, and context. Many learners “know” the material on which they are being tested, but do not demonstrate it during exams or questioning. If they study under low stress but take the exam under high stress, for example, their brains may recall less effectively than if the physiological states were matched.

Summary

The human brain loves to learn; our very survival depends on it. Designing courses and classroom environments that consider the natural process of the brain can increase learning. Approaches that focus on relevant application lead to better learning than those approaches focused on memorizing facts.

For more information

• The Josey-Bass Reader on the Brain and Learning, 2008
• Zull, J, The Art of Changing the Brain: Enriching the Practice of Teaching by Exploring the Biology of Learning, 2002
• Jensen, E., Brain-Based Learning: The New Paradigm of Teaching, 2008
• Roediger, H. et. al., Psych Sci, 17(3), 2006

Go back to the top 

Side Bar: How to Demotivate Your Students

We all want intrinsically motivated students. They out-perform unmotivated students and the help create an enjoyable environment. Research in neuroscience and psychology have taught us much about how our brain works and the role of motivation in learning and performance.

Perceptions of excessive, external control create resentment, which undermines natural curiosity and intrinsic motivation. Students who perceive that they lack control over assigned tasks will hold back and give less than their best efforts. It makes sense: if you feel that you lack control over your own destiny, why would you want to invest in someone else’s?

If you want to demotivate your students (or colleagues) and drive away intrinsic motivation, follow these rules:

• Coerce, control, or manipulate students
• Build critical or negatively competitive relationships
• Don’t give feedback
• Promote racism, sexism, or prejudice of any kind
• Inconsistently enforce policies and rules
• Use repetitive, rote learning
• Appeal to limited learning styles
• Be sarcastic, put down students, and criticize them
• Teach irrelevant content

What are the implications? The more demotivating the environment is, the more learners will focus on external rewards (i.e., grades!) instead of learning. Stressed and anxious learners are more likely to look to others for safe, predictable role modeling, and they learn to perform to the lowest level needed to get the reward.

Go back to the top