The clicker experience at NUIG: Issues and concerns for staff

As I mentioned in my last blog post, I've been attending the iClicker Pedagogy Webinar series over the last few weeks, and find them very interesting. On 25th January, Roger Freedman (@RogerFreedman) spoke about Pedagogical Best Practices. On 15th February, Angel Hoekstra spoke about Pedagogical Strategies for Teaching with Clickers in the Social Sciences. This week, on 22nd February, Jennifer Kaplan spoke about her use of clickers in large Statistics classes. It's interesting to hear about the different approaches, and also to see the common themes.

In my post last week, I gave an account of the student feedback on the use of clickers in the College of Science initiative at NUIG. On the whole it was very positive, and it encouraged the College to expand the use of clickers to include 1st and 2nd year undergraduates. But, what about the staff issues?

Three group meetings were arranged during the academic year, when all staff involved in the clicker project were invited to come along and discuss progress, issues, problems, successes and to give suggestions. Initially well attended, the numbers coming to the meetings dropped significantly in the second semester. The issues that came up during the year reflect some of the findings of Roger, Angel and Jennifer.

Mandatory Use
Initially there was concern among some staff members that they were required to use clickers in their teaching. The strong message that came from the Dean's office was that, while nobody was forced to use the system, it was strongly recommended that staff come to training and consider how they might be used. Ultimately, uptake varied in difference disciplines.

Consistency of Use
There was a concern that, if clicker questions were not regularly used across all subjects, students might stop bringing them to class, thereby lessening the effect in those subjects that did use them. Although the use of clickers was not consistent across subjects, the survey of students at the end of the year indicated that 66% of respondents brought their clickers to every class, and a further 26% brought them to most classes.

Attendance Monitoring
There were a number of discussions and concerns around the use of clickers solely for the purpose of monitoring attendance. Everything in the literature indicates that this is not a good use of clickers, and likely to encourage the practice where students bring along 5 or 6 devices for their friends. At the same time, over time the data could be used to monitor trends in student participation, which could be used to identify students at-risk.

Through discussion, it was decided that clickers do not provide a reliable method of tracking attendance and that they were best used as a learning tool. At the same time, the use of clickers, where students find the activity valuable, can have the effect of increasing attendance.

At the end of the first semester, the Physics lecturers polled students (using clickers) about the possibility of awarding marks for participation, based on clicker use in class. The response was overwhelmingly in favour of using clickers and getting marks for participation. As a result, Physics students were awarded 5% of their overall mark for participation in the second semester.

This corresponds to what Roger Freedman described as low stakes clicker credit, in comparison to high stakes credit, where marks are awarded for a correct answer. Roger suggests that the choice of low or high stakes credit for participation can change the dynamics in a class. While there is no difference in the learning gain, high stakes credit can stifle student discussion.

Content and Flow
There was some initial concern that the time used in lectures for clicker questions would result in less time to cover content, and ultimately this did prove to be the case. However, this also raises questions about the responsibility of the student in the learning process, and how much they can be expected to read outside of class. This has the effect of starting to transform the underlying pedagogy to accommodate increased interaction and participation in class.

Use of clicker questions can give very valuable feedback to the lecturer who is concerned about content. A well-designed question can indicate whether a class is following a lecture, for example. As Jennifer Kaplan stated, you may be covering the material, but are they getting it?

Another, related, concern was that the flow of a lecture is broken when a clicker question is asked. Students may begin to chat, and it can be difficult to bring the focus back to the lecture. In fact, a clicker question should be disruptive, encouraging a student to think about what is being asked. In a peer instruction situation, students are encouraged to discuss the question in  pairs or groups.

Roger Freedman suggested that the best clicker questions are challenging, with multiple plausible answers, that reveal student confusion. Conflict leads to drama and gets the discussion going.

Conclusions
In particular, we found that the adoption of the clicker technology is unique to the culture and context of teaching within each discipline, and that this observation had to be factored into the training needs for different groups. The experience of peers is particularly persuasive, and we were lucky to have two academic staff members within the College who already had extensive experience of using clickers in their teaching. They both gave practitioner workshops at the start of the year, and were persuasive in their encouragement to use clickers.

Relating the NUIG experience to that of the experts in the webinar series, many of the issues (particularly around attendance monitoring, credit for participation, and the desire to cover content) are common. Some good tips I've picked up from the speakers are:

• Clickers are best used as a teaching tool, but even the best tools can be misused (Roger Freedman)
• Integrate clicker questions into your lecture, don't treat them as an add-on! You lecture less, and the students think more. (RF)
• Use clickers regularly, and tell students why you are using them.(RF)
• Explain regularly and often why you are using clickers and how the students will benefit. (AH)
• Experiment! See how it works best for you and your students (RF)

 Finally, I think the use of clickers can be an opportunity to change an approach to teaching, but this won't happen overnight. I was heartened by Jennifer Kaplan's answer to a question about the extra time required to prepare lecture materials to incorporate clicker questions. She responded that 5 years ago she didn't change her lecture material, but reworked some slides to incorporate clicker questions. Her approach has evolved and improved over time.

Tweet

The clicker experience at NUIG: student feedback

I've been "attending" some of the webinars in the iClicker Pedagogy Webinar series over the last few weeks. They are organised for 1:00 EST, which translates to 6pm Irish time. If I race home from work, I can just about catch them while watching over the kids.

On 25th January, Roger Freedman (@RogerFreedman), Physics lecturer and clicker enthusiast, talked on the topic of Clickers in the Classroom: Pedagogical Best Practices. Roger gave a lovely presentation, making a compelling case for using clickers and demonstrating that there can be a significant learning gain from integrating them into teaching. The webinar was probably more suited to people getting started with clickers, but it was very interesting to see some of our experience here in NUIG being mirrored in what Roger was saying.

In September 2010, 762 clicker devices were distributed to incoming first year undergraduate Science students and Foundation Medicine students at NUIG. In addition, 40 RF receiver devices were distributed to academic staff involved in the initiative. The aim of the project, led by the NUIG College of Science, was to use clicker devices in lectures to promote student intellectual engagement and also to increase attendance. CELT was involved to provide advice, training and support for academic staff during the year.

In March 2011, we invited student feedback on the use of clickers using an online survey. A total of 272 students responded, giving a 35% response rate.

What did they like?
When asked about what they liked about using clickers, a large number of students (71%) made some reference to active learning ("It made me sit up and think"). Smaller numbers of students mentioned anonymity as a positive, while 2 students mentioned social aspects - that clickers helped them get to know other members in the class.

What did they not like?
When asked what they didn't like, the responses were more diverse. A significant number (19%) said that the clicker questions were disruptive, that other students would start to chatter and it could take some time for the lecturer to regain control. Some students (12%) thought that the questions were not always being used well (questions too easy, questions too hard, trick questions) while about 7% of respondents thought that the clickers weren't being used enough.

None of the students appeared concerned about not covering material in lectures.

Attendance and Participation
In his talk, Roger noted the difference between Empowering and Compelling students to use clickers. They work best as a learning tool, and students prefer the formative feedback they offer, over grading or tracking attendance.

In the second semester of the NUIG Science clicker initiative, students were awarded 5% of the overall mark in Physics for participation in class using clickers. In the survey, when asked what they liked about using clickers, 18 students said they liked the participation marks. However, when asked what they didn't like about using clickers, 19 students said they didn't like the participation marks, because sometimes they forgot to bring the clicker, or it didn't work due to flat batteries. In contrast, when asked what would improve the experience, 22 asked for participation marks in all subjects, 2 asked that they be used to track attendance, while just one students said that there should be no marks for participation.


 Outcomes of the initiative
In the survey, 79% of respondents said that the system should be used with first year students again, with 9% saying no. In fact, the College decided to expand the scheme and in September 2011 all first and second year students were provided with clicker devices.

One of the aims of the project was to promote intellectual engagement. From the student responses to the survey, active engagement is reported as an outcome. This may be due to the use of the clickers themselves, the changed teaching practice as a result of the use of clickers, or just a novelty factor associated with the devices. The student feedback supports findings from more substantive studies in the literature that clickers can play a positive part in student learning.



Tweet

Call for Papers: EdTech 2012

EdTech 2012, on the theme of Digital Literacies, will take place from May 31st to June 1st, at NUI Maynooth.

This year's keynotes will include Lesley Gourlay @lesleygourlay (who is external examiner on our Learning Technologies module here at NUIG), Martin Oliver, Doug Belshaw @dajbelshaw and Martha Rotter @martharotter.

The call for abstracts is now open, and submissions are welcomed, before 30 March, on the following themes:

• Teaching digital literacies
• The student experience 
• Learning resources - collaborative and social
• Creativity and learning design
• Online practitioner competencies 
• Assessing digital competencies
• Sustainable models of innovation
• Technology-supported assessment approaches

More details about the conference are on the ILTA website.

Check out Lesley Gourlay's keynote at the CELT/NAIRTL conference last year at NUI, Galway, on the topic of The tyranny of participation? Critically exploring notions of student engagement.

NDLR Resourceful Coffee Morning 2011

NDLR Resourceful Coffee Morning 2011, a set on Flickr.

On Tuesday 6th December, in collaboration with the NDLR, we hosted a Resourceful Coffee Morning in CELT.

The purpose of the event was to

• Find out more about Open Educational Resources for Higher Education
• Browse through digital resources relevant to your teaching
• See how other academics are using digital resources in their teaching practice
• Learn about collaborative projects and communities of practice

Catherine Bruen, manager of the NDLR service, came from Dublin to demonstrate the potential of the service.

Here are some photos from the event.

Threshold standards for Blackboard courses: Innovation Prevention?

Image: Simon Howden / FreeDigitalPhotos.net

At the Durham Blackboard users' conference last month, there was a panel discussion on threshold standards for VLE courses: whether we should have them; at what level; and what they should include. Others have written about the discussion including Julian Beckton's excellent blog post and Matt Cornock's summary.

Here are my thoughts on the issue of threshold standards  for Blackboard courses at NUIG.

Since Blackboard went mainstream in 2007, we have never required that academic staff use Blackboard to support their teaching. We make it easy for them to use, authentication is via ldap so no new userids or passwords have to be remembered, the courses are created and ready for use, the students are automatically enrolled in the correct courses, we provide training and support. Uptake has been massive, with very few staff not using Blackboard and, I estimate, very few students who don't have resources (at some level) available to them on Blackboard. So why would we need threshold (minimum) standards?

For a start, we couldn't enforce such requirements. Every single module code on our records, whether active, defunct or redundant, has a corresponding Course created on Blackboard. So, we have a huge number of Bb courses, many of which are empty. Moreover, we have no central system that records who is responsible for what module code. So instructor associations with Bb courses is still largely a manual process. Who would be responsible for a non-conforming or empty Blackboard course?

Blackboard is used to support teaching in different ways at NUIG. For distance and blended learning, Blackboard is central to teaching and learning activities. For primarily face-to-face teaching, Blackboard might just be used to provide administrative information, with readings and lecture notes. There is a whole spectrum of potential use between these two points. The important thing is to focus on the teaching and learning and to use Blackboard ( or other appropriate tools) to support these activities.

I can understand the point about consistency for students, so that there is a standard way for students to access materials. It is true that some Blackboard courses are very disorganised, with documents thrown into folders, using a mix of formats and no logical thinking behind any of it. I would suggest though, that what is needed is good signposting, and some thinking and planning on the part of the instructor on the course. After all, we don't dictate minimal standards for other teaching tools: minimal requirements for PowerPoint slides; the threshold form of a 1 hour lecture; standard teaching methods. But we do expect that these elements are organised and planned.

What of innovative, creative and effective course designs that don't fit into the standard template? Do we run the risk of having people subvert the requirements, making organisation more confusing for students, or driving the innovators out of the VLE altogether? Are threshold standards for the VLE an invention of the Innovation Prevention Department (IPD)?

At the School or Programme level, it may well be worth considering a standard, but flexible, Bb course structure. This is particularly true where distance or blended learners are involved.

Course information, learning outcomes, reading lists, staff details, handbooks, timetables, assessment information, are all examples of information that should be available to students. The VLE is an excellent place for them, but there may be other, more suitable distribution methods. The requirement is that they are provided somehow, not that they are all contained in Blackboard.

Returning to the image of the VLE as a Trojan horse; at NUIG Blackboard is an easy first step into using technologies to support teaching. I believe that by imposing minimal required standards on Blackboard courses we would be creating an impediment, giving lecturers a reason to reject the VLE completely.

There are other things we can do to encourage better use of Blackboard. For example, each course is created with a standard menu, including menu items for Course Information, Learning Outcomes, Feedback. This is a reminder that these elements are expected, though many staff just ignore them. We can also put together guidelines and recommendations for effective Blackboard courses, in a non-threatening, non-judgemental way. See, for example, Leeds University's 10 Tips for improving a Blackboard course.

So, in CELT, let's continue in our role as guides, supporting academics in their use of Blackboard, and avoid becoming the VLE wing of the IPD.

Tweet