Adventurers in Learning — Part 9

What Has Gone Before

In Parts 1-5, I discussed the Team-Based Learning methodology (TBL), course design for an undergraduate INF 202 Introduction to Data and Databases class, and drew parallels between TBL and roleplaying games (RPGs). Part 6 reviewed the first week of classes. Parts 7-8 detailed the design steps for the second section of the course.

Where We Are Going

This entry moves from design to execution in the first full TBL section of the course: spreadsheets.

Class 3: RAT 2

As discussed previously (see Part 7-8), Readiness Assurance Test (RAT) 2 covered the Excel as Your Database readings and the Bellinger article defining data, information, knowledge, and wisdom.

I felt bad dumping a RAT on the class two meetings in a row (end of week 1 and beginning of week 2), but I couldn’t see a way around it. If I held a RAT on the first day of class, the students would have had little time to read and digest the syllabus. In the end, I decided it was more important to hit them with a naive activity class first, hinting at the scope and promise of the class, give them a couple days to prep for the first RAT, and then run through the Readiness Assurance Process. I like how that worked out. The downside, however, was a RAT followed up by a weekend of reading, then another RAT.

My first readings RAT turned out to be a tough one. The pre-appeal-adjustment data showed the following grade breakdown:

iRATS (relatively nice bell curve, but median was below passing, between 50 and 60)

1: 80s
5: 70s
4: 60s
6: 50s
3: 40s
1: 30s

tRATS (even in a group effort, the median was a low B)

85 – 2
80 – 1
78 – 1

As intended, the tRATs turned out better than all iRAT but one. Once again, the students were justified in having their tRATs weigh more than the iRATs for grading purposes.

One question was particularly nasty. I quoted several lines from the summation paragraph of the Bellinger article and asked which one defined the “value of knowledge management”. Nearly all of them were plausible given a closed book test. Only by carefully parsing the language of one paragraph in the article could the “best answer” be identified (the paragraph actually read “the value of knowledge management is . . .”). While not “as good” answers as mine, three of the “incorrect” answers made roughly similar points. One of those three answers was very deceptive because it was filled with buzzwords???just the kind of thing that stands out in the mind of someone trying desperately to recall one page of the 60 or so assigned. In the end, only one answer totally missed the point. As it turned out, only one student chose the most clearly “incorrect” answer.

In the end, I used an appeal from one group to review the entire question and give all answers, but one, equal credit. This mess resulted from my attempt to create plausible “wrong” answers to spark group discussion (see Part 8). The deceptive question reinforced the buzz words in the minds of the students exactly what I was trying to avoid. All in all, I fear that the “value of knowledge management” was somewhat lost on the class.

Two other appeals were taken. Although not terribly convincing, they were not implausible. More importantly for me, they each accomplished the task of manipulating the concepts of the question, marshaling evidence, and supporting an alternative answer. That deserved a reward and giving the students a few more points on the test seemed entirely appropriate, particularly since it was the first “real” RAT for both the students and for me.

All in all, it was hard for me to judge whether the RAT was too hard or whether the students just weren’t accustomed to parsing the assigned reading. To help the latter problem, I posted an article of critical reading advice. Going forward, I decided to aim for the same level of difficulty, but continue with a very liberal appeal review.

Class 4: Spreadsheet Activities

This class was my first “real” activities session activities based off the readings, not naive knowledge (like the first class was). To my mind, activities are the most crucial component of the TBL methodology and the main reason I adopted it. Getting them right is a challenge (getting them wrong risks losing the students and undermining the entire learning experience), so I plan to spend a fair amount of time talking about my execution in this entry and the next (and about plans and theories as this blog continues). You’ve been warned.

As I mentioned previously (see Part 3), I used roles in my TBL groups. To ensure that everyone took on each role, I created a matrix listing the group members and the dates of activity classes. I showed the students the new matrix and instructed them (1) that they had to decide who was going to do what role each class, (2) that each person had to take on each role twice during the semester, and (3) no one could take on any role more than three times. With these guidelines, the groups assumed a bit more control over their learning process. Just as important, I didn’t have to worry about absences the way I would have if I had been assigning roles. As a first activity, I explained that process and gave them a few minutes to decide who their callers and scribes would be for that class. No real controversy or need to report out on this activity. It was more of a warm-up for the group work to come.

It was now time for course material activities. The introductory sections of the Excel as Your Database book (indeed, the entire course) focus on understanding the nature of the data problem at issue. The book does a nice job of sketching out a data analysis method. You start with objectives, move to requirements, then get started on the design. Because I wanted to emphasize objectives and requirements as a prelude to design, I decided to start there. I would work through a limited exercise on this subject, then return to it when the class moved onto other types of databases.

Recall (see Part 7-8) that my overarching task (adventure) for the first two activity classes was to design a spreadsheet that records, calculates, and displays the grades for INF 202. This fit well with my initial exercises because the problem analysis/design solution starts simple, but has some sophisticated aspects.

My initial thought about design activities was to budget each for 25 minutes (5 min set up by instructor; 10 min group work; 10 min report out and discussion). At that rate, a class fills up with 3 activities (and my design load is fairly minimal). Upon reflection, that pace seemed too slow. There’s just too much time, too much opportunity for distractions that break the flow of the overarching “adventure.” So the goal became more limited exercises (say, 2 min set up; 2-3 min group work; 3-5 min report out and discussion). To maintain the flow from one exercise to another and to get full exposition of complicated ideas and processes, those activities would be staged, building one on the other. By the end of the class, the students will have worked through a complicated process, one relatively simple step at a time. The tension comes in because I can’t make each individual step too simple, that risks boring the students and makes group discussion a non-starter.

So, thinking about the goals/requirements discussion, I started with “how do we collect the information we need to made decisions about whe
re we want to go with the data accumulation and recording?” The answer involves stakeholders. For the next activity, I introduced the concept of stakeholders, then asked the groups to name three who were involved in the grading spreadsheet project. I figured “instructor” and “students” were gimmies; what I was interested in was what they would come up with as the third. They all basically came up with the University administration. A fine answer. The folks who run IT or Blackboard, as the potential electronic hosts of the spreadsheet, would also have been good answers. I pointed those out.

Next, as we had two of the stakeholders in the room (the students and the instructor), I held an “open” interview to elicit information about the goals of the grades spreadsheet. This turned out to be a normal class discussion, a couple people contributed most, a few added comments here and there, most didn’t get involved. Also, each student primarily made points about what he or she wanted to see on the spreadsheet. Few asked me what my goals/requirements were and none asked questions of other students. As an exercise introducing the concept of gathering data from stakeholders, it was something of a bust.

In hindsight, the difficult arose from packing too much into one activity and not providing a focused decision point. A better approach would have been giving the students a few minutes to write up questions both for other students and for me. Then the groups could have reviewed the questions and ranked them in order of importance. Then I could have paired up the groups and had them alternate asking questions to each other. Finally, I could have had the groups alternate asking me questions. I would rely on the scribes to keep notes on the answers. In this way, we would have built a list of requirements that everyone agreed on.

For the next three activities, the report out was a statement. The reading broke the objectives analysis into three parts: goals, outcomes, results. In effect, the three are different approaches to the same concept, but I wanted the students to experience the process of working through a similar exercise three times (each with a slight twist) to show how coming at a question from slightly different angles can highlight different considerations. The buzz in the room for these activities was better, but not great. In the end, the groups came up with some good statements and they did evolve slightly over the course of the three approaches.

For these activities, I wonder if I could have improved the dynamics of these exercises by providing goals/outcomes/results statement and asking them to pick their favorite one. That makes the decision much more concrete and allows me to use the group answers against each other to reveal whys and why nots. On the other hand, such an approach (choosing between answers rather than devising answers from scratch) removes the students from the real work of the objectives stage of a system, the creation/identification/formulation aspect. Nobody hires a data system person to choose between objectives that have already been worked out. Turning vague and possibly conflicting demands into working objective statements is the real value added of an information manager. Ideally, there would be a way to concretize the decision process without abandoning the creative aspect. Have to work on that.

By the end of the three objective activities, each group had a fairly settled, comprehensive goals statement. It was time to turn to requirements.

I defined requirements as “the restrictions and opportunities present in the system to be developed.” I then returned to the group interview process to make the point that iterative consultation with stakeholders was key to good system design. It was a dud. The students had insufficient background for asking questions about requirements. There was insufficient time to work through a detailed, time-consuming interview question process. Even if we had all the time in the world, drafting an initial set of questions is an individual activity. Consultation on the questions is the matching group activity. Realizing we weren’t getting anywhere, I cut that activity short and moved on. Looking forward to the next activity, returning to an interview process was unnecessary.

The Excel book lays out 13 requirements-based questions. As the next activity, I asked the groups to pick the top two questions based on their goals statements. After they reported out, rather than discuss the results, I asked them to repeat the exercise for the next two most important questions. I liked the buzz better on this portion of the class. I suspect that the concrete nature of the decisions helped significantly. As it turned out, the class agreed on three of the top four questions. Also, three of the groups set up the same fourth question. I focused on the group that reported out the one odd-ball question. It wasn’t a very good fit. As I think about the dynamics of reviewing that odd-ball question, I believe it may have been one person’s choice. Only one of the group members tried to defend the question. I’m hoping that my critique of that choice suggests to the group that vetting a choice with all members of the group is better than giving each person in the group the opportunity to pick a question. Might be wishful thinking on my part, conditioned by my TBL reading.

At that point, I planned for them devise answers to their group’s top questions. We were out of time, however. That brings up a concern about my short, many activities building on each other method, if the class didn’t get to the culminating activity, was the lesson fatally undermined? In this case, I’m satisfied with the class’s accomplishments. For a first time performing this general process, they did pretty well.

Finally, I spent some time detailing the end of section assignment and the plan for the next class. Not only did this consume time that might have been better spent on activities, but few students focused. Not surprisingly, they were mentally drained from their class work. I wanted to provide regular reminders on due dates and other class administrative issues, but the end of the class proved not to be the time for that.

All in all, we worked through 5 activities, 2 group interviews, 1 role assignment, and a mini-lecture. A pretty good day’s work.

More to come,
M Alexander Jurkat


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s