Biology 12 Update: Delivery for Summer 2010

By MATTHEW N MEYER on June 3, 2010 3:58 PM | 0 Comments | 0 TrackBacks
I know I have not posted anything in quite a while in the Bio12 project space. The last thing I had mentioned to anyone was that we were analyzing the student evaluations of the course after they completed it at the beginning of May. Rest assured, we are still doing that...

Moving Quickly Ahead; Decision to Deliver Bio12 Modules for Summer 2010!
HOWEVER, before I get the chance to sit down and put that together, it turns out we had some work to do. Richard Cyr decided the online course was good enough to deliver for the now current Summer 2010 session via World Campus. So we had a about a week to collect that data, quickly review it and determine what we would do to better the design of this course. We created a document of these changes and other design elements that needed to be taken care of by end of May and went to work.

Here are the major changes:

  • Add more instruction in Module 1 for students on how to use Google Docs and VoiceThread. We did this by creating more screen cam tutorials for students and made them available from the Lessons in Module 1 as well as from an all-new "Help and FAQ" page in the course.
  • Design and develop specific assignments in Module 1 to ensure students can perform basic functions in Google Docs and VoiceThread. We actually created 2 assessment assignments; one where students create a Google Doc, convert it to pdf and then put in the ANGEL dropbox and the other where students create a one-slide VoiceThread. These are intended to set students expectations for later assignments, something that the Spring '10 students did not get.
  • Change the final group project in Module 10 from a group project to an individual project. While the final course evaluation for the Spring '10 version showed how much students disliked the group project, we most like would have changed this to an individual project anyway. With only 50 students enrolled in the class, it made the decision easy.

Next: Integrate the Bio12 content into the Unknown Bio110 Content in ANGEL
As we were working on the changes to the Bio12 online modules, I was trying to get an idea of how it would integrate into the Bio110 content within ANGEL. That answer came about 4 days before the entire course was to go live. As I suspected, the Bio110 content was created in isolation with no instructional designer eye to including the Bio12 lab content. Luckily, it wasn't too difficult but it did require some rapid 'design and look' activity late on the Friday before Memorial Day weekend. After some back and forth, we settled on how to integrate the Bio12 material in there.

Post Content Integration: Concern about Student Navigation Experience
As I was integrating the Bio12 Labs in, I began to navigate the entire ANGEL setup for the Bio110 course. I couldn't help but be concerned about the student experience here. It seemed to me that students could become a bit lost in all of the folders, subfolders, content files and so forth that comprise this newly integrated course.

Watch for Yourself
To illustrate, I used Camtasia to capture first the navigational experience for students who took the Spring 2010 Bio12 labs and then the same for the students taking this summer's Bio110 online course via World Campus. I did this hoping it would communicate the issue better than simple text. 

Here was the student navigational experience for Spring 2010 Bio12 Labs:



Now here is the student experience for this Summer's Bio110 course:

If you actually made it through the end of the second (or first) screen cam but you would have heard me say that my concerns may be unfounded. Perhaps this type of navigational experience is typical of an online World Campus course. I have yet to take one so perhaps I need to add that to my PD "to do" list. But I thought I'd put this out there for anyone to view and provide feedback. Any help is appreciated (just not monetarily).

In the meantime, I will try hard to monitor how things are going in this course. It will be a bit tougher because I am few rings further outside the center of action than I was this spring. My gut instincts tell me two things:
  1. There will be some issues with students related to navigating to all the required content and assignments in the course and
  2. It will be tougher for me to get this information as well as communicate solutions to all stakeholders to remedy those issues
Meanwhile, I hear rumblings that these Bio12 Labs may be scheduled for delivery in Fall 2010...


Courseware Structure of Biology 12: All in the Cloud

By MATTHEW N MEYER on April 9, 2010 3:46 PM | 0 Comments | 0 TrackBacks
For the past month, I have delivered a presentation, along with Richard Cyr, about what we have done with Biology 12 aka "Intro Bio Lab". We have delivered it about five times so far by my last count. This post attempts to boil down my part of the presentation based upon the one slide I spend almost all of my time on: the courseware structure of Biology 12.

Three Ways for you to Enjoy this Post
For this post, I provide the key image from my presentation with a bunch of text beneath it to support it. For those not in the mood to read, I have also included a VoiceThread where you can sit back to watch and listen instead. And for those who are into viewing presentation slides with no presenter, I've also included the Google Presenter version of Richard and my presentation.

The Image that Describes It All: How Online Bio12 is Built
My part of the presentation entails me talking for about 15 minutes to this single image. It is meant to represent the courseware structure of the Biology 12 course that was delivered this spring to 372 students in Bio 12 and Bio 110. This slide began as a drawing on my office whiteboard a few months ago as a means to help me get a complete visual picture of how we have pieced this course together. I might as well have titled this "E-Learning Authoring and Delivery in the Cloud" or something to that effect because that is what we have done with Bio12.

Take a look at it, decipher it as best you can and then read the bullet items below for some description. For those who don't feel like reading, skip down below to my VoiceThread version where you can listen to me try to explain the courseware structure quickly.

Content Structure of Biology 12: Spring 2010
Slide08.jpg   

I will describe this from the bottom up:

  • Student Access Through ANGEL: The students' experience entails them logging into ANGEL and accessing their course, like so many other courses. However, ANGEL is used only as a "passthrough" to get to the content. The content DOES not exist in ANGEL. The content actually is created and hosted on the...
  • Blogs at PSU platform: We authored and published the course content in this platform. Biology wanted this course to be "open" so we marked the content with a Creative Commons license. This is the hub of our "course development in the cloud" approach. Take a look at Module 4: Graphs to see the look and feel of a typical module. We embedded rich media within the course and one of our 'anchor' media delivery strategies was to use...
  • VoiceThread: VoiceThread is a web-based application that allows you to create online media albums in which you can upload images, videos and documents. You can then comment on each media element via webcam, microphone or text and then share this 'thread' with others via weblinks. For most of the modules, our SME used it to create annotated presentations. We then embedded these into the content in the Blogs at PSU module content pages. See this example that explains how to calculate Standard Deviation. Another "cloud" app was...
  • Kaltura: This is a video editing, publishing and hosting application service. Our SME was comfortable enough to create a video on his own and simply provide the link. Just like VoiceThreads, we simply embedded it in the content page.
  • YouTube: Our SME found a number of applicable videos and we, once again, merely embedded the video into the content pages. Are you noticing the pattern? For other elements of course delivery, we used...
  • Google Docs: Google Docs is the mother of all web-applications that allow for sharing and collaboration. The design team used it exclusively for development activities but that is a story for another post. For now, I'll focus on it's use in the delivery of the course:
    • Module Assessments: We employed a strategy of students doing their assessment work in a google doc and then handing it in. Watch this tutorial to see it in action.
    • Course Evaluation Surveys: We used Google Forms to create and deliver surveys. This made life easy, trust me.
    • Content/Exercices: We put embedded Google Spreadsheets for students to practice certain calculations. This helped prepare them for their assessment work.

Watch and Listen to the VoiceThread Version
I've created this VoiceThread where I attempt to verbally explain the courseware structure. Click on the play button below to play the VoiceThread within this page. You can also choose to view this VoiceThread in a separate browser where it will be more clear visually.

View the Full Presentation
For those who are somehow still hungry for more, you can spin through the slides of our entire presentation titled "Innovations in the Redesign of Biology 12". Enjoy!

Finally...the Module 10 Group Assignment is Ready to Go

By MATTHEW N MEYER on March 25, 2010 10:31 AM | 0 Comments | 0 TrackBacks
At long last, the final Module for the Intro Bio Lab is ready for launch (goes live March 27). Module 10 is designed to be a unique experience not only for the 372 students but also for all TAs, faculty and the creative instructional design team here in ETS.

Module 10 Design and Scale Issue- Recap
In my previous post about this issue, I described that our design for this module was to have student teams work to research a current scientific topic and present their information via VoiceThread as their final assessment. I also explained (perhaps whined) that this design became extremely ambitious when we discovered there would be 372 students taking this course instead of the 80-110 we had assumed until January. At that point, I realized our ID team would really have to dig deeper than we originally planned to ensure this assessment has the best chance for success. We needed to build that "bigger boat".

Module 10 Design
One important action our ID team took was to establish a relationship with the 5 Teaching Assistants that are responsible for administering this course. One of our "lessons learned" from the Google Doc rollout in Module 4 was that we needed to include them in more discussions and communications about the course design. The TA's have given very valuable feedback that we used immediately. As part of the strategy for communication, I initiated conversations with them through VoiceThread. My main reason for this was to get them using VoiceThread so they would get a clearer understanding of how it works. They responded quickly to this technology. I decided to explain the Module 10 design to them via VoiceThread so that I could a) quickly explain the design asynchronously, b) get their feedback and c) make sure they were prepared for how this assignment would work by actually using the core technology we are basing it upon.

Below is the VoiceThread that communicates the Module 10 design to the TA's for the first time. Click the play icon to learn hear the high level description of the assignment. And feel free to comment!


Getting There
Back when we assumed we'd be dealing with about 80 students, we thought even at a high level this was a pretty big scale design idea. When we learned it was 372 students, we knew it was a big scale idea. As Tyrone would say, "no worries" (I'm not sterotyping here because he tells me that all the time). We decided we were committed to the concept and simply figured out our best methods for ensuring success.

We decided from an account setup perspective that we would take a centralized approach. By this I mean that I would set up the VoiceThreads, the student accounts and managing the assignment of appropriate users to each group. So here's what I ended up doing:

  • Export each section roster from ANGEL into .csv file. Repeat 16 times total.
  • Massage each csv file to ready it for import to VoiceThreads management account (mine).
  • Call VoiceThread and beg for 350 additional credits for free (saving about $650)
  • Import all 16 csv files into VoiceThread (easy).
  • Create a 'master' VoiceThread 'stub' and get approved by Richard.
  • Make 92 more copies of the master VoiceThread giving each a distinct title that will indicate class, section and team number.
  • Get TA's to use ANGEL team creation function to make 93 teams.
  • Copy/paste team data from ANGEL to Google Spreadsheet. Turn into 16 separate worksheets and then sort each worksheet by team #.
  • Print worksheets out to ready for team assignments
  • Assign an average of 4 students to each of the 93 total VoiceThreads. This took about 4 hours and was simple but not too enjoyable.
Other than that, this was pretty easy. It was clear to me that if this assignment is to ever be used in the future, it cannot be done this way. I just can't imagine any faculty member having that kind of time to do this setup. What can be done is to introduce VoiceThread earlier in the semester to students so that when it comes to this type of assignment, they can create and share their own VoiceThreads. But, for the sake of making it as easy as possible for students in this go around, I did it for them.

So, the cart has been pushed up to the top of the hill, admired for a brief moment, and pushed down the other side. Gravity is now in control. The students are off and running with it. I'll try to keep it's story updated.

Results of Student Responses to Modules 4-6

By MATTHEW N MEYER on February 26, 2010 10:23 AM | 1 Comment | 0 TrackBacks
Per our assessment plan for Intro Bio Lab (formerly Bio12), we delivered the second (of three) course evaluation surveys to students after they completed module 6. This is a significant survey of student feedback because, starting with module 4, they were introduced to maneuvering in Google Docs to perform data calculations and create assessment documents which they would convert to PDFs and hand in to their ANGEL dropbox. In short, this is the first survey of their thoughts after we had taken them out of their comfort zone. The research question for this survey was something to the effect of "how pissed are they?"

First, something smart that we did: used Google Forms
Before getting into the results, I wanted to explain our "smart move of the day" with this second survey. This time, we delivered the survey via Google Forms. We had delivered the first one via the survey tool in ANGEL. The biggest problem with that method was that we had to compile survey data from 16 different sources and do our own compiling work. That was painful. This time, we used a Google Form so that all data came back to one location and allowed us to share this data in real time with anyone we shared the Google Form with. It worked beautifully and saved more than a few hours of data labor on our end. Call that a lesson learned for this project.

They liked Modules 1-3...of course
Relatively speaking, modules 1 to 3 were easy both in content and assessment delivery. The assessments for those modules (module 1 didn't even have an assessment) were created and delivered in standard ANGEL format and many students already are familiar with the question format offered via that tool. Their responses reflected our expectations for their experience of modules 1-3 which was basically "we like this".

They still like the course after Modules 4-6, just not as much
This survey, however, catches students after taking 3 modules (4 through 6) in which not only was the content more challenging but they also had to learn how to use Google Docs and Spreadsheets to do their assessment work and hand it in. In this survey, we had three 'carry over' questions and two new questions specific to working with Google Docs.

As expected, the 'lovefest' of the course we measured after module 3 has softened. The good news, however, is that it's not over. If you take a look at our summary results, you'll see that the favorable ratings are still well ahead of the unfavorable ones but the gap has narrowed.

Comparison of our Carry Over Survey Items
We had three questions that were carried over from the first evaluation survey we delivered. You can view the comparison for these questions for yourself. Overall, the favorability of the course survey items show a slip of about 22% and the unfavorable items with around a 10% increase. The good news is that the overall favorability of the course is still much higher than the unfavorable, just not as much as it was after module 3.

Student Impressions of Using Google Docs
Going into this course, I was very nervous about student reactions to using Google Docs, especially for something as important as their module assessments. This course is such a paradigm shift as it is and I'm guessing many if not all students had no idea how this course was to be delivered. Basically, my worst fears were that students would wholesale reject the idea when we asked about it. Luckily, these results did not show that at all. In fact, 50% of the students who responded felt that using Google Docs "was easy" compared to 31% who disagreed with that. It's not a home run yet (in my eyes, anyway) but I think this sets us up well for the end of course survey that will ask them about Google Docs again.

Student Impressions of support materials for Google Docs
To support students use of Google Docs, we had produced a number of narrated tutorials in the module content to show them how to perform a number of functions. After talking to the TA's after Module 4, it was clear the 'mechanics' of Google Docs operation were not getting through to the students. One of our key corrective actions was to create new tutorials that were more specific to the basic operations for getting started with Google Docs. We also made them short and to the point. Most importantly, we provided links to these tutorials from within the Assessment document themselves, starting with the Module 6 Assessment. I think it was this corrective action that explains why 60% of respondents agreed that the learning material and screen casts were enough to help them work in Google Docs (only 22% disagreed). 

Open Ended Questions
We asked the same two open-ended question in this survey as we did in the first one:

Q6: What would you change in modules 4 through 6 to improve its effectiveness for future classes?

Results are out of 24 responses:
  • More defined directions: 33%
  • No changes: 21%
  • Modules should be taught in  class: 13%
  • Start using Google Docs earlier: 8%
  • Don't use Google Docs at all: 2%

Q7: Add other comments you feel would be useful- both negative and positive comments are fine

Results are out of 12 responses:
  • Other (submitting assignments via google docs was strange, course was easy to follow,would like each module to be open for 2 weeks, would like feedback on assignments before turning in the next assignment): 50%
  • No comment: 25%
  • Class should not be taught online: 25%


In summary, I think these results are about what we expected. Perhaps better than expected from my viewpoint. Not a lot better but just a little better. As I've posted before, we have used the feedback we are getting to install 'fixes' to the modules students haven't taken yet. In fact, it is really helping our ID team come up with a much tighter plan for delivering Module 10, the group assignment where student teams will deliver their findings via a VoiceThread.

Intro Bio Lab: Submission Rate for First 2 Google Doc Assignments

By MATTHEW N MEYER on February 24, 2010 4:38 PM | 0 Comments | 0 TrackBacks
As of right now, the Intro Bio Lab students are in the middle of Module 6. They have survived the first 2 modules (Modules 4 and 5) worth of Google Doc assessments and are working on their third. I thought I'd provide a brief analysis of how students have performed in regards to completing their Google Doc assignments and properly putting them into their class's ANGEL dropbox. This analysis is based on data from all of the Bio section's ANGEL dropboxes in each of the 16 sections of this course (4 for Bio 12 and 12 for Bio 110). This analysis only addresses the issue of assignments 'handed in' (which we will term submission rate), not other metrics such as grades, grade distribution, etc. This post covers what has happened from Module 4 through Module 5.

Note: You can view our most up to date submission rate information for yourself.

Module 4 Assessment Submission Rate: Results
Our overall data for, the submission rate of the Module 4 assessment was 82%. Simply put, this means that 82% of the total students in 'Intro Bio Lab' properly turned in their Module 4 assessment. Broken down by course, it was:
  • Bio12- 86%
  • Bio110- 81%
While this rate was "not too bad" considering this was the students' first Google Doc assignment, we were generally concerned that it was not higher than this. Richard Cyr indicated he would like to get these rates up and over 90%. 

Working To Improve These Results
As this data rolled in, the students were already moving through Module 5. Richard and I met to discuss remedies to help raise the submission rates for all sections. The first thing Richard did was to set up a face to face meeting with the 5 TA's for this course to solicit their feedback and thoughts of their experience with Module 4. I was able to join in via Skype call and after hearing their thoughts, our immediate action plan was this:

  • Change the module availability schedule. Each module to that point was made available to students at midnight on Monday and available until the following Sunday at midnight. The TA's felt that most students put off doing the assessments until the last moment (Sunday night) and that many had questions about the Google Doc mechanics. Because it was Sunday, many of these last-minute questions went unanswered until Monday morning. Everyone agreed that it would help students and TA's to manage their time better by making the next module available on Friday midnight and give them until the following Friday at midnight to turn it in. This will provide the TA's time on their weekend to do their grading and provide any feedback to students who may still need it.
    • Status: Done and implemented
  • Activate Chat Rooms. Richard and the TA's agreed to set up ANGEL chat rooms in the sections so that students may be able to help each other with the Google Doc mechanics of the assessments. TA's would make sure to monitor them so that students were not just giving answers to each other.
    • Status: As of today, I have not seen any chat rooms set up in the sections.
  • Schedule Mandatory Face to Face Session with 'Non-compliant' students. Richard will lead a meeting of all students who have not handed in an assessment for any of the Modules to this point. His primary message will be to inform these students that they "must to the work". He will also use it as an opportunity to demonstrate the Google Doc operations and as a Q&A session. 
    • Status: Scheduled for 3/2/10 from 6-7:15 in 12 Boucke.
  • Give non-compliant students opportunity to hand in past due assessments with only a small 2 point penalty. They decided that they will provide a window of time for non-compliant students to hand in past due assessments with only a small penalty. The justification, simply, is that this is a 'new' course and that we understand that they are being asked to do some very different things than past classes.
    • Status: TA's currently administering this.
  • Develop new, shorter Google Doc tutorials that will be available within each remaining Assessment Document. The TA's said that while they understood there are Google Doc tutorials within the module content, they observed that students are simply not watching them. To bolster student's access to just-in-time help, we created new tutorials the very basics of how to work in Google Docs. These were produced by showing the student first in ANGEL and then proceeding through the Google Doc process. The TA's felt these would be a great help.
    • Status: Tutorials complete and were available to students starting with Module 6 assessment on 2/19.
Module 5 Assessment Submission Rate: Results
As we developed a plan to help with the submission rates, the student were going through Module 5 so there was no opportunity to make these steps operational for that particular module. Fortunately, the submission rate for module 5 went up without intervention to 88%, an improvement of 6%. Broken down by course, it was:
  • Bio12- 84% (down 2)
  • Bio110- 89% (up 8)
We are glad to see the overall rate increase but we still would like to see it bump up over 90% for Module 6 and then further after Module 7 after Richard conducts his mandatory session. I will report the results of Module 6 early next week.

All Data In for Bio12 Modules 1-3

By MATTHEW N MEYER on February 15, 2010 8:47 PM | 1 Comment | 0 TrackBacks
Last week, I posted on the initial results of the student feedback for modules 1-3 of the Intro Bio Lab online course. We now have the data in from all 16 sections of Intro Bio Lab (4 for Biol 12 an 12 for Biol 110).

Overall, we ended up with a total of 184 responses out of 380 total students for a response rate (if that is the correct term) of 48%.

Q1:The way the content was presented in these modules makes it easier to understand how the study of biology is relevant to my life.

Q2:I found the media content (VoiceThreads and Videos) enhanced my learning more so than just reading about it.


Q3: How easy was it to navigate the website?

Q4:  Have you enjoyed the course to this point?

If you read my post from last week, you will notice that the results change very little from the data that only represented the 4 Biol 12 sections. If you'd like to look at the raw spreadsheet of this data, you can view it here. In the raw spreadsheet, you can compare the Biol 12 sections with the Biol 110 sections.

To this point, we are pleased with the student feedback but not too surprised. Last week, the students went through Module 4 where they encountered their first Google Doc assessment. This is where this course starts to get interesting.

Open ended questions
We asked two open ended questions in our survey. You can review ALL comments students made to both of these questions for modules 1-3. We took the liberty of quantifying the responses as follows:

Q5: What would you change in modules 1 through 3 to improve it's effectiveness for future classes?

Results are out of 131 responses:
  • Do nothing; no changes needed: 49%
  • Improve assessment question wording: 12%
  • More practice and work examples: 11%
  • Improve narration of VoiceThreads: 6%
  • Other: 22%
Q6: Add any other comments that you feel would be useful- both negative and positive comments are fine.

Results are out of 99 responses:
  • Everything has been fine: 23%
  • N/A; no comment: 13%
  • Enjoying convenience of an online lab: 9%
  • Need to improve quizzes/assessments: 8%
  • Need more feedback/communication from TA: 7%
  • Other: 39%

Bio12: Initial Data of Students Feedback to Modules 1-3

By MATTHEW N MEYER on February 12, 2010 1:42 PM | 0 Comments | 0 TrackBacks
We have compiled the first batch of data from the students taking the new online Bio12, er, I mean 'Intro Bio Lab' course for the Spring 2010 semester. The students were to complete a quick survey after taking the first 3 modules of the Intro Bio Lab course to provide feedback on their perceptions of the course to this point.

This data covered in this post only represents data from the four (4) sections of Biology 12 (we are still working on the 12 sections of Biology 110 that are also taking this course). Another reason to separate Biology 12 from the 110 sections is that we've been told that for the first time starting this semester, Biology 110 sections consists only of juniors and seniors while Biology 12 is made up of freshmen and sophomores. The response rate for the Biology 12 sections was 40% (34 of 94 students submitted feedback).

How We Compiled
We delivered this survey through the ANGEL survey tool to students at the end of Module 3. We decided to use ANGEL's tool to use it's data export feature assuming it would be simpler than using Google Forms. However, this decision was made before we discovered that an additional 12 sections of Biology 110 would also be taking this course. We assumed we only needed to collect and integrate 4 sets of data, not 16.

I tried manipulating the exported .csv files from the 4 Biology 12 sections but I am not a spreadsheets guy' so it became difficult pretty quickly. Instead, I went into ANGEL's 'Response Summary' layout and copy/pasted the data into a Google Spreadsheet. Why? It was easier for me to understand and manipulate. In addition, I wanted to use the chart features in Google Spreadsheets to create graphs and then publish them for the web, as you will see below.

For the reasons above, we are leaning toward using a Google Form to capture students feedback after they complete Module 6.

Here are the results of the Module 1-3 survey to students for the Intro Bio Lab online course.

Q1: The way the content was presented in these modules makes it easier to understand how the study of biology is relevant to my life.


Q2:I found the media content (VoiceThreads and Videos) enhanced my learning more so than just reading about it.

Q3: How easy was it to navigate the website?


Q4: Have you enjoyed the course to this point?

Here is a link to see the responses to last two questions of the survey, which are: 
  • Q5: What would you change in modules 1 through 3 to improve its effectiveness for future classes? 
  • Q6: Add any other comments that you feel would be useful - both negative and positive comments are fine. 

Quick Analysis 
These findings are very positive: 
  • It seems that one of the objectives of the design, to make science content relevant to students, is working as almost 70 of students either agree or strongly agree with that objective. 
  • Not surprisingly, just over 60% of students feel the media is more effective than reading. Personally, I take from that response that the media has not been a hinderance to their learning more than students liking it more than reading. I think it means we designed the media well. If we had not, fewer students would agree. 
  • Students felt the navigation was easy and, to this point, almost 70% of responding students have enjoyed the course to this point. We will keep an eye on this number in the next 2 surveys because there's a good chance this number will drop once students enter the more difficult modules of 4-6 as well as navigate through their Google Doc assignments 
Overall, these results are very encouraging. Stay tuned.

We're Going to Need a Bigger Boat...

By MATTHEW N MEYER on February 5, 2010 1:14 PM | 3 Comments | 0 TrackBacks
Last week, I discovered that one of our very first assumptions about this Bio 12 project was a bit, uh, shall we say "off". During the discovery phase of the project, I was told this Bio 12 course would be delivered to 80-110 students. This week, we found out that number was off by about, oh, 300% give or take. Yes, the actual total number of students currently enrolled to take this course is 380. 

I think this scene sums how I felt:

 

Once I picked my jaw (pun intended) off my desk, it was time to think about impacts. Here are my thoughts:

  • Overall Course Content: No impact. If 380 students had few to no problems in the first 3 modules, they shouldn't have many in the next 7.
  • Modules 4 through 9 Content Assessments: Possible high impact. After doing some quick math, it occurred to me that each TA will be responsible for 76 students. After a quick chat with Cole about this ratio, he confirmed that ratio, 76:1, is "about right" for large enrollment classes. Knowing that this ratio is "about right" put me at ease about the workload for these TA's. Apparently, they should be ready to grade 76 assessments that will be put in an ANGEL dropbox by each student.
  • Module 10 (Final) Assessment: High Impact. Just some brief background: In the current Module 10 design, students will work in teams over the course of 4 weeks to research a scientific issue and then present their analysis in a 'team' VoiceThread that they will 'hand-in' by sharing it with their TA. The main reason this has a high impact is the combination of two main factors:
    • The fact we are introducing a 'new' technology to students (VoiceThread) to create their final project in a team environment was daunting enough when we assumed 100 students would divide into a total of maybe 25 teams. In this course, students are learning to use the Google Docs platform starting with Module 4 and we finish the course by having them use a second new platform. Student anxiety may already be a tad high.
    • The needed resources to help and support 380 students (95 teams) is a scale issue we hadn't planned on. Simply put: more students using VoiceThread = more technical/design support necessary. Assuming most student teams will wait until the last minute, we have the potential for a major 'slam' on our team and the Biology TA's to help everyone in the amount of time left to complete their projects. This is similar to what the Media Commons experiences the last one to two months of a semester. The problem with this is that we did not plan on utilizing Media Commons resources to 
How We Are Designing Module 10 Now
The Biology ID team is currently fine-tuning the design for the Module 10 Assessment. Even before we found out how many students were taking this course, we had planned on figuring out ways to make the steps of this process as seamless as possible for students and TAs. That objective is even more important now. I will have more on the Module 10 Assessment soon, but I will say that we plan on keeping the group project aspect of it now (perhaps even more important now) as well using the VoiceThread platform as the deliverable.

With all this in mind it's time to go get that bigger boat.

Assessment Plan is a Go

By MATTHEW N MEYER on February 1, 2010 3:11 PM | 0 Comments | 0 TrackBacks
Last week, I finally met with Richard Cyr regarding the status of the Bio 12 course, which is now in it's 3rd week (students are taking Module 3 as I write this). Here are the key takeaways:

  • His TA's are reporting little to any issues at this point. In fact, they are reporting very few work hours in administering the course so far. I was pleased to hear this but not surprised. The students have not had to do significant work yet as Modules 1-3 are relatively 'easy'.
  • Richard reviewed the assessment plan and approved moving ahead with it. His only change was to not start assessing his TA's until after the completion of Module 3. 
    • On Friday, I created the first faculty/TA survey and submitted for Richard's review. I'd like to determine the delivery strategy of that survey. I'm assuming, for now, that he would like me to email it to his TA's.
    • I also created the student assessment survey in ANGEL now that I have access to the live course.
  • We are both reaching out to Tyrone to send us the content for Modules 8 and 9. This has been an outstanding item for this project for 2 months and we have reached a point where we must have it in our hands in order to turn it around and make it ready in approximately 4-5 weeks.
    • Follow up- I finally received an email from Tyrone today (2/1/2010). He will be getting back to port soon and will begin assembling the content for modules 8 and 9.
  • I notified Richard of the issue that ANGEL has with the Safari browser. Students are unable to hand a document into a dropbox in ANGEL if they are using Safari.
  • I demonstrated to Richard how his staff will be able to use the 'Revision history' feature in Google Docs to spot-check student's work in order to determine if they are entering their own answers or merely making a copy of someone else's google doc.
Expectations for first batch of assessment data to arrive
Assuming that students have through Sunday, February 7th to complete Module 3 and the TA's will take the early part of the week to do grades, I expect to start seeing TA survey data come in sometime around mid next week (Feb 10th).

I'm also hoping that the TA's will direct students to complete their course assessment survey available at the end of Module 3. 

For Modules 1 through 3 feedback, I expect the data to show that students are having few problems accessing and learning from the content objects within each module. These modules are constructed in a fairly straightforward manner and the assessments are traditional ANGEL assessments. In other words, there's probably not much that students haven't seen already. I will, however, be interested to see how they react to the VoiceThread presentations that we've designed in these modules. I will keep an eye on the 'instant feedback' data capture mechanism we have in the course (a Google Form available in every module) to see if the submissions increase; it has been very light to this point but I expect it to pick up now.

I expect next week (Module 4) to be full of activity for the Intro Bio Lab TA's. This is the first module where the assessment is delivered via Google Docs. Stay tuned...

Bio12 is Underway: Now for an Assessment Plan

By MATTHEW N MEYER on January 25, 2010 3:39 PM | 0 Comments | 0 TrackBacks
The Bio12/Intro Bio Lab course has been underway for over a week now. I've been monitoring the 'Course Feedback' comments since it started and the feedback has been minimal; only 5 students have bothered to actually submit a comment. Most of the 5 are pretty positive. My favorite is the last one where a student wonders "How is an online lab going to teach us how to use basic scientific instruments?" (Answer: It won't). I'm interested in seeing if this student submits future frustrations with this new design and delivery. Normally, I'd put the student feedback link here for anyone who is interested to review all comments but since students include their names in the comment, I think it best to not do that for now.

Faculty Feedback?
I have also not heard much from any of the TA's assigned to administering this course. I'm hoping to be included in their feedback loop after a meeting with Richard this Wednesday. But, as of now, I have very little feedback on this course. My impatient nature is being tested...

Assessment Plan: First Draft Ready
Our design team has just finished a draft of an assessment plan for this course. You can read it here or come to the Bio12 project management site and select the 'Assessment Plan' tab to link to it from there. As I've said, it is a draft and I'm hoping to get further input from Vicki Williams, our ETS Assessments expert, this week. I'm also showing it to Richard Cyr in our meeting this Wednesday and hoping for some good feedback from him as well. Originally, we had expected our subject matter expert on the course to create the assessment but he's a half world away now so we figured we'd take matters into our own hands. There's a lot at stake with this course and we wanted to be proactive. My goal is to have this plan ironed out and agreed upon by the end of this week (I might be a bit overly optimistic but, hey, it's a goal) and begin creating the assessment instruments early next week.

I have to admit, I get the feeling the assessment plan is just a bit 'off' for some reason. It's just that nagging feeling it's missing something. If anyone would like to offer their feedback, please feel free to do so either with comments to this blog or using the Comment feature in the Google Doc.
Archives

Search This Blog

Full Text  Tag

Recent Comments

Recent Assets

  • Slide08.jpg

Tag Cloud