Disaggregation and Higher Education Policy

By KYLE PECK on April 18, 2009 11:15 AM | No Comments | No TrackBacks
IIt's Saturday, April 18, 2009 -- a beautiful Spring day, probably the first day we'll hit 70 degrees this year.  I'm at the Penn State conference center with about 450 others who have dedicated the day to thinking together about teaching and learning.  (That makes me proud of Penn State!) 

David Wiley, from BYU (one of the leading advocates for "Learning Objects" and an all-around great guy) was our keynote speaker.  David opened the day by proposing that education's monopoly is endangered by others doing what we do, better, cheaper, and more conveniently.  He cited good examples, and proposed that we (formal education) look weaker and weaker each day, as those outside move forward and we do what we have been doing.  He cited certifications by the likes of Intel and Cisco as examples, which compete well with a BA in Computer Science and he cited a course he offered in which he invited others from around the world to participate in a course, using blogs and Wikis.  About 60 "outsiders" participated, enriching the course, and at the end some of the outsiders who had completed all of the assignments asked for some recognition.  He offered them a certificate to acknowledge their efforts, and they accepted.  Then, The Chronicle of Higher Education found out about it and wrote an article on it...  "When Professors Print Their Own Diplomas, Who Needs Universities?"  Interesting.

Dr. Wiley forecasts "disaggregation," in which the pieces of what we currently do pull apart.  The content, much of which is now offered openly (see the MIT Open Courseware project, for example) might be one component.  A second component would be assessment of student learning, and places like Western Governor's University (which offers no courses, only assessments) stand ready to provide the assessments and give grades and credits once students demonstrate proficiency.  Why would the content and assessment need to come from the same place?

David encouraged us to engage in policy change, proposing that if we do not, student learning may suffer and/or the students may turn to a new set of markets to get what they need, and proposed that our employment may be in danger.

As a good keynote session should, his talk made me wonder...

Will Penn State understand that its primary role is no longer the delivery of content, and see the primacy of assessing what students know and can do, and certifying that they have mastered content and higher-order skills?    Will we understand the importance of creating connected learning communities that support students and the learning process?

Is it time for Penn State to dust off and enhance its policies on "Credit by Examination" and "Credit by Portfolio Assessment?"  I think so!    Do we care WHAT students know?  I think so.  Do we care WHETHER they can perform?  Yep.  Do we care WHERE they got the knowledge and skills?  Nope. 

Proof that the Media Doesn't Get Assessment

By KYLE PECK on March 10, 2009 10:04 AM | No Comments | No TrackBacks
Geez.  What a disappointment.  After months of hard work by a team led by Dr. Rayne Sperling and Dr. Jonna Kulikowich, researchers at Penn State, a report examining the quality of school districts' alternative assessments for high school proficiency was released.  That report was then covered so badly by WTAJ in State College that I had to bring it to your attention.  The report can be found HERE, and the TV report on it can be found HERE.  

WTAJ's x minute coverage contained no fewer than 7 serious misrepresentations of a simple executive summary and press release, which I believe illustrates how little non-educators know about assessment and/or how little effort goes into the work of some "journalists."
 
Despite a note detailing the problems and requesting an on-air correction, the article remains visible to the public.  

Some of the errors, as communicated to WTAJ, include the following:

1)  This was not a study of "test scores and graduation rates at every high school in Pennsylvania" but a review of the local assessments that each school district provides to students who do not pass the PSSA's in 11th or 12th grade. These assessments are required by the State Board of Education, and districts must develop them. Your news report never even mentioned that these assessments exist and were the focus of this study.


2)  "Giving kids diplomas even if they don't make the grade." This is not true. The districts are graduating kids who have passed the district's local assessments. What researchers found is that many of these assessments are not aligned with current standards for math or reading, and/or may not yield a valid measure of assessment.

3)   "18 districts held back kids that didn't pass a state test." This is completely false. Researchers found that 18 districts offered local assessments that were valid in standards and practice. This study did not collect data regarding individual graduation decisions in any district.

4)   "400 districts allowed kids to get their diplomas despite failing scores." This is not what the study discovered. Again, they did not collect data regarding students, their graduation decisions, or their test scores.

5)  "400 districts allowed kids to get their diplomas despite failing scores. That came out to 56,000 students." This is not an accurate reflection of what the 56,000 students are. The press release, which I distributed yesterday explains clearly what this number is: 

"The Pennsylvania State Board of Education requires that districts offer a local assessment that provides an alternative method for students to graduate if they do not successfully meet proficiency of standards through the Pennsylvania System of School Assessment (PSSA) 11th grade administration and 12th-grade retake.  In 2007, more than 56,000 Pennsylvania students graduated with the local assessments as one requirement to earn a high-school diploma."

Clearly, the 56,000 students were graduated statewide using these assessments, and represent all districts in the state, not just the 400 you reference in your report. 


6)   "Penn State recently found"  - Although this study was conducted by Penn State faculty, they used expert panels of educators from across Pennsylvania to determine whether these assessments were aligned with standards.

7)  "If they still can't pass, they are not supposed to graduate." This is not true.  As stated in the quote above, school districts are required to provide an alternative assessment to help students graduate.

The fact that they haven't pulled the story from the web is very interesting.  Your thoughts?

 * 

Daniel Pink: The Disconnect between schools and the World Outside

By KYLE PECK on February 10, 2009 7:57 AM | No Comments | No TrackBacks
I'm watching Daniel Pink (author of "A Whole New Mind" and former speech writer for Vice President Al Gore) deliver a keynote presentation at the Pennsylvania Educational Technology Expo & Conference (PETE&C).  

PA Secretary of Education, Dr. Jerry Zahorchak, introduced Daniel, sharing a few notes on Pennsylvania's progress and plans for the future.  He noted that, even in these tough times, Governor Rendell has managed to sustain a high level of educational funding, including $20 million for the Classrooms for the Future program.  

Daniel Pink began by noting that he is an outsider, not an expert on education, and noted that "education as preparation for the workforce" tends to dominate the conversation on education.  

His major premise is that the requirements for success in the future are changing.  He uses the brain as a metaphor.  He says that our brains are elegant and specialized.  He described the left brain / right brain differences.  He said that the left brain is about analysis and the right brain is about synthesis, and that in the past the main attributes that were valued were left brain abilities.  While these left brain abilities are still required, ("absolutely positively indispensable") they are "necessary but not sufficient" for success in the future.  Artistry, empathy, and the big picture are now also crucial.

Pink described himself as a left-brain dominated guy, who loves numbers, charts, graphs, etc., but he feels that big changes are underway.  He laid out a cause/effect argument.  He says that the shift has three causes: Asia, Automation, and Abundance.  "Off-shoring" has been overhyped in the short run, he believes, but is understated for the long haul.  At the moment, India has an advantage, in that their salaries are low and they have a billion people.  The top 15% of India's workforce is larger than our entire workforce.  Next year the world's largest English speaking population will be in India.  And, there is now approximately no cost to communicate with them, a fact that enables workflow.  This makes the work that is "routine" (work that can be sent to the cheapest cost provider) vulnerable to off-shoring.  Unfortunately, this routine work (work that has a process and a right answer) is what we were advised to seek and trained to do.  The left hemisphere, Pink proposes, gets offshored or automated.  

Automation:  Machines are relapcing our muscle.  Pink used as other examples jobs that don't require muscle that can be automated, and described and displayed web-based systems for getting divorces and doing taxes.  

Abundance:  We are going through a tough period right now, but there is still abundance, also described as "material prosperity," and that is not likely to end.  He used the growth in the presence of refrigerators and TV sets as examples.  (There are now more TVs than people in the typical American home.)  He also talked about the proliferation of cell phones, and reported that today's cell phone contains more computing power than existed in our grandparents' time.  He used iPods as another example, asking how many people had them (almost all) and then how many people knew they were missing one eight years ago.  

To make it today, Pink proposes, our students' futures are determined by 3 questions:
  • Can someone overseas do it cheaper?
  • Can a computer do it faster?
  • Is what you are delivering in demand in an age of abundance?

He proposed that the jobs we should be preparing for should be high tech, high concept, and/or high touch, and lamented that the business world is about novelty, nuance, customization, while school is about routines, right answers, and standardization.

According to Pink, the six abilities that matter most are:
  • Design
  • Story
  • Symphony (seeing the big picture)
  • Empathy (standing in someone else's shoes)
  • Play (laughter, humor and games)
  • Meaning (not just accumulation)

Pink offered five ideas about Education, or "How to prepare students for their future, not our past." 
(a slogan I picked up about 20 years ago from a Superintendent from the Pittsburgh area, Sam Sava.)

1.  Experiment with new metrics -- What gets measured gets done.  We need to develop metrics for right brain attributes, and highlight the limitations of left brain metrics.  He mentioned the Rainbow Project - Robert Sternberg -- an alternative SAT -- funded in part by the College Board, that asks very different questions than a typical test.  For example, it will give them a title and ask them to write a story, or give them a cartoon and ask students to develop a caption.  Or, it might ask them for ideas on how to get friends to help them move furniture.  He cited another example, the JSPE -- JEfferson Scale of Physician's Empathy.  The scores on that scale correlate better with patient outcomes than other predictors of success in medicine, including measures of content knowledge.

2.  Get real about STEM -- Pink proposes that we have a dangerously narrow view of what STEM subjects are all about.  They are not about right answers, but are about observation and asking the right questions.  He cited medical schools that are adding art education precisely as education is removing it, saying that we need "non-routine savants."  Creativity.  What does Creativity mean?  He described a tally of the responses of superintendents and employers.  Superintendents' top ability defining creativity was problem solving, while employers said it was problem identification.

3.  Tear down those walls -- The MVP (most valuable prefix) is "Multi."  We need people who can do a whole lot of things.  Listen to what the STEM world tell us:  Engineering employers said that they want engineers with passion, systems thinking, and ability to work in a multi-disciplinary environment and the ability to change.  (I missed a few other attributes, because I couldn't type fast enough.)

4. Infuse arts education throughout the curriculum -- It's not supplemental, it's fundamental. He said that the Chinese get this, and he cited an expert from China: "Creative arts are no longer a frivolous luxury, but are essential to achieving a competitive edge." 

5.  Promote and defend autonomy --  Educators have much less autonomy than most professionals, and students have too little as well.

He ended by thanking teachers, as the people who are going to move students into the future.

His remarks were very well received by the audience and the Secretary of Education.  Let's see if there are any longer-term effects!

Jason Ohler at the Pennsylvania Educational Technology Expo

By KYLE PECK on February 9, 2009 8:39 AM | 1 Comment | No TrackBacks
Jason Ohler has come down from Alaska to share his stories with us.  His first point was that the great teachers in his past were "door openers."  He says that today's students are banging on the door and begging teachers to allow them to use technologies in the act of learning.

"Students need to be able to write whatever they read."  If they read television, they need to be able to write television, etc.  You are not literate until you have command of the media collage.  Students need to write a lot in pursuit of developing great media collages. 

He says that Web 3.0 will be the "semantic web," in which what is returned by a search is a report, not a collection of separate items.  Much of the work of synthesis will have taken place before we see the response to a search.  

He predicts that writing will be more valuable than ever.  The ability to craft really good wrigint will go up in importance.  It will not be the end product, but it will be a crucial element in the path to the final product.  He teaches "visually differentiated text" (text with headings, boxes, etc., to make it more useful.  He proposes "Art" as the "4th R"  Reading, wRiting, aRithmetic, and aRt.  

He also says, "Attitude is THE aptitude," and encourages us to help students put things in perspective.  He feels that the goal must be fluency, not literacy, and he encourages us to "harness both report and story... embrace story."  Kids come to school wired for story.  At school they don't get stories, they get information in list-oriented formats.  Teachers need to take units and turn them back into stories.

A very impressive thing he does with students is "Green Screen Storytelling," in which he has students tell stories in front of a green screen (that he created by painting a wall of his classroom) and then produce art that is displayed behind them in dynamic presentations.  Wow!  A simple but powerful idea -- if the story is good.  He leaves the last 20% o the time, to the kids for tweaking and the technology side (leave the clicks and the tricks to the kids) and he concentrates on helping them tell the stories well.

As for assessment, he cares deeply that the teachers not be captivated by "anything that moves," but rather concentrate on helping students think about how well the product tells the story and moves the audience.  Assess the researching, the writing, the planning, the media fluency, and the literacy blending.  "Never yield executive producer status."

I would like to see this take root in Pennsylvania, and am willing to help teachers who want to make this happen with their students.  Contact me if you want to collaborate on this!

Jason Ohler speaks with passion, knows what he is talking about, and cares deeply about what he is doing and what we are doing as educators.  Watching his presentation was time well spent!!

An "Assessment Impasse?" Why We're Not Making Progress in Assessing 21st Century Skills

By KYLE PECK on February 8, 2009 11:56 AM | No Comments | No TrackBacks
In what I consider to be an excellent article titled "The Assessment Impasse" (Inside Higher Ed, February 5, 2009), Merilee Griffin describes a "longstanding pressure" that advocates of assessment for the purpose of accountability have placed on advocates of assessment for program and institutional improvement."  She points out that Arne Duncan, the new Secretary of Education relied heavily on standardized testing to drive the educational reforms in Chicago.  

All of that was interesting, but what she did that I really appreciated was turn our attention to the crux of the problem -- the fact that "when it comes to measuring learning outcomes in the higher cognitive skills, we literally don't know what we are talking about."  She points out that in assessment, the first step is to define what it is that students are to learn, but although we can agree that critical thinking and other higher order skills are important, we have not yet come to agreement on what these 
labels mean, and what students would be able to do if they had developed different levels of proficiency.  Amen.

That's a critical thing to understand if we are ever going to get started in making progress in this area, and if we don't take that step, education will continue to be driven by standardized tests and other assessments covering domains we have come to understand.  Ms. Griffin, a doctoral candidate at Michigan State University, makes a series of seven recommendations for how we should go about the work of defining 21st Century Skills, and I applaud her for getting that conversation started with what seems to be a plausible set of recommendations.  

Let's see if these ideas can get a little traction as we begin a new era of American education!  Count me in.  Thanks for your contribution, Merilee.

An "Obama Effect" on Educational Testing?

By KYLE PECK on February 8, 2009 11:10 AM | 1 Comment | No TrackBacks
I just read an article that I found rather stunning.  I knew that teacher expectations made a big difference in education, and I also knew that motivation and self-concept were important factors, but I really didn't understand the magnitude of self-image until I read "Education is All In Your Mind" in the New York Times.  

Among other interesting evidence, the article cites an as yet unpublished study in which, "Adult subjects ... answered comprehension questions from the verbal sections of the Graduate Record Examinations before and just after the presidential election.  The black participants who were tested before the vote performed worse than whites; those tested immediately afterward scored almost as well as whites."  I look forward to that study's publication, and to other evidence of the importance of self concept and academic performance.

In the meantime, one of several questions this raises for me is...

If performance on tests (even on those with good reliability and validity like the GRE) vary significantly based on affective considerations, how well can they be measuring mastery of the content?

Your thoughts on this are welcome.

Do Classrooms Grades Accurately Represent what Students Know?

By KYLE PECK on February 8, 2009 10:44 AM | No Comments | No TrackBacks
For a long time I have felt that teacher-made tests and factors like attendance, class participation, timeliness of submitting homework, and others that teachers often use in determining a student's grade have made class grades a better representation of student compliance than of what students have learned about the course content.  

A recent article in the Atlanta Journal-Constitution titled, "Are Schools Inflating Grades?  Marks from Teachers, Tests Vary Widely" has documented the problem well. 

I would love to hear from you after you've read the article, to see if you are willing to defend course grading or whether you think that there should be common tests used in courses offered across the nation that should be used to determine grades.

Or, if you want to come out against grading students in general, that's fine, too.  (I feel that grading is perhaps best used on "finished products" like eggs and meat, and that we should probably just track and report students capabilities without attaching a "grade" to such descriptions.)  What do you think?

Mastery Learning at Penn State?

By KYLE PECK on February 5, 2009 9:25 AM | 1 Comment | No TrackBacks
Cole Camplese, Suzanne Beinert, and I were in a session at the Penn State "e-Education Council" meeting today, where we heard a report by professor James Sellers of the math department, in which he described the results of their new mastery learning approach to Math 21.  The slides are available here: Math21-EEdCouncil.ppt.  As you will see from slide 5, the program resulted in far more students getting a grade of "A" in the course, and approximately the same number failing the course.  They allow each person to take each of the 12 tests up to three times, and when they reach 80% or higher the content is considered "mastered" and the student gets one point.  At the end of the semester, students who have attained 10 or more points get an "A" in the course.  As a student of Assessment and learning, what do you think about this approach?  (Cole or Suzanne, anything to add?)

Archives