Back to Cut Scores

Posted by ktmahoney | Posted in comp/rhet | Posted on 13-05-2009

1

Last night I wrote that there seemed to be a shift in yesterday’s meeting away from cut scores and to a discussion of what our writing work group thought more important.  Well, today Ira Blake, Assistant Vice Chancellor of Academic and Student Affairs, has been waving the “cut score” banner again this morning.  She said that our work groups need to come up with cut scores where we should begin.  

I just asked Ira Blake for a very clear statement about what are task was.  I asked, “so, let me get this straight, the task of our work group is to come up with cut scores.”  Her response, “yes.”  Yes, she did say a lot about us being experts and our imput important and that it was important that we had discussions about what is college ready, but in the end, the task is a cut score.  It will be interesting to see what will happen if our writing work group does not/refuses to come up with a cut score.

PaSSHE College Ready and Placement Work Group–End of Day 1

Posted by ktmahoney | Posted in comp/rhet, literacy, writing | Posted on 12-05-2009

0

It’s been an interesting day here in Harrisburg as day one of PaSSHE’s College Ready and Placement Work Group comes to a close.  Interesting because there has been a bit of a shift over the course of the day when it seemed like the sole purpose of this work group was going to be to choose one of the major standardized placement exams–COMPASS, ACCUPLACER, or ASSET (notice all of them are in all-caps)–as the standard for the whole State System.  By the day’s end that imperative was less clear.

The day alternated between whole group sessions led by Vice Chancellor Jim Moran and break out sessions: math, writing, and reading.  The general sessions were not super formal.  Rather, they consisted of Moran making some preliminary comments and report-backs/questions from the break out groups.  Initally Moran say that by the end of our two-days here in Harrisburg, he wanted us to come up with “baseline cut-scores” for each of the areas.  In other words, we were supposed to come up with the scores that would determine when a student would be forced into “remedial” courses.  The primary target was to “identify a standard across the system for college comp and intro to math.  Moran said that this is part of a larger effort–a “two-pronged effort”–to standardize the State’s definition of “college ready” and to create a common set of criteria for college placement.  The other prong is to standardize the entire State’s high school curriculum.  Moran said that PA was one of only five states that does not have a standard curriculum across the state.  

Our discussions were supposed to be guided by “all the research,” or at least the research that we were provided less than a week before today’s meeting.  Most of this research was conducted by ACT and some organizations assosicated with college testing agencies.  In fact, one of the key documents, “Rigor at Risk,” published by the ACT explicitly saw its study as a follow up to the federal government’s 1983 publication, A Nation at Risk.  The report suggests that since the publication of A Nation at Risk

almost every state has made significant efforts t o improve its educational system.  Nearly a quarter-century later, in a climate in which U.S. workers are dealing with new forms of technology and facing challenges of a global economy, it is not only reasonable but increasingly urgent to ask: Have we succeeded in fulfilling the goals of A Nation at Risk? (ACT, “Rigor at Risk).  

Of course we might draw attention to the fact that the 1983 study may have contributed to our current problems by narrowly defining curriculum to suit specific educational agendas.  I think it would be useful to revisit John Trimbur’s “Literacy and the Discourse of Crisis” in the midst of this discussion.  

Anyway, at several points in the discussion, Moran suggested that he would prefer one standard placement process across the State System.  However, he stopped short of saying this was necessary.  But, at several points he suggested that any “locally designed assessment” (directed self-placment being one of them) would need to demonstrate how the results could be compared to one of the standardized tests or standard State placement processes.  He said that it would be up to the people responsible at that university to “assess whether or not it would be worth the effort” to demonstrate compatabilty.  This approach seems very much like the State System’s buy-in to the  ”Voluntary System of Accountability,” which I don’t have time to go into here. 

There seems to be some pressure coming from the State Legislature on this score.  Moran told us that “it would not surprise me if there is some move for legislative action on remediation.”  In fact, he is giving testimony before the Republican Caucus tomorrow morning on precisely this topic.  

The good thing about today’s process was the breakout sessions with other faculty across the State System.  One of the most productive aspects of the day was that we talked about our different approaches to placement, different student bodies, and different configurations of composition (some universities have two semesters of comp, others just one).  If anything, I think most members of this group are committed to keeping local control over the placement process.  While no one was opposed to discussing the kind of skills and practices that are necessary for a student to succeed in our first-year composition classes, I remain concerned that the outcome of our work is beyond our control.  For example, we could generate some very good principles and the State System could then turn around and look at the standardized tests that seem to reflect those principles.  I am not suggesting that is what will happen, only one of my concerns.

I’ll have to leave it at that for now.  I’m meeting some of my fellow compers for dinner.  Hopefully, I’ll be back later with an update.

INFO: PaSSHE College Ready and Placement Work Group, Harrisburg, May 12-13

Posted by ktmahoney | Posted in comp/rhet | Posted on 12-05-2009

0

Hey all.  So, guess where I am.  Harrisburg.  I just checked into a Days Inn at around 11:30pm on Monday.  Tomorrow will begin what is being billed as a “Work Group” to determine a common definition of “college ready” and to “standardize” placement  in “remedial” courses. Presumably the outcomes of this group will contribute to a standard for Math and Writing across the State System.  

Now, anyone who knows me and/or knows our program at KU might question what it is that I am doing here.  Well, as the Coordinator of Composition I was asked to represent KU for the “writing” work group.  To be honest, I don’t know exactly what to expect.  The initial request was fairly vague.  However, a couple of days ago, we got another  email laying out the agenda a little more.  That email came with several attachments (which I will post tomorrow).  

Since I believe in working in public where we can all see what’s going on and contribute to the conversation, I am posting the email I received in it’s entirety.  I wil also be posting updates to this blog to let you know what seems to be the general thrust of the meeting.  The one article we were  asked to read before our meeting tomorrow is a policy brief called “It’s Not About the Cut Score: Redefining Placement Assessment Policyto Improve Student Success” authored by Michael Lawrence Collins for a Boston policy institute called Jobs for the Future

So, in the meantime, here’s the email I received on May 6th:

Colleagues: 

Thank you for your willingness to come to Harrisburg to develop a more consistent model for determining the standards for “college ready” and placement across PASSHE.  We have attached a number of documents to help shape the discussion and would request that you review the paper titled, “It’s Not About the Cut Score” prior to the meeting.

Let me first provide you with some context for our discussions, then provide some background for the attachments and finally talk about the details regarding arrangements and compensation. 

Context:

The primary goal is to achieve consistency in the definition of “college-ready” across the System.  Right now the standards vary considerably from university to university. The “Achieve the Dream” initiative at community colleges has focused on this effort and Pennsylvania is participating in that effort. We believe that a common standard, not necessarily a common test, will, taken in context, be one of the factors that will serve to improve the quality of the preK-16 educational system across the Commonwealth.

One may ask: “Why now?”  We believe that we have the support of the current administration and PDE to move in this direction; community colleges are receptive to us taking the lead and are willing to join in on the effort at the appropriate time. 

Concurrent with this effort is an attempt to identify for the System a high school curriculum required for regular admission (e.g., four (4) years English; three (3) years Math including Algebra II; three (3) years Science, etc).  A set of common placement standards for reading and mathematics helps set the expectation for such a curriculum.

During the System-level deliberations related to the standards for concurrent enrollment programs, the CAOs identified a set of standards that students should meet to be identified as “college-ready.” Also, Middle States’ teams have connected placement standards with measures of student learning outcomes in assessing general education programs and inquired about the consistency across System universities. 

Moreover, as part of the transfer and articulation legislation, standards for transfer of foundation courses were established. However, in some cases courses listed in the System are identified as remedial courses, which should not transfer. Currently, legislative hearings on remediation are taking place and the concept of consistency has been raised. Several states have moved forward with creating standards, either at the state or System level (e.g., New Jersey Community Colleges just recently agreed on common standards).

The interest in the Keystone Examination (formerly known as graduation competency assessments – exit exams from high school) has raised the ante in all of the aforementioned discussions. Local assessments within this context are acceptable only if the local district demonstrates comparability to the basic standard.

As the Office of the Chancellor has prepared the data for these hearings, it is increasingly difficult to explain the variation among our universities in the number of students required to take remedial or developmental courses.

The advent of the PIMS project (Pennsylvania Information Management System) and the connection of the higher education databases to the preK-12 databases (made possible through the use of the PA Secure ID effort) enable significant opportunities to provide detailed feedback to high schools on the success of students and nature of the curriculum. Consistency in standards allows for greater power of analysis to provide better feedback for improvements and curriculum revisions.

We also recognize that setting a “cut score” will not be sufficient and revisions will be required over the next several years as we are able to research the success of students. The ability to track students through the high school, community college and PASSHE will enable us to have an excellent database to make subsequent revisions as necessary.

A few years ago there was considerable discussion about whether four-year institutions should be in the business of remediation. We believe there is clear recognition that in Pennsylvania, with limited access to community colleges and given our mission as Pennsylvania’s public universities, the PASSHE recognizes that we will be involved in some form of developmental education to provide students with higher education opportunities, and that we should have in place better assessments and support for students that have potential.

We believe that now is the time for PASSHE to take a leadership role in the Commonwealth to set standards to improve the education of all students. Placement testing is but one aspect to this larger effort but is important in helping set the context of expectation.

Note that this does NOT mean that the admission standards across the universities are to be common or that we intend for universities to become more or less selective. Those decisions are governed by the universities themselves and by other factors. We are also looking at how to revise the accountability and performance funding measures to better reflect the access and quality mission of PASSHE and may need to consider how any change in the placement process affects those key measures.

Resources:

An excellent paper to read is, “It’s Not About the Cut Score” put out by the Achieve the Dream initiative.

We have attached material that details the System level reports from universities about the number of students taking “remedial” courses. We also attached a preliminary paper that provides a summary analysis of some of this data.  In addition, included are some summary data of college and university standards we collected a few years ago (and may require updating).

ACT has published several major studies focusing on the readiness of students based on performance on the ACT. “Readiness” is defined as the likelihood of receiving a C or better in the first college course.  Note that although several states are using the ACT as a placement exam (e.g., Arkansas), most researchers agree that the SAT should not be similarly used and should be used only as a possible screening tool to determine who should take a placement test.  We have included material from ACT.

Note that the link to the foundation courses that are approved for transfer is www.patrac.org

Details:

We are asking that you make your own hotel and travel arrangements. Attached is a list of hotels in the area with links to their website so that you can view their facilities when making reservations. Reimbursement and compensation will be through your own university. Travel and meals must fit within the standard PASSHE travel guidelines. To assure that your “take home” compensation is approximately $500/day, we have set aside $1,700 gross compensation for each of you. 

NOTE:  You will receive two additional e-mails with the reference material.

 

guidebooks for cyborg forests–advancing composition

Posted by K. Mahoney | Posted in Uncategorized | Posted on 19-05-2008

0

After a nice five days in the mountains, I am returning to this electronic space to revise a draft of some of my reading. While it is true that the bulk of my time in the mountains was spent doing mountainy things, part of relaxing is having the time to read more of what I want to read. The last several weeks of any semester always take a toll on reading. Or should I say, reading anything other than student papers.

I read the bulk of Coming of Age: The Advanced Writing Curriculum. I say “the bulk of” because I have yet to read Sections III and IV which can only be found on the included CD-ROM. But, given that the concept of “the book” is still wedded to the printed page, I am willing to say I “read the book” while still quite aware that I have still have two sections of the book to read. In any case, the part of the book that I read has given me a whole lot to think about. As it turns out, I was in the mood for thinking about building, constructing, or, in that lovely neo-capitalist turn of phrase, “growing” our composition program.

snippets:

I thought it was telling that over the 50 years of discussions seeking to define “advanced composition,” she found that there was a consistent inability to come to a consensus concerning what “advanced composition” or the advanced composition curriculum is or should do. But what I found more interesting is that she notes that “The conventional academic faculty–not TAs and part-timers who teach freshman composition–with an academic orientation are the usual teachers of advanced composition” (Bloom 16); while at the same time, many of the courses and textbooks she surveyed were difficult to distinguish from the first year composition courses. Hogan’s survey of 311 American colleges and universities in 1980 and Shumaker, Dennis, and Green’s 1990 survey revealed at best little connection between “advanced” composition and the first year course (6, 12).

On the one hand, it makes sense why there would be little consensus among composition faculty at different institutions about what advanced composition is given the lack of an established “discipline” until fairly recently. However, what is more puzzling is the lack of connections between the first year course and the advanced course at each institution. That is, it seems puzzling that faculty would not thinking about the relationship between composition and advanced composition when they were creating the course.

That brings me back to the question: who is teaching composition? Bloom’s recognition that “conventional faculty” teach advanced composition, is also a recognition that “non-conventional faculty”–i.e. TAs and contingent faculty–are the one’s teaching composition at most colleges and universities. While certainly not a new observation, this fact only returns us to the unavoidable need of making labor issues an integral part of the construction of any undergraduate composition concentration/major. That is, I think one of the key reasons that there is a persistent disconnect between advanced composition and the first year course is because the same people are not teaching both courses.

And, frankly, the necessity to integrate labor issues into curricular development becomes even more so today. As Marc Bousquet so nicely lays out in How the University Works,

Thirty-five years ago, nearly 75 percent of all college teachers were tenurable; only a quarter worked on an adjunct, part-time, or non-tenurable basis. Today, those proportions are reversed. If you’re enrolled in four college classes right now, you have a pretty good chance that one of the four will be taught by someone who has earned a doctorate and whose teaching, scholarship, and service to the profession has undergone the intensive peer scrutiny associated with the tenure system. (Bousquet 2)

In this world, one can see those calls to “abolish” first year composition courses as an attempt to (as I have argued in another context) out-source service courses.

Reading Miller’s essay felt, at times, like talking to a kindred spirit. I can’t count the number of times I’ve argued along similar lines. At the heart of his essay is a call for civic literacy: “This civic domain is the field of study that I hope rhetoric will reclaim as it expands its frame of reference beyond first-year composition courses” (39). Given that I want to leave campus and go home soon, I’m going to just post this longish passage from his essay that seems integral to developing our CRLS concentration:

A critical awareness of the process of constructing shared beliefs is essential to a civic philosophy of rhetoric that makes sense of what we value. The contradictions contained within this process mark the sites of controversy that can evoke a dialectical awareness of the negotiation of morals and mores. As students examine what is up for debate, how it was called into question, and why it is useful to view the debate from multiple standpoints, they can learn to value critical reflection as a means to practical action, rather than an end in itself. Students can develop this rhetorical stance by reflecting on their expectations about a text and its expectations about them, the experiences that validate and challenge those expectations, and the codifications of those experiences in discursive, moral, and social conventions. In other words, students can learn to question what is assumed, where those assumptions come from, and what gives them authority. If these are to be rhetorical questions, their answers must include action. Critical judgment is generally understood to be the end of inquiry within English departments, as elsewhere in the academy–which is, after all, a product of the Enlightenment–but our own tradition treats critical thinking as a prelude to practical action. (Miller 40, italics mine)

I’m checking out for the day.

Mostly cloudy with breaks of sun, 56 degrees.

NCTE statement on 21st Century literacies

Posted by ktmahoney | Posted in comp/rhet, literacy, professional orgs, teaching, writing | Posted on 08-05-2008

0

Toward A Definition of 21st-Century Literacies
Adopted by the NCTE Executive Committee
February 15, 2008

Literacy has always been a collection of cultural and communicative practices shared among members of particular groups. As society and technology change, so does literacy. Because technology has increased the intensity and complexity of literate environments, the twenty-first century demands that a literate person possess a wide range of abilities and competencies, many literacies. These literacies—from reading online newspapers to participating in virtual classrooms—are multiple, dynamic, and malleable. As in the past, they are inextricably linked with particular histories, life possibilities and social trajectories of individuals and groups. Twenty-first century readers and writers need to

• Develop proficiency with the tools of technology
• Build relationships with others to pose and solve problems collaboratively and
cross-culturally
• Design and share information for global communities to meet a variety of
purposes
• Manage, analyze and synthesize multiple streams of simultaneous
information
• Create, critique, analyze, and evaluate multi-media texts
• Attend to the ethical responsibilities required by these complex environments

NCTE on 21st Century Literacies

Posted by K. Mahoney | Posted in Uncategorized | Posted on 08-05-2008

0

…and then there’s this statement by the National Council of Teachers of English:

Toward A Definition of 21st-Century Literacies
Adopted by the NCTE Executive Committee
February 15, 2008

Literacy has always been a collection of cultural and communicative practices shared among members of particular groups. As society and technology change, so does literacy. Because technology has increased the intensity and complexity of literate environments, the twenty-first century demands that a literate person possess a wide range of abilities and competencies, many literacies. These literacies—from reading online newspapers to participating in virtual classrooms—are multiple, dynamic, and malleable. As in the past, they are inextricably linked with particular histories, life possibilities and social trajectories of individuals and groups. Twenty-first century readers and writers need to

• Develop proficiency with the tools of technology
• Build relationships with others to pose and solve problems collaboratively and
cross-culturally
• Design and share information for global communities to meet a variety of
purposes
• Manage, analyze and synthesize multiple streams of simultaneous
information
• Create, critique, analyze, and evaluate multi-media texts
• Attend to the ethical responsibilities required by these complex environments

While falling a little short of the kind of project suggested by the multiliteracies folks, NCTE’s statement does support the kind of hybrid/cyborg approach to CRLS. Interesting.

Cloudy, spitting rain, 69 degrees.

cyborging the robot army

Posted by K. Mahoney | Posted in Uncategorized | Posted on 07-05-2008

2

robotsThe end of the semester is for the most part here. Students are finishing project, papers, and exams. And faculty are knee deep in grading. And as I sit at my desk reading and responding to student papers, I am having my android dreams of the robot army to come.

In particular, I am thinking about hybrids…cyborgs to be more precise. Cyborgs that cross disciplinary boundaries and who neither respect, nor desire, disciplinary fortifications. That does not mean that cyborgs are wishy-washy about their agenda or intellectual commitments. No, cyborgs just start from a different place.

In fact, the whole “literacies” family of CURLS robots is a little underdeveloped in the whole scheme of things. The more I think about it, the more I think that cyborgs are more fitting than robots when it comes to literacies. Take digital rhetorics/literacies, for example. On the one hand, we could develop a robot that would approach its task from the rhetoric angle–and do it quite well. However, digital writing/design bleeds into several other areas–even in the immediate family: desktop pub, info design, and media studies, for example. So, when thinking about designing a “digital rhetoric” robot, it would make more sense to turn to a cyborg.

Haraway book jacket imageSuch a proposal will be tricky, though. After all, cyborgs do not exactly have a glowing reputation–think Blade Runner, Terminator, and, of course, the BorgDonna Haraway notwithstanding. But an interesting way of approaching the task at hand, methinks. Diversify the robot army.

Yes, it’s the end of the semester.

Yes, I’m punchy.

Sunny skies, low humidity, 78 degrees.

building

Posted by K. Mahoney | Posted in Uncategorized | Posted on 11-03-2008

0

I can’t put my finger on it, but it seems like it’s time to build.  Maybe it’s because I’m just worn out.  But I don’t think that’s it. All I know is that I continually find myself looking at other Comp/Rhet program and imagining the possibilities at Kutztown.  It is just coincidence that I stumble across an issue of Composition Studies from last spring dedicated to the emerging trend toward undergraduate majors in writing, writing studies, cultural rhetoric, and the like?  Is it happenstance that I’m reading Rebecca Moore Howard’s article in said journal and hear echoes of my own arguments? Signs.  Swing away, Merill.

As I’m reading about different programs I’m sketching visual representations of curricula.  Overlapping egg shapes circled by satellites of course clusters.  I need to see things visually if I have any hope of holding onto my thoughts.  A few months ago the possibility of building courses and curricula seemed like wishful thinking.  Or, at least, hardly worth the effort.  Something has shifted.

Best I can tell, it’s linked to the work I’ve done with the union these past several years.  I’m thinking that because of a comment I made to a colleague last week.  I said to her I wanted to bring what I learned over the past couple of years in the union back home.  I don’t know if it’s stranger that I called our little comp program “home” or that I made the link between my union work and my comp work.   Maybe it’s because our vote of no confidence discussions–which have occupied most of this semester–have begun to bear fruit.  Maybe in this little spring break pause has been enough for me to reflect a bit.

I mean I look back at how far we’ve come as a union over the past 4 years and I am amazed.   Most of our new leadership were untenured and/or temporary faculty when we were elected to our Executive Committee.  While some people warned us to “keep our heads down” until we got tenure, or reminded us that “things are not done that way at Kutztown,” we worked.  We worked for the kind of union we thought we should have.  We didn’t settle for “good enough” or “no other choice.”  I look back now and am pleasantly surprised to see the synergy between my writing and my union work.  I guess it’s time for that kind of work to come “home.”

I also said to that same colleague that my preferred approach to leading our program is to create spaces for discussion and conversation.  But now, I said, I think it’s time to begin to assert our program a little more.  Time to work for what we would like to see, not only what is “good enough” or safe.  I’m looking forward to that.