The link to Dr. Wade's article about the SmarThinking study at West Kentucky Community and Technical College is now available. This is useful and practical information. (Thanks Dr. Wade) http://www.kctcs.edu/newspublications/exchange/index.cfm?eei_id=11#77
A Pilot Project with KCTCS, WKCTC, and Smarthinking Editor's Note: William Wade, Director of Distance Learning and Professor of English, West Kentucky Community and Technical College, outlines a pilot project to improve the way we teach English composition.
The question was "Can we increase the number of students taught in a composition class without taking unfair advantage of faculty and without loss of quality in instruction?"
The answer is not nearly as easy as the question. Teachers must teach grammar, structure, clarity, order, and encourage specific and concrete support. So, the only way to increase numbers in composition classes is by having better essays after that instruction. If students "got it" after instruction, the teacher would have higher quality essays, and, therefore, could teach more students in less time. Here is the typical scenario. The teacher begins with a writing sample which determines student readiness for the class. Then, remediation is applied where needed. Next, the teacher defines an assignment by explaining that assignment's characteristics and requirements. The student begins the task of writing the prescribed paper. Writing assignment one is submitted in draft form, comments are made on that draft, and it is returned to the student. It is revised. Draft two is turned in to the teacher who either grades the paper or returns it for further revision. If a third draft
is given, usually a fourth choice is not allowed, the paper is graded and the student moves to the next assignment. Reducing the number of readings by the teacher would allow that teacher to increase student numbers and the teaching of important concepts and evaluation progress for the student and aid that student in reaching the desired outcomes.
What if the teacher could assess a writing sample, remediate obvious difficulties, make an assignment, and grade a final draft? Wouldn't that decrease the time spent and allow more students to take the class? The answer is yes, but how can that happen?
When the student reads the assignment, accepts the recommendations of the tutor and teacher, revises the assignment, and turns in better work, the instructor can grade the paper and move on.
The K-CoRE Plan
In June of 2005, West Kentucky Community and Technical College (WKCTC) and the Kentucky Community and Technical College System (KCTCS) combined with Smarthinking Tutorial Services of Washington DC to put together a basic English composition pilot course for the fall semester which would begin in mid August 2005. The trial course was designed to offer composition to a greater number of students. Based on a Kentucky initiative referred to as K-CORE, The Kentucky Collaborative Online General Education CoRE, a year long discussion that began as a statewide collaborative effort to define ways in which online course presentation could be more efficient. With a kickoff date of October 2004, the team began looking at ways to improve the number of students served with the existing faculty. For the next year, three postsecondary institutions, the University of Kentucky, Murray State University, and the Kentucky Community and Technical College System, began an in-depth study of what
it would take to define and then create an efficient online class in beginning composition. In addition to defining course competencies, the majority of the institutions represented found and agreed upon a grading rubric to use in the pilot class.
Beginning in June, 2005, KCTCS and WKCTC began working out the details of a pilot course. Seven people were involved in the West Kentucky effort: Michelle Martini, master teacher and course content director (teacher); William Wade, Professor English, Director of Distance Learning, and course designer; Rebecca Wilson, help desk and technical assistant for course design; Sandra Tucker, online academic advisor; Pat Blaine, English Professor and course evaluator; Rhonda Thacker, student course evaluator; and Shelby Townsend, student course evaluator. All seven people worked in their various capacities during the fall semester 2005. Excerpts from their input will be included below.
The Course Design and Smarthinking
The course was designed using small learning units. Each learning unit had a specific purpose and led the learner to either a new step or a number of smaller reviews. The first step in the class was an initial orientation quiz that gave the student an overview to mastery learning where the learner is an active participant. Once the quiz was complete, the student moved on to a course pretest. The pretest assessed the student's current knowledge of grammar, mechanics, and development. If the student scored less than an 80%, that student was routed to interactive reviews identified by the test. If a student scored 80% or higher, that student went directly into a writing assignment. Successful completion of the first writing assignment opened the door to the second writing assignment. This process continued until four graded assignments were completed. The student had access to written information on each assignment, length requirements, strategy for writing, and basic
instruction on outcome expectation. The class teacher was available to provide added instruction, clarify technology needs, and give guidance when written instructions were not clear. A technical help desk at the local level provided the student in-course information when technology failed, and a national help line was available when software or hardware ceased to function. Two types of tutor assistance were also available. Smarthinking, a Washington DC based online tutoring group, gave advice to the student as that student wrote through the first draft of each graded paper. Once the student completed the assignment, it was turned into a second set of tutors provided by Smarthinking. These second-line teaching assistants' output was called Grade Guidance. They not only marked the paper for the student to see strengths and weaknesses, they also sent the same material to the class teacher with a suggested grade. The idea was that with the input from the Grade Guidance
tutors, the student would write better drafts allowing the teacher to grade fewer major errors and, therefore, work with a larger number of students.
The college opened registration for this trial class on August 3, 2005. Within about twelve days, we had reached an enrollment of 49 students, and the college closed registration. The regular size of a single online basic composition class is 24. As with all online classes, the first several days were spent explaining the structure of the new class and how its content was arranged. The student was given a walk through the initial quiz and the pretest. Once writing began, they were also given a quick look at the online tutoring and an introduction to Grade Guidance. The assignments had submission rates illustrated below in a section written by the class teacher.
At the beginning of the class, the students had typical questions about course make-up and where certain bits of information were located. An outline of the semester's activities was posted on the Home Page of the Web CT offering. Individual links and assignments were posted that led students to new assignments and gave feedback about completed assignments. Each assignment had an overview, detailed information about what was expected, and a student-written paper illustrated each assignment strategy. A Public Discussion link was placed on the Home Page along with a link to Private internal E-mail. In these communications areas, the student could post threaded discussion notes for all to read and react to or post private questions on evaluation and direction to the instructor. By the third week, most students were navigating the class and finding information on assignments.
With this initial pilot, the attrition rate for the class was high. Of the 49 initial students, 10 successfully completed the class. A number of factors may have come into play with the numbers. August 3 was the first registration date for the class, and that was just ten days before the beginning of class. The entire enrollment came from late registration, and students were not given an orientation to the course make-up and process. By September 27, 18 students had dropped or never attended the online class. While 30 students did not drop, only 10 made a C or higher grade. This percentage is neither acceptable or typical of an online class. The average successful completion rate for a WKCTC ENG 101composition class is currently 61%. The nature of the class and the late registration numbers probably both played a major role in the percentage of successful completion.
An online orientation to the class is under construction. Because of a change in Course Management Systems (CMS), the orientation was not designed for the spring semester but will be in place for the Fall of 2006. Other changes that were evident were that students will be educated about the use of Smarthinking, they will be given an overview of the new CMS which is Blackboard, and they will be given insights into the workings of a modular, interactive, mastery level composition course. Smarthinking was an important tool for this class. Further study is required before the advantages of Grade Guidance can be verified for the faculty. It is very clear that the advantages are strong for the student.
Observations by the Instructor of record
As the instructor of record and class teacher for the English section, I had high expectations of the KCoRE project and its outcomes. First, the pressure of designing the course was lifted, and the initial technical problems students encounter including access to the course and navigation were handled by another source. I did not expect Smarthinking's involvement to abdicate my responsibilities as instructor. Rather, the idea was that students' repeated,
systematic revisions and submissions to Smarthinking tutors would result in a higher quality essay. Therefore, my job in grading essays would be easier because the focus would be on content and fine tuning rather than concentration on grammatical and organizational issues.
Although I did not design the course, which I thought initially would make things easier, I found that becoming familiar with the course content was time consuming. Like the students, I had to navigate through unfamiliar material and learn about the modal set up of the site. However, it did not take long to adequately be able to help with student questions. The first weeks of any online course consist mainly of answering "Where is the link?" type questions, and just like any other course, much of my time was spent re-opening assignments that had closed and directing students to the appropriate links. Obviously Smarthinking did not assist in this process, but the help from the course designer was valuable in dividing the task of answering students' navigation questions.
Once students had access to Smarthinking's tutors and understood the process by which they were to submit drafts, they began submitting drafts to Smarthinking. Twelve students submitted drafts of the first assigned essay to Smarthinking tutors. No positive correlation appeared between Smarthinking submissions and higher grades on the first essay. Upon viewing a cross section of students' drafts submitted to Smarthinking as compared to their final draft, I came to several conclusions after the first graded essay: 1) the more responsible students, and therefore those likely to receive higher grades regardless, were more likely to submit to Smarthinking and heed some of the tutors' comments, 2) the students who did poorly on the final submission but still submitted to Smarthinking did not heed Smarthinking's tutors' feedback, and 3) the majority of students were not taking advantage of the service. The tutors' comments, therefore, often were the same suggestions I made. If the
students had utilized Smarthinking's service, I would have been able to concentrate on other, less serious errors and instead concentrate on areas that needed more analysis or content development. However, the majority of the issues I addressed in the essays were related to organization and thesis development - the most basic essay components. As a whole, the essays were below average in content and grammar. It's important to point out that lecture material provided within the course and assigned reading in the textbook addressed essay construction and grammar. Before submitting the first essay, students should have participated in a grammar review and read information about organizing an essay.
My process for grading the first round of final essay submissions was as follows: reading and commenting on students' final submission; assigning a grade; reviewing the Smarthinking report to see if students had submitted drafts; if students had submitted a draft, reviewing the draft(s) and tutors' comments to see if the student had made some of the suggested changes; and finally, suggesting (if necessary) that the students heed tutors' suggestions, which often mirrored my own. I recognize that because this course was a pilot, reviewing Smarthinking's input was crucial. However, the process was much more involved and time consuming than I anticipated and much more difficult than grading essays in my other online courses. After grading the first assigned essays, my assessment was that Smarthinking was valuable when students used the tutors' comments in the draft
stages. I also felt that the "Grade Guidance" portion of the Smarthinking report was not useful to me as the instructor. I assigned the grade I felt was appropriate, and while the students could benefit from having grades assigned by Smarthinking, I didn't need confirmation from an outside source to validate the grade I assigned.
As the process evolved and students felt comfortable submitting drafts to Smarthinking, we received positive feedback from students regarding its help in the drafting stage. However, the same students kept submitting drafts, and the number of students submitting drafts to Smarthinking did not increase. Eleven students submitted drafts of the second assigned essay, and ten students submitted drafts of the third assigned essay. I began to see significant improvement in those students who submitted drafts and heeded tutors' suggestions. For example, Student A received an 89 on the first essay, a 92 on the second, and a 97 on the third. Because I did not review draft submissions to Smarthinking until after the final grade was assigned, the results of the students grades were not affected by the fact that they submitted drafts. However, upon review, the quality of the essays seemed to improve with increased involvement in Smarthinking's service.
While the data I have collected is by no means scientific, students who routinely submitted drafts of each essay to Smarthinking received higher than average grades. This result could be because higher-than-average students were more likely to participate anyway, but in at least one or two cases average writers received higher-than-average grades because of extensive drafting and feedback from qualified tutors. The student to whom I referred earlier received an A for the course. Another student whom I'd like to use as an example, Student B, submitted drafts of all assigned essays to Smarthinking. I would consider Student B slightly above average for freshman composition. Her first final submission received a high C; however, her essays progressively improved, and Student B earned an A in the course. Upon reviewing her draft submissions as compared to her final submissions, I found evidence of extensive revision. She began the semester revising mechanical issues and
organization of the essay, and by the end of the semester, she was able to concentrate on content and style. Student B was an exception; the average grade for the course was well below average. It is difficult to quantify the results because of so many extenuating factors; students who did not submit final drafts of all essays failed the course (part of the policy), and several students withdrew. My assessment is based mainly on those who participated in the course throughout the semester.
My work continued to be more involved than in other online courses. Smarthinking continued to adapt its reports to our needs, and each round of essays required evaluation of the procedure and reports given to us. Constant monitoring of the site and students' progress was necessary, and the large number of essays to read and evaluate was daunting at times.
If the students utilize the Smarthinking tool, it will continue to be valuable in the revision process. In order to be valuable to the instructor of record, the reports should be more user-friendly. The Grade Guidance report was lengthy and difficult to print and view.
Rather than having to view each document submitted to Smarthinking, it would be helpful if the instructor received a report containing the main suggestions for revision for each student's essay, perhaps in a bullet-pointed list. Also, the students should understand Smarthinking's participation and the procedure for submitting drafts as soon as the class begins. Perhaps a tutorial with sample submissions would help explain the process to students. Finally, students' incentive for participating in Smarthinking's assessment process should be evaluated in order to find a way to encourage more students to submit drafts throughout the semester. I look forward to seeing the progress of the project.
Conclusions and Improvements
"Can we increase the number of students taught in a composition class without taking unfair advantage of faculty and without loss of quality in instruction?" Probably. But, a number of questions must be answered and those questions have many options. A number of composition teachers are currently seeking answers, and those answers are needed as students continue to stand in line for classes in composition, math, and many other subject areas. Composition is an excellent place to search for answers as grading and instruction are subjective and individual in composition. Currently, objective tests and computer grading are not sufficiently accurate to replace the teacher, and that technology does not seem to be on the horizon. WKCTC, KCTCS, and Smarthinking hope that this trial course has helped in that discovery process.
William Wade, Director of Distance Learning
Professor, English West Kentucky Community and Technical College
K-CORE ENG 101 course designer
[log in to unmask]
Michelle Martini, Instructor English, West Kentucky Community and Technical College
[log in to unmask]
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
To access the LRNASST-L archives or User Guide, or to change your
subscription options (including subscribe/unsubscribe), point your web browser to
To contact the LRNASST-L owner, email [log in to unmask]