Sara, I think those are great statistical analyses to do, but I would argue
that the statistical model still omits the important factor of
motivation/self-regulation/grit/what-have-you.
I thought propensity scores, the new darling of education research, would
help address this factor, but from my reading/discussions with
statisticians, we still have the same problem: there is an aspect we
believe accounts for some significant portion of the dependent variable
(here, motivation accounts for some portion of a student's final course
grade/retention rate/graduation rate/etc...) and we don't have any way to
include that aspect in the model. The best we can get to is using the
students previous gpa (at the institution because high school GPAs weren't
as accurate). We can make the claim that a student's previous gpa is an
indicator of motivation, but it's also an indicator of many other things
such as background knowledge, coursework rigor and attendance. Then we
need to worry about how effective an indicator GPA is, plus if it interacts
with other factors in the model.
We have tried including a segment of the MSLQ in a first-year campus wide
survey to get at the motivation question. We would then use that score as
predictor variable, but the response rate was not high enough to get any
statistically significant results.
Sigh. I wish there was an easy answer to all this.
-Marcia
On Thu, Jan 9, 2014 at 10:14 AM, Sara Weertz <[log in to unmask]> wrote:
> Ah, I love this question...one I think I can answer because this used to
> be a typical response to Supplemental Instruction (SI) which has a history
> of empirical evidence indicating that students who use SI on a regular
> basis get better grades. Faculty often opine that SI students would have
> gotten successful grades no matter what; they argue that SI students (or
> those who self-select) are already the "good" students. Faculty continued
> their criticism of the numbers even after I added qualitative
> data--feedback from the students themselves, in their own words, saying
> they excelled in their coursework because of SI.
>
> It was, however, more difficult to be critical of my interpretive report,
> which pulls the following data on students enrolled in SI-supported classes:
>
> * GPA (at the beginning of the term)
> * ACT/SAT scores
> * Classification
> * Ethnicity
> * Residency (on/off campus)
> * Major/Minor
> * Academic Standing
> * Cohort attributes such as athletics, provisional status, international
> student, etc.
>
> If I run the interpretive reports at the beginning of the term, I get a
> bird's-eye view of the class, which allows me to also create individual
> student profiles.
>
> The beauty of the interpretive report is its use as a tool to make
> predictions about the students in our SI-supported classes. An example
> would be to examine how a freshman with several at-risk factors and low ACT
> scores (which tests science acumen) might fare in a traditionally difficult
> biology class. Since our SI support focuses on traditionally difficult
> classes where many students struggle, we then make predictions on success
> (A, B, or C) depending on whether the less proficient students and those
> considered at-risk attend SI, how often they attend, and when they attend.
> The interpretive report allows us to compile some fascinating reports for
> variety of departments and student services. Our measurements consistently
> show that no matter how many at-risk factors a student may have, the more
> SI visits, the higher the final grade.
>
> While something like an interpretive report is more difficult to generate
> with tutoring, it can be done.
>
> sal
>
>
> Sara Weertz, M.Ed.
> Executive Director, First Year Experience
> ASU Station #10915
> Angelo State University
> San Angelo, TX 76909
> (325) 942-2595
> [log in to unmask]
>
> CRLA President-Elect 2013-2014
> www.crla.net
>
> ****************************************************
>
>
> -----Original Message-----
> From: Open Forum for Learning Assistance Professionals [mailto:
> [log in to unmask]] On Behalf Of Marcia Toms
> Sent: Thursday, January 09, 2014 8:32 AM
> To: [log in to unmask]
> Subject: Re: ROI on Academic Support Services? -- Different Take
>
> That is great, Leonard.
>
> One question, though: Do students voluntarily come to your center? If so,
> how do you address the motivation issue? In other words, who is to say
> that these students wouldn't have higher retention rates anyway?
>
> Best,
> -Marcia
>
>
> On Thu, Jan 9, 2014 at 9:06 AM, Roberta Schotka <[log in to unmask]
> >wrote:
>
> > Leonard,
> >
> > That is brilliant, especially since it is so difficult to link grades
> > directly to tutoring, given all of the other contributing factors.
> >
> > -Roberta
> >
> >
> > On Wed, Jan 8, 2014 at 4:52 PM, Geddes, Leonard G.
> > <[log in to unmask]
> > >wrote:
> >
> > > Melissa and any others who are interested,
> > >
> > > I have attached part of a report that I sent up to the "powers that
> be"
> > > about the influence our services are having on the bottom line --
> > > retention. In the past, we communicated how we were affecting
> > > academic performance. However, when it seemed like reporting how
> > > students were improving academically was not generating the traction
> > > that we thought it deserved, I decided to speak the administration's
> > > language by adding a retention element to the report. In short, we
> > > compared the re-enrollment rates of students using our services to
> > > general student retention,
> > athletic
> > > teams, etc. Our numbers rocked! (I've attached an abbreviated
> > > report since I don't think the administration would like us to share
> > > financial info publically.)
> > >
> > > In the actual report, we put figures to the report by factoring in
> > > the "real" revenue that is generated per student. For example,
> > hypothetically,
> > > if the overall retention rate was 70%, but our numbers were 86%,
> > > then we showed numerically how much revenue 16% more students added
> > > to the bottom line, thus showing that we are revenue generating.
> > >
> > > As a result of changing to reporting this way, our reports have been
> > going
> > > all the way up the chain to the Board. Recently, they specifically
> > > referenced our center and services in the new strategic plan! We
> > > are now preparing for a significant budget increase as well -- yay!
> > >
> > > I hope this is useful.
> > >
> > > Leonard Geddes
> > > Associate Dean of Co-Curricular Programs Director of the Learning
> > > Commons Division of Student Life Lenoir-Rhyne University www.lr.edu
> > > [log in to unmask]
> > > (828) 328-7024
> > > (828) 328-7702 (fax)
> > >
> > > The LearnWell Projects Blog:
> > http://www.thelearnwellprojects.com/thewell/
> > >
> > >
> >
> > ~~~~~~~~~~~~~~~
> > To access the LRNASST-L archives or User Guide, or to change your
> > subscription options (including subscribe/unsubscribe), point your web
> > browser to http://www.lists.ufl.edu/archives/lrnasst-l.html
> >
> > To contact the LRNASST-L owner, email [log in to unmask]
> >
>
>
>
> --
> Marcia Toms, Ph.D.
> Associate Director
> Undergraduate Tutorial Center
> Division of Academic and Student Affairs North Carolina State University
> Campus Box 7118 / 101 Park Shops Raleigh, NC 27695-7118
> 919.513.7829
> http://www.ncsu.edu/tutorial_center/
>
> Public Record Reminder: All electronic mail messages in connection with
> State business that are sent to or received by this account are subject to
> the NC Public Records Law. They are retained and may be disclosed to third
> parties.
>
> Confidentiality: Nothing in the NC Public Records Law diminishes the
> privacy protections afforded by federal law (e.g., FERPA, HIPAA, etc.)
>
> ~~~~~~~~~~~~~~~
> To access the LRNASST-L archives or User Guide, or to change your
> subscription options (including subscribe/unsubscribe), point your web
> browser to http://www.lists.ufl.edu/archives/lrnasst-l.html
>
> To contact the LRNASST-L owner, email [log in to unmask]
>
> ~~~~~~~~~~~~~~~
> To access the LRNASST-L archives or User Guide, or to change your
> subscription options (including subscribe/unsubscribe), point your web
> browser to
> http://www.lists.ufl.edu/archives/lrnasst-l.html
>
> To contact the LRNASST-L owner, email [log in to unmask]
>
--
Marcia Toms, Ph.D.
Associate Director
Undergraduate Tutorial Center
Division of Academic and Student Affairs
North Carolina State University
Campus Box 7118 / 101 Park Shops
Raleigh, NC 27695-7118
919.513.7829
http://www.ncsu.edu/tutorial_center/
Public Record Reminder: All electronic mail messages in connection with
State business that are sent to or received by this account are subject to
the NC Public Records Law. They are retained and may be disclosed to third
parties.
Confidentiality: Nothing in the NC Public Records Law diminishes the
privacy protections afforded by federal law (e.g., FERPA, HIPAA, etc.)
~~~~~~~~~~~~~~~
To access the LRNASST-L archives or User Guide, or to change your
subscription options (including subscribe/unsubscribe), point your web browser to
http://www.lists.ufl.edu/archives/lrnasst-l.html
To contact the LRNASST-L owner, email [log in to unmask]
|
