Hi, Matt, et al.,

Thanks for sending that paper and the Carpentry's response.  I first saw 
that paper a couple of weeks ago and have not yet had a chance to really 
give it a careful reading, but I will do so.  Anyway, I think questions 
about the efficacy of various instructional models are extremely 
important, and the more we can understand about when short workshops 
work well and when they don't, the better.  So, if there is an effort to 
more formally evaluate outcomes here at UF, I'd be happy to try to help out.

In addition to the "experimental treatments" that Matt mentioned in his 
original email, note also that next semester we will have the 
alternative workshop format that Justin (I think -- my apologies if I am 
not recalling the correct name) is helping organize, with a regular, 
two-day SC workshop parceled out over multiple weeks.  So that could be 
yet another instructional format to evaluate.


On 12/19/2017 09:36 AM, Smith, Plato wrote:
> Yes, Matthew I am interested. I am also working with Adrian Gritz, UF 
> College of Business Web/IT Developer, on GitHub@ UF assessment. Both 
> are interests to the DMCWG. The DMCWG just secured a partnership with 
> UF Innovation Square this morning to host an evening reception and 
> tour of the Innovation Hub on March 19, 2018. Thus, future metrics, 
> outcomes, and impact on students'success and job placement are of 
> interests and worthy of funding support.
> Thanks.
> Plato
> Sent from my Verizon Wireless 4G LTE smartphone
> -------- Original message --------
> From: Matthew Collins <[log in to unmask]>
> Date:12/19/2017 9:14 AM (GMT-05:00)
> To: [log in to unmask]
> Cc:
> Subject: Studying the effects of workshops & short courses
> The paper "" is currently making the rounds:
> The Carpentries have a considered response to it:
> Several of the details about the study data struck me. The sample size 
> of workshop participants was 48 among 294 PhD students across 53 
> institutions. One of the Carpentries concerns was that it is hard to 
> know what workshops these students attended and how the material was 
> reinforced in their regular courses. We at UF have an n of similar 
> size to this study at one institution and we know positively (we can 
> get access to course history and transcripts w/ appropriate IRB) what 
> has happened to our learners. We further have Ethan's semester course 
> which is a direct extension of the Carpentries workshop so we could 
> consider 4 populations: no intervention, workshop, workshop + course, 
> course. There are other courses too that we might consider like ones 
> from stats that teach a little R and Gitzendaner's BIO course as well 
> as other activities like DSI or ACM participation.
> I've started trying to reach out to Academic Advising to see if they 
> can get us some help designing a study of this stuff. Does anyone have 
> a strong motivation to pursue this next semester or know someone who 
> would?
> Matthew Collins
> Technical Operations Manager
> Advanced Computing and Information Systems Lab, ECE
> University of Florida
> 352-392-5414