One of the things we don’t do so well in academia is to talk about the stories behind the the structured, impersonal research articles we write. So think of this post as the commentary on the DVD for a movie you watched, or in this case the academic article you just read, or should read, or will eventually read at some point in the very distant future.
The first draft of this blog was really long so I’ve broken it into three parts. The first part talks about how I naively plunged into designing this study with little mentorship or guidance. Part II talks about how long it took me to do much of anything with the data and how I got my undergraduate students involved. Part III is the perseverance part because getting this study published was an adventure.
The Naivety: Planning and Conducing the Study
It was about October 2011. It was the second year of our “new” Baccalaureate Program and I was in the second year of teaching my newly minted “for credit” Scholarly Writing course offered to our first-year students. The idea had been festering in the back of my mind for a year to study my Scholarly Writing course and its effectiveness. My credentials at that point in time: I had been teaching research to undergraduates for about 4 years at this point; I had done a research based thesis for my Master’s; I sat about 5 feet away from one of our psych instructors who was also teaching stats for the program (my second author Dr. Tom Harrigan, who would never let me call him Doctor anything in real life); and, perhaps the most critical factor, there was no one around who would tell me not to do it.
So what the hell, lets do a research study.
I had read very little writing research literature. Actually it would be more honest to say I had read nothing about writing research. I knew nothing about Self-Efficacy Theory other than that it existed. I had one meeting with our Research and Planning Director who was also the chair of the research ethics board at the College. I knew him from cycling before I knew him as the research guy. He gave me a few study design tips and tips on how to ensure ethical approval and off I was writing an ethics proposal.
I think maybe it worked in my favour to walk into planning and implementing a study on my own. I didn’t have anyone around me asking questions I’m glad I didn’t think of myself like, “shouldn’t you have a PhD to conduct a research study?” Knowing myself as I do, it is highly likely someone did question my credentials and experience in some way and I just didn’t hear it or process it. Being the girl who went out and rode a 300km bike ride last spring on almost no training, it is safe to say that when I get an idea in my head I’m pretty hard to stop no matter how foolhardy that idea might be. I suppose, I had Tom, but Tom’s main involvement didn’t come until I had the data and the stats needed to be run.
The course started at the end of November 2011 and I am pretty sure that I didn’t get ethical approval until just before Christmas. My manuscript says I collected data for the pre-test 4-classes into the term which would be about three weeks into the term because the class was held for 2 hours once a week.
I used Google to find a questionnaire. That’s how I found the General Self-Efficacy Questionnaire. I didn’t know Writing Self-Efficacy (or that other people had made questionnaires for it) existed. I developed the writing self-efficacy questionnaire following the pattern of that General Self-Efficacy Questionnaire. I consider myself lucky (now three studies deep and some reliability and validity data in my pocket) that the questionnaire works. It is probably best that I don’t go into too many details about how very naive I was to go out and just MAKE my own questionnaire. It didn’t occur to me not to. I had developed a questionnaire for my Master’s thesis too. I have a publication describing it. (Yes, I’ve changed my name. We’ll get to that). So of course I was going to develop a questionnaire. I was an experienced writer, writing teacher, and I knew the things that students sat down and talked to me about while clearly overwrought and distressed in the midst of the writing process.
So I plunged forward and ran the study and made oodles and oodles of mistakes. For example:
- I wasn’t really pretesting these students I was early testing them. They were 3 weeks into the course before they got their pretest.
- I had them develop a personal identification code so I could keep the students anonymous. That code asked them to use their Mother’s initials and date of birth. Well… some have more than one mother, and their mother has had more than 2 names in her life, and sometimes the birth date was used and sometimes the birth month. And some of them still forgot their code. Lesson learned, nothing is fool proof.
- I did the post-test measure into the following term in someone else’s class. The person who was teaching was not willing to give me much of her class time for the students to complete the questionnaire. Some of them took the questionnaire home and I never saw it again. When I got beautiful written reflections about their writing experience in the early part of the course, in post test, the qualitative questions were left blank or were one or two sentences long.
- I didn’t think about program attrition. I’m not really sure we realized the degree of our own program attrition at that time but there were many students due to multiple course failures, that didn’t even make it to that third term. I lost all my low grade students from the sample and they are an important voice in writing self-efficacy.
- Anxiety — I asked them to rate their anxiety based on their “next” scholarly paper which worked well for the first data collection period because they were about to write the paper assigned in my course. But I made a stupid and naive decision for the post-test. These students didn’t have a known scholarly paper coming up at the end of the course. What exactly were they rating or imagining when I asked them to rate their anxiety for their next scholarly paper?
Nevertheless, when I went forth and analyzed the data some interesting things emerged. And I realized that since I’d last written a research study paper, that all the rules had changed. Two things I will discuss in future parts.