Sunday, March 13, 2011

An Analysis of a Meta-analysis

This week we were asked to read a professional article named A Meta-Analysis of the Effectiveness of Teaching and Learning with Technology on Students  by Waxman, Lin and Michko. You can get a pdf copy of the article here

The article was a statistical analysis of 42 different studies done on the impact of technology in the classroom.  In all instances the studies cited had a positive impact on student learning.  This lead the authors to state that technology usage had a "small, positive, significant" impact on learning and the classroom in general. 

As a science teacher I am always a little leery of social science reports.  By virtue of me being in this class I am biased towards the positive impact of technology in the classroom.  I would like to say that my technology use in the classroom makes me a more effective teacher and my students' use of tech makes them more engaged.  That all being said, the collation methods the authors used in this article admit that much of the data being used was shoe-horned to make it an apples to apples comparison.  The authors state that they use a series of formulas that they used to back calculate much of the data they needed to order the data effectively.  I would love to see the inner workings of this statistical analysis at play.

Aside from my nit picking of the statistics, I thought the article was a very useful piece.  There is a lot of data out there and it was good to see some of it coming together in a very usable fashion. The part that I found the most interesting was the assertion that small group use of technology had more positive student impact than having students use technology on a one to one basis.  This has taken me aback a little.  On one hand student engagement increases for most students during group work projects instead of one on one, but there are always those students who hog the computer from their classmates.  Also, I find that group work decreases in engagement the second there are more than three students in a group.  It would be interesting to see what group dynamics are helped and which are hurt by the inclusion of technology into the classroom.

2 comments:

  1. I too was a little skeptical of the methodology, but I'm not knowledgeable enough in research methods to judge. Your comments about the effectiveness of cooperative learning interests me in terms of how it could be applied in the virtual environment. Recent collaboration tools makes group work possible online (as we have demonstrated in our small group) but how could you study and quantify the effect on student outcomes?

    ReplyDelete
  2. The problem with educational statistics is that there are too many variables that affect the results of the tests. To say that it is unequivocal that these studies "tell the truth", I would have to say that these studies tell the truth that the study designers want to be told. If students are engaged, and instructed how to do a task, I would be willing to bet that they are able to have larger gains in education than a student who uses the same technology but is not engaged and does not receive complete instruction on how to do a task. To say anything other than the results of the study are one of many possible outcomes would not be ethical. However, when large sums of money are at stake, sometimes ethics are mutable.

    ReplyDelete