TQ# 8 – Reflecting on Meaning of some Technology in Education Research

I read the Metiri group report instead of the study by Wenglinsky because I had read the latter last semester. I also read the first study on Reading and Mathematics software effectiveness done by the U.S. Dept of Education.

The research in the first study (the Effectiveness of Reading and Mathematics Software or products, done by US Dept of Education) was done under the mandate of NCLB rules and was conducted by two research forms. The study was done with a very significant sample base (23 school districts). The second study I looked at in summary was the one sponsored by Cisco systems and conducted by Metiri, a highly respected organization.

In assessing what we are learning about the effectiveness of educational technology in our schools, I read the reports with great interest. The amount of dollars spent in schools on technology is significant and one wants to see if these tools are actually contributing to improved learning. As more and more studies occur and, depending on relatively negative or positive findings, the future of technology in schools may be affected at some point.

After reading the first report, I was left with some questions as well as with a general “flat” feeling. One almost had to read the entire report in order to get a clear view of the validity of the study. This study seemed to follow a pretty solid methodology. The sample size was large enough and the data collected on the object of study combined a nice mix of quantitative and qualitative means. Also, both the treatment and control groups were working with software and not just non-technological methods, which thus equalized the groups in some ways. One thing that concerned me was the fact that the teachers were given training on the products by the vendors and not by the testers or by people necessarily in education. We do not know whether any of the trainers might also have been teachers; it could be assumed many were not. One thing we do know was that teachers said they felt confident right after having their training, but not so confident after actually using the products in a classroom. Also, I believe the time the products were actually used in classrooms (a percentage of the total school day) was rather small. No technical difficulties seem to have gotten in the way of the use of these tools.

It is difficult for me to say what was actually going on in this study and in these classrooms, mainly because I did not know most of the products that were actually used in the study. We have teachers reporting that children are working more on their own and not being subjected to lecture style teaching, and these are, of course, good things. However, these observations seem almost unnecessary to make because when you use software you, almost be definition, are not lecturing. I believe that maybe the test scores were little if at all changed due to the following factors or reasons: 1) the teachers did not have adequate training on these products (i.e. training by educators themselves), and 2) the products were not used nearly as much as they should have been to get a good gauge of their capability in enhancing scores and learning. For the study, teachers should have been required to use them more often.

As for the Technology in Schools report by Metiri, this was really a ‘study about studies’ of educational technology and its effects on learning. It made some very good points, I thought, concerning the kinds of skills we should look for that represent the deepest and best kind of learning (things such as automaticity, authentic learning, information processing and visualization, and higher order thinking). It also castigated some studies for not using proper controls (i.e., not even having a control group). Finally, it also reminded us that high quality teaching when using the technology is an absolute requirement for learning advances to occur. They state that there must be “a triangulation of content, sound principles of learning, and high-quality teaching” for technology really to work and be worth the effort. The report, thus, is pretty much a critique of the “Effectiveness” study I looked at above. It also mentions that the technology must be used for a longer term than sometimes has been looked at, and this also was one of my criticisms of the “Effectiveness” study above. The report also says that research is getting better, but that much more good research is needed to assess what effects technology is having in our classrooms.

As the report scans down through the various technologies, going in roughly chronological order, it notes the many places where research has yet to even appear. For some of the cognitive function areas, like visualization skills—which I know a bit more about through personal experience—I found myself agreeing with what the report was saying: these kinds of tools assist students in picturing what-if’s and seeing causal relations in a better way. In sum, this report excited me because, on the one hand, it pointed out that there has been some research which shows some definite positive gains where technology has been implemented, but also, on the other hand, drew attention to the many opportunities for research. I also thought that the study was cautious where it needed to be. All in all, it was a nice review of what has been going on in the area of educational research over a period of some time. Our intuitions are correct, I believe: technology is doing some good things, but doctoral students also will not run out of dissertation topics anytime soon.