Comments on: ICT4E Assessments Help Avoid Wasteful Tragedy https://edutechdebate.org/assessing-ict4e-evaluations/ict4e-assessments-help-avoid-wasteful-tragedy/ Educational Technology Debate Wed, 08 Feb 2012 18:33:29 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Ian Thomson https://edutechdebate.org/assessing-ict4e-evaluations/ict4e-assessments-help-avoid-wasteful-tragedy/comment-page-1/#comment-944 Ian Thomson Fri, 04 Dec 2009 00:18:52 +0000 http://edutechdebate.org/?p=521#comment-944 I think there is another aspect to consider about introduction of ICTs in education, especially in developing countries. Normally Education in such countries is very ridgid and bureaurocratic and teaching is generally transferring the teachers information to the students in a top down approach. I see ICTs as an "agent of change" in such situations, for both teachers and students So perhaps we sould evaluate what changes have occurred (both in teaching and learning) rather than students test scores. With such feedback, we can introduce a feeback loop into a continuous change model to improve education. I should add that I am an engineer by training, but have a bit of expereince in introducing OLPC into the Pacific region I think there is another aspect to consider about introduction of ICTs in education, especially in developing countries.
Normally Education in such countries is very ridgid and bureaurocratic and teaching is generally transferring the teachers information to the students in a top down approach.
I see ICTs as an "agent of change" in such situations, for both teachers and students
So perhaps we sould evaluate what changes have occurred (both in teaching and learning) rather than students test scores. With such feedback, we can introduce a feeback loop into a continuous change model to improve education.
I should add that I am an engineer by training, but have a bit of expereince in introducing OLPC into the Pacific region

]]> By: Stop Wasting Children with ICT4E Assessments « Educational Technology Debate https://edutechdebate.org/assessing-ict4e-evaluations/ict4e-assessments-help-avoid-wasteful-tragedy/comment-page-1/#comment-852 Stop Wasting Children with ICT4E Assessments « Educational Technology Debate Fri, 20 Nov 2009 14:26:16 +0000 http://edutechdebate.org/?p=521#comment-852 [...] ICT4E Assessments Help Avoid Wasteful Tragedy [...] [...] ICT4E Assessments Help Avoid Wasteful Tragedy [...]

]]> By: John LeBaron https://edutechdebate.org/assessing-ict4e-evaluations/ict4e-assessments-help-avoid-wasteful-tragedy/comment-page-1/#comment-845 John LeBaron Tue, 17 Nov 2009 02:26:10 +0000 http://edutechdebate.org/?p=521#comment-845 Mary et al. From the Phase two Report commissioned by GeSci we have informally discussed the paucity of attention devoted to ICTs in the broad educational literature. ICT specialists seem to be chattering robustly among themselves. This concern is reinforced by a recent US-based meta-analysis commissioned by the US Department of Education on empirical evidence of online learning outcomes in comparison toF2F or bended settings. The only research findings included in this study were scientific, quantitative approaches comparing outcomes in one environment versus others. It was interesting to note that virtually none of this research targeted the K-12 demographic. Indeed, until 2006, there were no qualifying K-12 studies at all. This, despite the billions of dollars that had already been invested in virtual schooling by that time. Now, there's a lot more to assessment than empirical outcomes-comparing research (apples to oranges?), but learning of its total absence among the studies found valid by the USDoE is a trifle unsettling. The USDoE Report is available in PDF at <a href="http://www.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf" target="_blank">http://www.ed.gov/rschstat/eval/tech/evidence-bas...</a> Mary et al. From the Phase two Report commissioned by GeSci we have informally discussed the paucity of attention devoted to ICTs in the broad educational literature. ICT specialists seem to be chattering robustly among themselves.

This concern is reinforced by a recent US-based meta-analysis commissioned by the US Department of Education on empirical evidence of online learning outcomes in comparison toF2F or bended settings. The only research findings included in this study were scientific, quantitative approaches comparing outcomes in one environment versus others. It was interesting to note that virtually none of this research targeted the K-12 demographic. Indeed, until 2006, there were no qualifying K-12 studies at all. This, despite the billions of dollars that had already been invested in virtual schooling by that time. Now, there's a lot more to assessment than empirical outcomes-comparing research (apples to oranges?), but learning of its total absence among the studies found valid by the USDoE is a trifle unsettling.

The USDoE Report is available in PDF at
http://www.ed.gov/rschstat/eval/tech/evidence-bas…

]]> By: Clayton R. Wright https://edutechdebate.org/assessing-ict4e-evaluations/ict4e-assessments-help-avoid-wasteful-tragedy/comment-page-1/#comment-831 Clayton R. Wright Fri, 13 Nov 2009 11:45:17 +0000 http://edutechdebate.org/?p=521#comment-831 From a western perspective, the benefits of technology are obvious. But in the developing world, perhaps only the benefits of mobile phones, ATMs, and radio seem obvious. When given a choice, local officials would probably spend funds on providing clean drinking water, toilet facilities, medicine, and seeds for crops rather than spend it on ICTs. Local educators would want more teachers not computers. Thus, I believe that evaluations are necessary to demonstrate to the local officials and national policy makers that ICTs are worth the investment. They need to know what local problem(s) ICTs can address or opportunities that are possible. Because a new technology or method comes along does not necessarily mean that we need a brand new tool to measure changes. We can still use existing evaluation models (providing we know their limitations) such as Tyler’s Model (measuring gains within a treatment group), Stufflebeam’s Context-Input-Process-Product (CIPP) Model, and Stake’s Responsive Model. Or, simply ask: Did the introduction of “x” achieve the project goals? If not, why? In an ideal world, assessments need to be more rigorous. In the developing world, this can be a challenge - especially the achievement of large scale applications unaffected by various intervening variables such as culture. Qualitative studies add depth, but not necessarily breadth. I agree with you that the results of any assessment and research “must inform and shape policies and programmes”. Having conducted a number of evaluations myself, the real challenge is not about conducting the assessment – the real challenge is to get decision-makers to take action and to act in a manner that benefits the learners and their community in the long-run. You can conduct a variety of assessments and make sound recommendations, but they have limited impact unless decision-makers take action. (Often, once the splashy news/media coverage disappears, the decision-makers do not monitor the project, identify and address challenges, and provide the necessary support.) From a western perspective, the benefits of technology are obvious. But in the developing world, perhaps only the benefits of mobile phones, ATMs, and radio seem obvious. When given a choice, local officials would probably spend funds on providing clean drinking water, toilet facilities, medicine, and seeds for crops rather than spend it on ICTs. Local educators would want more teachers not computers. Thus, I believe that evaluations are necessary to demonstrate to the local officials and national policy makers that ICTs are worth the investment. They need to know what local problem(s) ICTs can address or opportunities that are possible.

Because a new technology or method comes along does not necessarily mean that we need a brand new tool to measure changes. We can still use existing evaluation models (providing we know their limitations) such as Tyler’s Model (measuring gains within a treatment group), Stufflebeam’s Context-Input-Process-Product (CIPP) Model, and Stake’s Responsive Model. Or, simply ask: Did the introduction of “x” achieve the project goals? If not, why? In an ideal world, assessments need to be more rigorous. In the developing world, this can be a challenge – especially the achievement of large scale applications unaffected by various intervening variables such as culture. Qualitative studies add depth, but not necessarily breadth.

I agree with you that the results of any assessment and research “must inform and shape policies and programmes”. Having conducted a number of evaluations myself, the real challenge is not about conducting the assessment – the real challenge is to get decision-makers to take action and to act in a manner that benefits the learners and their community in the long-run. You can conduct a variety of assessments and make sound recommendations, but they have limited impact unless decision-makers take action. (Often, once the splashy news/media coverage disappears, the decision-makers do not monitor the project, identify and address challenges, and provide the necessary support.)

]]>