{"id":521,"date":"2009-11-12T14:10:32","date_gmt":"2009-11-12T18:10:32","guid":{"rendered":"http:\/\/edutechdebate.org\/?p=521"},"modified":"2012-09-27T10:37:34","modified_gmt":"2012-09-27T14:37:34","slug":"ict4e-assessments-help-avoid-wasteful-tragedy","status":"publish","type":"post","link":"https:\/\/edutechdebate.org\/assessing-ict4e-evaluations\/ict4e-assessments-help-avoid-wasteful-tragedy\/","title":{"rendered":"ICT4E Assessments Help Avoid Wasteful Tragedy"},"content":{"rendered":"

ICTs can be powerful, essential tools for learning: understanding, interpreting and communicating about the real world OR they can be black holes into which we pour our money, intelligence and time, getting very little in return.
Dr. Patti Swarts, Education Specialist (GeSCI)<\/i><\/p><\/blockquote>\n

1. Do we really need to assess ICT4E initiatives?<\/b><\/p>\n

In a word – yes. Yes we do need to assess ICT4E initiatives more particularly when we are working in environments with scarce resources as in the developing world where investment in ICT can constitute what Unwin (2004) describes as a \u2018wasteful tragedy\u2019 if it is not managed and utilized properly. <\/p>\n

There is no doubt that ICT in education presents countries with great opportunities. I suppose this would support the perceived notion presented in this forum discussion that the benefits of technology are obvious – on a par with the obvious benefits of electricity. However the use of any technology whether ICT or otherwise also bring challenges – which need to be addressed if the benefits are to be realized. <\/p>\n

Some of the ICT challenges in Education relate to, cost, sustainability, optimising usage and making teaching and learning meaningful for students, and relevant for the development of the country. Many policy makers, planners, managers, and practitioners still lack experience, knowledge, and judgment capabilities in the systems, methods, and media involved in what is still an emerging field of ICT in Education. <\/p>\n

This lack of experience is particularly evident when dealing with the integration of the newer ICT technologies such as 1:1 saturation models. In my organization, the Global e-Schools and Community Initiative (GeSCI), we believe that assessment is essential for addressing such challenges. <\/p>\n

2. Are ICT4E assessments effective in measuring outcomes?<\/b> <\/p>\n

I find that the belief presented in this discussion that ICT evaluations are necessarily flawed – because we do not have the correct tools to assess ICT impact in conventional systems – has some resonance in the literature. In a GeSCI commissioned meta-review research on ICT in Education, Le Baron and Mc Donough (2009) discuss the \u201cgrammar of schooling\u201d (Arbelaiz & Gorospe 2009 cited in ibid.) – as in the entrenched practices of conventional schooling where \u2018rules constrain transformational curricular innovation especially ICT integration\u2019. <\/p>\n

In this context Angeli and Valanides (2009 cited in ibid.) support a view that the transformational impact of ICT would require \u2018a clear, commonly understood epistemological framework\u2019 in order for \u2018teachers to understand ICT\u2019s transformational potential or for educational decision makers to assess whether or not high standards are being met\u2019. <\/p>\n

Despite the lack of such a common framework for understanding and assessment, I would agree with Clayton\u2019s comment<\/a> in this forum that even if we don\u2019t have a perfect tool to evaluate ICT impact as yet, we should still employ what we have and in the process refine our tools. <\/p>\n

3. Do we even have the tools to tell if they are effective? What tools are those?<\/b><\/p>\n

There are several methods for monitoring and evaluating ICT in Education projects. Two of the most common approaches cited by Wagner et al. (2005) are Outcome Mapping (OM) and Logical Framework Analysis (LFA) \u2013 the former focusing on assessing the changes that programmes bring about in the behaviour and actions of stakeholders (e.g. officials, teachers, parents) \u2013 the latter focusing on assessing results in terms of products and services (e.g. curricula, teacher training, educational software) of a programme. <\/p>\n

4. Are we really using these assessment tools correctly?<\/b><\/p>\n

There is an emerging view that M&E traditional frameworks are not enough. There is a need to include components of applied research for \u2018proof of concept\u2019 \u2013 to provide more rigorous, field-tested knowledge about a) what works and why, b) how the initiative could contribute to educational development priorities (access, quality, capacity and relevance), c) the enabling conditions and barriers for taking the initiative to scale, and d) what might be the result of large scale application ( Batchelor & Norrish n.d.). <\/p>\n

In GeSCI we take the process further \u2013 we believe that for M&E\/research on ICT in Education to have any impact, the results must inform and shape policies and programmes and be adopted in practice. It is the interaction between these three dimensions that we believe is necessary to maximize the potential of ICT use for transformational impact in education systems.<\/p>\n

Figure 1. Stakeholder facilitation in GeSCI research framework<\/b><\/p>\n

\"gesci-framework\"<\/center>
.<\/p>\n

5. And regardless of the outcomes, should we really wait for long-term results, or should we implement ICT4E deployments now, as the case is compelling enough already?<\/b><\/p>\n

In a word \u2013 maybe. The case is not compelling. However governments made nervous by the digital \/ knowledge divide between developed and developing worlds are implementing ICT4E deployments anyway. What we should do is focus our efforts to pursue a deeper and broader research and evaluation agenda.<\/p>\n

In my view such an agenda should shift from the current focus of much contemporary ICT \u2013Education evaluation and research on quantitiatively measured learning outcomes (long term results and potentially problematic in terms of cause\/effect attribution) \u2013 towards a focus on examining the qualitative potentially \u2018distruptive\u2019 force of technology such as saturation models to shake the \u201cgrammar of schooling\u201d in transofmrative ways.<\/p>\n

What do others think? Is the case compelling? Should we assess? What should be the focus of our assessment?<\/p>\n

References<\/b><\/p>\n