Should We Shift ICT4E Assessments From Technology to Adoption?
At the start of this debate, I proposed what I thought was a radical idea – “Do we need ICT4D assessments?” I was under the impression that we’d hear a few calls for dismissing them, but there would be an overwhelming validation that evaluations were not only necessary but instructive for decision makers and the educational stakeholders faced with pressure to accept technology in the classroom as inevitable.
From the onset, my assumptions were proven wrong. As Rob van Son stated in his opening post:
However, the most remarkable thing about any ICT4E assessments to decide on the introduction of ICT in education would be their uniqueness in history. One reason such assessments are so scarce is that there are few (if any) historical examples of assessments of any kind done before the introduction of an educational reform. Even less examples where the outcomes of the assessments really mattered in decision making.
This opinion was backed up in the comments with the general consensus similar to John LeBaron’s comment that while assessemnts themselves are not bad, but we don’t have the right assessment tools and their outcomes would be misused anyway.
Mary Hooker’s response was more in line with my thoughts – she opened with:
Yes we do need to assess ICT4E initiatives more particularly when we are working in environments with scarce resources as in the developing world where investment in ICT can constitute what Unwin (2004) describes as a ‘wasteful tragedy’ if it is not managed and utilized properly.
But then surprisingly, quickly agreed with many of the issues raised by Rob van Son – ICT4E assessments don’t capture outcomes very well, and are often disregarded because the entrenched practices of conventional schooling has strict rules that constrain and retard transformational curricular innovation, especially ICT integration.
Yet, Clayton Wright brought up a great point:
From a western perspective, the benefits of technology are obvious. But in the developing world, perhaps only the benefits of mobile phones, ATMs, and radio seem obvious. When given a choice, local officials would probably spend funds on providing clean drinking water, toilet facilities, medicine, and seeds for crops rather than spend it on ICTs. Local educators would want more teachers not computers. Thus, I believe that evaluations are necessary to demonstrate to the local officials and national policy makers that ICTs are worth the investment. They need to know what local problem(s) ICTs can address or opportunities that are possible.
How can we assure them that technology should be on an equal footing with other educational investments then? Can we really take Rob van Son’s title, “Stop Wasting Children with ICT4E Assessments” to heart? Is he right when he says:
There is little hope that ICT4E assessment as it is organized today will uncover anything that will actually influence educational practices. History is plainly against us. A full blown assessment about “Is [fill in ICT4E solution] a cost-effective improvement” will take years to complete, cost serious amounts of money, and will be irrelevant when published. The worst effect will be the delay which will deprive yet again several cohorts of school children in poor countries of adequate education.
In addition, if all teaching methods that are based on “transmitting knowledge” instead of “recreating knowledge” (aka Constructivism) fail to change the naive preconceptions of students, how does ICT then impart a better knowledge creation system than an increased focus on better teachers and more involved instruction? Rob van Son responded with a reasonable argument:
I think the old Oxford model of a lector and a few students is best. What ICT can do is help in situations where there is a lack of adequate teachers. But such a support cannot be “proven effective” using standard evaluation procedures based on current grading practices. On the other hand, evidence based educational methods are still not generally accepted. So I suggest to try to find a common ground where both the “establishment” and “evidence based research” can meet to improve education for disadvantaged schools. This is much better than waiting for an (elusive) consensus on evidence based educational reform.
In her second post, Mary Hooker agrees with Rob – there are inadequacies of evaluating the use of ICT and its potential for transformational innovation in education systems that are intent on simply harnessing it for maintaining the status quo. It doesn’t help that ICT is also multifaceted, with effects and impact beyond the expected, event by experts.
Worse, ICT is also very fast moving. With the quick change in technology (netbooks, mobile phones, Internet access) over the last few years, long-term studies of ICT impact can be obsolete before they are even finished. To this Mary Hooker responds that the focus should be not the devices then. ICT does not exist in isolation but within a larger socio-cultural context of the school and education system.
Which to me, brings us to classic Change Management – the systematic process of creating organizational change.
In that context, should ICT4E assessments ignore the technology – it changes to fast and is too hard measure its impact – and focus on assessing the ability of educational systems to best adopt ICT usage to their goals? A shift from “device X helps Y amount in reaching Z learning outcome” into “school system A has B ability to incorporate C amount of technological change within its current resource constraints” and then leave the school systems themselves to experiment with different technology?
Education development practitioners and the inteon rnational cooperation agencies must make an effort to understand the development context in which education is taking place. If assessed from the Maslow Triangle perspective, ICT is at the very top. Therefore, we must find ways to reach the low hanging ICT fruits first, provided that the enabling facilities are secured (i.e., infrastructure, electricity, and sustained maintenace) Humanity has to walk faster towards a revolution on ICT in the education sector of developing countries. This will then address collateral MDG issues sus has HIV, premature pregnancy, food security, and the like
Please hear my audio comment at the link below:
http://fpamediaserver.wcu.edu/~jlebaron/ICT4E/Lin…
References for the two studies cited in the mp3 voice file are:
Lin, J. M. (2008). ICT education: To integrate or not to integrate? British Journal of Educational Technology, 39(6), 1121-1123. doi:10.1111/j.1467-8535.2008.00825.x
Peltenburg, M., van, d. H., & Doig, B. (2009). Mathematical power of special-needs pupils: An ICT-based dynamic assessment format to reveal weak pupils' learning potential. British Journal of Educational Technology, 40(2), 273-284. doi:10.1111/j.1467-8535.2008.00917.x
Hi Wayan,
Now you are opening up an interesting angle in this discussion – and I suppose in this way – the discussion has come full circle – because when we talk of change management we will inevitably consider effective leadership – and all of these processes you refer to – systematic processes for creating organizational change – all of these are are all co-related with assessment and curricular efficacy – and the critical role of leadership to make them work –
" After years of work on structural changes – standards and testing and ways of holding students and schools accountable – the educational policy world has turned its attention to the people charged with making the system work… Nowhere is the focus on the human element more prevalent than in the recent recognition of the importance of strong and effective leadrership" (Policy Focus Converges on Leadership, Education Week 2000 in Fullan 2007)
So perhaps the question in the debate is not so much whether we need Assessment4ICT as whether we need good leadership to make it assessment work – whether it be assesement4ICT or other changes processes – there is a need to balance the push for assessment / accountability with the equally important task of building school capacity (knowledge, skills, resources) for improving and transforming practice on the basis of assessment – as in assessment of and for learning …
Fullan, M. 2007. The new Meaning of Educational Change. 4th Edition. New York: Teachers College Press
Hi Wayan,
Do not forget that to initiate change, you need to have a picture of where you are going.
I think any kind of assessment in education, or anything else, should start with an evaluation what it is that education should achieve, and where you are now. As the times are changing (as they have always done), any school system should evaluate what it is their pupils need to know and do when they leave school.
There is one thing sure, they will never ever again have to "do the exam" 🙂
Rob
Good point, Rob. I recall John Cowan (at the time of the British Open University — Scotland) saying in his thick Scottish accent: "If you don't know where you're going, any bus will do." I can still visualize him saying this, and see that John is still active giving seminars and workshops based on his passion for formative evaluation and student-centred learning.
At Jhpiego we use task analysis of recent graduates to guide curricular review. As you point out, it is silly to initiate educational change if it won't better prepare students to perform in the real world.
To take the debate on assessing ICT for Education onto the next level, it is likely to be necessary to use some sort of formal assessment methodology that uses internationally-comparable indicators. The UNESCO Institute for Statistics, one of the members of the Partnership for Measuring ICT for Development, has recently published a Guide to Measuring ICT for Education. The report is available at: http://www.uis.unesco.org/template/pdf/cscl/ICT/I…
OLPC has been children centered and mainly introduced into children who find XO more of an novelty (as does any child when given a computer to play with city or rural).
Emphasis should be the development of the teachers before the children. Our current system of education has been teachers guiding/teaching the children. 90% of most of the developing world's education budget has gone to teachers' salaries.
The problem I feel is the wrong kind of technology and approach to education that is at fault. I always say, dont try to use rich men's tools to reach the poor guys. It would never work.
Now if we are able to empower teachers to the next level from blackboards centered to mixed blackboard and digital through intelligent use of training and software, it would be much much better than spending money on giving every child an XO. It is really a big confusing class where each child has a computer. Today's school evironments are just not suited for such. An XO and a projector is the best addition to classes today … never one pc per child in class. What a waste of resources… and I can imagine the headaches teachers to go through controlling those children. On paper they never mention these but very glossy excitements children became when given the XOs.(really which child in this world wouldn't be).
Then and only then you see ICT in mass education start moving.
As it is, ICT4D , ICT4E or ICT for anything… hardly any real results. The digital divides are still widening. We are still talk talk and talk from UNESCO TO World Bank. Mostly talks only. Every conference you go, you hear problems and not practical solutions. You see the exhibitions displaying their solutions.. very high tech from electronic black boards to 3Ds etc. You would be lucky to find one showing you how to reach the rural poor child when the conference is mostly about how to close the digital divides.
Alan
The internet has the capacity to reach a lot of poor children, certainly those living in cities. There are some educational reasons to deploy computers in different ways. You mentioned whole class teaching which is one, but a group working around a computer as a team is another as is the individual using the machine as a research, communication and presentation tool. No doubt there is some need to make priorities when resources are limited but it is very difficult to put a value to different educational scenarios. Integrating staff development in the learning process is highly desirable and curriculum development is generally weak where the staff development is weak.
It is true that technology capacities morph constantly, creating a moving target that is difficult to assess with accuracy or relevance. But I wonder…? Is it "technology" that we ought to attempt assessing, or is it teaching strategy and learner production, which technology might or might not support? And what do we assess? Test results? If comparative test results show divergence between technology-supported and technology-scarce environments, should we conclude that technology is responsible for the divergence? Probably not.
While I agree with Negroponte's recent assertion that ICT should now be accepted as a basic utility, I disagree that we can therefore do away with attempts to assess the contribution technology can make to learning. Rather, I suggest that such assessment informs best when it is naturalistic and when it is keyed to deliberate intentions. Is the technology-supported strategy conceived imaginatively, to worthwhile purposes, and are the observable results consonant with intentions?
Readers might be interested in a recent hour-long radio interview with UCLA's Mike Rose on the "Meaning of Intelligence." His remarks touch upon some of the questions driving this debate, especially the issues he presents relative to schooling, the valuing of different ways of knowing, our predominant methods for assessing student performance, and alternative ways of viewing such challenges. You'll find the link at
http://speakingoffaith.publicradio.org/programs/2…
Some very interesting points but i believe your analysis and bias leaves lots to be desired. Then in fact, that’s just my opinion. Have an excellent day undoubtedly a thought-frightening post.