Stop Wasting Children with ICT4E Assessments
A large problem with educational evaluations of any kind is that the “public” (aka, the media) are only interested in national and international competition scores, like the PISA scores. Any reflection on the value of these competitive tests for the children is lost in the media noise.
It seems that it does not matter so much in what you excel, only that you excel. Not coincidentally, this was also the main driving force behind the Chinese imperial examinations in the previous post.
In academic circles, there is a lot more interest in really measuring performance. But in these studies, very specific questions are asked in relation to bounded problems. Nothing like, “Are computers useful?”, because such a question is unanswerable in principle. There are well researched evaluation methods, see the companion post by Mary Hooker. A lot has been learned how children respond to formal education, and how they will learn (better). So the question is, why are these methods not used?
To get a glimpse of an answer, it is illuminating to listen (literally) to David Hestenes who has done ground braking research on the understanding of basic physics in students. His talk “Naïve beliefs about physics and education” is available on-line as a presentation and audio talk.
Simplified to the bare basics, almost everyone seems to believe that education is about the transmission of a substance, called knowledge, to the memories of the students. The schools function as a retail outlet of knowledge. Evaluation of education centres around determining how much of this substance ends up inside the heads of the students. Research has shown that students can indeed reproduce a lot of the factoids sprinkled in the teaching and textbooks when tested. However, when tested in ways that require real understanding of the basic concepts, a majority of students fail completely (see examples in the slides).
His research brought David Hestenes to the conclusion that all teaching methods that are based on “transmitting knowledge” instead of “recreating knowledge” (aka Constructivism) fail to change the naive preconceptions of students.
Summarizing the above in combination with the previous post, there is little hope that ICT4E assessment as it is organized today will uncover anything that will actually influence educational practices. History is plainly against us. A full blown assessment about “Is [fill in ICT4E solution] a cost-effective improvement” will take years to complete, cost serious amounts of money, and will be irrelevant when published. The worst effect will be the delay which will deprive yet again several cohorts of school children in poor countries of adequate education.
I think we must take a common sense approach instead. While waiting for the dissemination of the results of scientific research into the general population we should start with trying to find common ground with teachers, politicians, and parents. That is, seek for approaches that will allow all involved to reach a consensus on that will benefit children now.
Computers alone have benefits
Computers are useful in disseminating information, eg, electronic books, libraries, wikipedia, and as communication devices, eg, email, IM, video. They are great for writing and calculations and can greatly improve collaborative efforts. They are also great at stimulating children to read and write, eg, email, stories, their own blog, to explore their society and the world, and to get exposed and experienced in new languages. And they require and exercise skills that will be valuable in the workplace later.
There might be problems in languages with very little digital content for children. However, experience taught us that the existence of a large “market” of on-line computers is quickly followed by content. Even if the content has to be free (as on the Internet).
Computers in education have more benefits
One really unique benefit of computers is that it is possible to set up applications that allow students to practice skills that could before only be practiced in the presence of a teacher. Teachers can then spend more time on children who need personal attention.
This has been used in language learning for decades. Speaking practice, and reading and writing assignments, have been automated before. Students listen to recordings and record their own speech. Just listening to recordings of your own voice helps you correcting mistakes. There are currently even (limited) applications where a computer application can actually help students correct their writing and pronunciation.
As an example, I will plug here a Free Software (GPL) project to learn Mandarin tone distinction in which I participated: SpeakGoodChinese.
Enough with Assessments – Implement Already
I conclude that schools that lack resources like books and libraries, and over all, the required number of qualified teachers, will greatly benefit from implementing sensible ICT solutions that substitute for these shortages and improve teachers’ effectiveness.
All parties involved seem to agree that ICT solutions could ameliorate a least some of the problems in resource poor schools. But this approach also implies that ICT solutions should not be restricted to the classroom. Practice, studying, and reading should be done at home, collaboration and communication is done everywhere. What use is it to have school books and email on a computer when the student has no access to the computer?
There is obviously one caveat: ICT can only be useful if the (real) total cost of ownership can be made bearable.
Okay, so if all teaching methods that are based on “transmitting knowledge” instead of “recreating knowledge” (aka Constructivism) fail to change the naive preconceptions of students, how does ICT then impart a better knowledge creation system?
I'm the biggest ICT cheerleader around, but I still come back to the educational achievements of classical Greek scholars – the very foundation of much of Western Civilization – who taught us the basic concepts we still hold dear 3000 years later without the aid of a single electron. Its the teachers, not the technology.
So no matter how much we think ICT4E assessments are flawed, its hard for me to then jump to the concept that ICT should be standard.
@Wayan:
"how does ICT then impart a better knowledge creation system? "
No resource necessarily improves education. The quoted research shows that a range of different teaching methods fails to change the naive (pre-existing) believes of students. Whether or not you use ICT to support those teaching methods is irrelevant.
However, if you try to evaluate ICT4E using existing "transmissionist" methods, the results won't make a difference: They will show what you put in them (mostly, they will show nothing)
Going back to the old Greek as you suggest. There is this old story from Plato in the Meno (http://en.wikipedia.org/wiki/Meno) where Socrates argues that the knowledge is already in the students, and the teacher just has to "extract" it (a kind of midwifery). What Socrates then demonstrates would be more like a constructivist example in modern times. I think modern practices are more followers of the Spartans than the Athenians.
And I would be the last to deny the importance of good teachers. I think the old Oxford model of a lector and a few students is best. What ICT can do is help in situations where there is a lack of adequate teachers. But such a support cannot be "proven effective" using standard evaluation procedures based on current grading practices. On the other hand, evidence based educational methods are still not generally accepted.
So I suggest to try to find a common ground where both the "establishment" and "evidence based research" can meet to improve education for disadvantaged schools. This is much better than waiting for an (elusive) consensus on evidence based educational reform. As far as I see it, there are good practical ways to show where and how ICT can help schools. Communication and practice are two obvious areas where ICT can help.
Rob van Son
Hi Rob,
Thanks for this input. I am working in the Pacific on introducing OLPC and despite strong commitment from 15 countries to start trials, it is proving difficult to get funds. Most funders want "hard evidence" or "proven business cases". How do you suggest we get over this hurdle without M&E?
See my response below.
Mostly, that Mary is more likely to be able to answer your specific questions.
Rob
I agree that teachers are very important in children's learning, but if the teachers have few, if any, resources to teach with, and if there is little possibility of them receiving training because there just isnt any budget for that (both very common situations in developing countries), then introducing laptops can be a good intervention. If done properly, there will be many other benefits, but justifying it on accessing quality resources is enough.
In general, I agree with the author of the article, but unfortunately risk averse funding agencies want the research to justify the spending.
Hi Wayan,
In the developing world, there is an acknowledged "wave" of young children flooding the education system (due in part to better health care etc). I have seen reports by UNESCO stating that we do not have enough resources (schools, teachers and just plain money) to provide universal education for these children if we keep trying to educate them in the traditional way. UNESCO strongly suggest that ICT must play a part in meeting this challenge, or we will have children that have never gone to school. This is true today in the Pacific with many countries having only a 50% attendance rate and even if they do get to school, the quality of education is doubtful due to lack of appropriately trained teachers. The challenge is to feed ICT into this equation as part of the solution. IMHO, OLPC is one of the best ways to do this.
Ian
Fortunately, in those nations the poor have grown tired of waiting for their government and wealthy, western savior to come through. They're just putting together such schools as they can under the assumption that something now is better the unfulfilled promises stretching into the unknowable future.
I'd suggest anyone wondering about the role of education in the the less-developed nations take a look at some of the findings of Dr. James Tooley. I think it's possible to infer the importance of computers in education from Dr. Tooley's findings as well.
Instead of commenting I just want to bring matters to the fundamentals. Pl. read what an educationist has to say,-
J.J. Guilbert a well known peer on education, in his hand book on education commented this way, “For curriculum building, the method traditionally used, is to bring together eminent professors and result of their deliberation is presented as a list of chapter headlines. Often existing programmes are used as main source of data for the preparation of the new programme. The professors indicate the number of hours to be devoted to the various subjects to be dealt with. This generally leads to a conflict of personalities and it is the most forceful, most persuasive, sometimes the mot irascible or nosiest of the participant in the discussions who gets her/ his view accepted.” Subsequently he states, “It is reasonable to believe that an educational programme will be effective if its purpose is clearly expressed. This helps the precise evaluation as well.” (Guilbert, 1977, Educational hand book for health personnel, World Health Organisation, Geneva ,1977)
The objectives are often not tackled not because of neglect, but because it is considered understood. It is traditionally said, “We train students for international quality. It is not necessary to develop a description any further since science/ education is universal”: But, when we try to get these experts/ teachers to define a little more, what they are talking about, we see how wide the conflicts among the advocates of divergent shades of thinking are. “This conflict becomes acute during the period of curriculum reforms.” Say, Guilbert.
The result is that the ‘policy variable’ becomes a constant and ‘comptence’ become a variable. For example, if the policy content of a course is fixed by administrative or political decisions unrelated to academic needs, then, there would be hardly any definition on the competence of the student. Educational goals must be defined by using behavioural terms corresponding to the task to be accomplished. In other words the definition must indicate what the graduates of a given school will be able to do at the end of a period of learning, that they were not able to do before. These educational objectives are called learning objectives to distinguish them from teaching objectives; because, they define what the student should be able to do, not the teacher. For this reason it is also called competency objective.
A teachers’ expectation of high student learning motivation will not become a reality until the students feel the freedom to learn, that is to say, when become involved in their own educational objectives. They must not only understand what they are learning, but also what for. Ideally, these objectives have to be explicitly defined and offered for their choice, even better the students will have helped in establishing them in clear terms and elicited by a leading questionnaire.
So what does that common ground where both the "establishment" and "evidence based research" can meet to improve education for disadvantaged schools look like? Have you seen it exist in practice?
"How do you suggest we get over this hurdle without M&E?"
First, you might find Mary an expert on this matter (I am not). She is the one with the extensive field experience.
For what it is worth, I see two obvious courses:
1 Look what the sponsors want, and deliver that
2 Convince the sponsors that the children will get better education
We know that ad 1 will, in all likelihood, not be the best for the students, or yourself, or the sponsors.
For ad 2, Mary Hooker has listed a very good evaluation method (see the picture on her second post), the "GeSCI ICT- Education Matrix". This can be filled in using a questionair. That will bring you up to steam with relatively little effort (compared to a randomized field trial).
In the end, it comes down to formulate what you want to achieve for the children, convincing the sponsors that your approach will bring education closer to this end, and then convince the sponsors that this is worthwhile given the costs.
You can use the links given in the overal debate to show that:
1 There is little hard research on business cases world wide, even in wealthy countries
2 Anything that improves productivity of teachers will improve education
3 Anything that gets the children to spend more time practicing will improve their capabillities
4 With a full business case, you would be the first to apply "evidence based education methods" in the world
(for some reason, that seems to be a good deterrent)
Sorry that this all sounds very "generic" . Without more information, there is little specifics that can be given as I am not familiar with your case. And I really think that Mary is the person who can help out better. Or at least tell you where to ask for your specific case.
Rob
I’ve been showing my Kindle to everyone. This is the best purchase I’ve made in a long time. I’m a psychologist and it has been really helpful in doing some research for a PTSD therapy group I’m starting for members of the military.
My vision isn’t the best, so I really like the ability to change the text size. I use the text-to-speech feature at times. I’ve shown it to one kid who is dyslexic and he volunteered that he thinks it would be helpful to him. I showed it to a girl who also has a reading problem. I downloaded a sample of Twilight for her to try and switched on the text-to-speech. She couldn’t put the Kindle down, reading along with the audio. Her parents were very pleased and I’m sure they ordered a Kindle within a day or two.
People universally have thanked me for showing them this product.
I would love to assume that everything we know is wrong and that somehow kids in the hands of computers would realize the value that we are looking for (and we are talking extremely significant proportions of GDP in many cases)…
But I wonder what is the basis for believing that the resource will be efficiently used? Efficient use of resources is not after all that normal in many places. Preservation of assets sometimes, but utilizing the maximum potential is not something that happens in many fields in the developing world (agriculture, business, you name it).
In villages with little connection to the outside world and their own values, how do we know that the laptops might not simply turn into status symbols, or that we just look at the games instead of learning the maths behind them… Huge amounts of anthropology questions thrown up here really…
And as this is indeed completely new technology don't we really need to gain a deeper understanding of what is going on here in order to achieve an acceptable return on the technology?
What would you say in the US if the government advocated spending $23,450 to buy something you that you didn't have definitive evidence of it's benefit for children in school? There'd be outrage.
And if you take a very conservative laptop + $50 cost and a GDP per capita of a country in the developing world of $500, so the laptop is half of an average annual income (per child over 4-5 years mean time to failure)…
I'm sure that done carefully we can achieve results that more than justify that spending … but let's not waste children for getting it wrong.
@Mike Dawson:
"I would love to assume that everything we know is wrong and that somehow kids in the hands of computers would realize the value that we are looking for (and we are talking extremely significant proportions of GDP in many cases)… "
Actually, that kind of thing DOES happen. At the turn of the 19th/20th century, getting a bicycle made you smarter, because it allowed you to go beyond the boundaries of your village or neighborhood.
The same held for books, newspapers, and radios/TV sets. Giving people cheap books (moveable print), newspapers and journals, bicycles, radios, TV, cars, they developed a much broader level of intellectual endeavor. That is even true when most content is kitchen maid literature and most communication is chatter.
In general, whenever you give people the ability to travel or communicate more and better, they get smarter. In a deep sense, this idea is also one of the foundations of the OLPC.