Role of ICT in Improving the

Quality of School Education

in Bihar
Chirashree Das Guptaa and Haridas KPNb

Ambedkar University, Delhi (contact: chirashree@gmail.com), b Samakaal – An Initiative for Social Change (contact: haridaskpn@gmail.com)
Working
a

Paper 12/0169
March 2012

International
Growth Centre
London School of Economics
and Political Science
4th Floor, Tower Two
Houghton Street
London WC2A 2AE
United Kingdom
For media or
communications
enquiries, please
contact Mazida Khatun
mazida.khatun@theigc.org

Directed and Organised by

www.theigc.org
 1

Role of ICT in Improving the Quality of School Education in Bihar

Chirashree Das Gupta and Haridas KPN1

Government of Bihar has taken an initiative in the implementation of Computer

Aided Learning (CAL) in school and has been recognised for these efforts through the
Manthan South Asia award for e-governance for the year 2010 under the e-education
category for its project "Implementation of computer aided learning in 244 schools in
Bihar under the BEP-India". The literature on use of ICT for pedagogy has set out certain
benchmarks for integration of ICT into the learning process as a way to improve the
quality of school education in specific social contexts. The objective of this study is to
bring out the role of ICT in improving the quality of school education integrated within
pedagogical approaches taking on board the social constraints of schooling in Bihar and
comparing it with the documented results of similar initiatives, experiences and impact
in other parts of India. Government of Bihar has also announced a shift in the focus of
its current IT policy to Information and Communication Technology (ICT). This study will
provide inputs to the education component of ICT policy and school education policy of
Government of Bihar on possible ways to create and improve the learning environment
in schools in Bihar.

1
The authors thank IGC India-Bihar Programme for the financial support for this study. The authors are
also particularly grateful to Professor Anjan Mukherji for his encouragement, support and constructive
criticisms and comments. Pronay Sarkar and Bakshi Amit Kumar Sinha provided invaluable research
assistance to this work. The authors are also grateful to all the field team members for their enthusiastic
work and precise observations which have enriched the results. Mr Sudip Pandey and Mr Suryakant
Kumar’s help for logistical arrangements are gratefully acknowldeged. The authors are grateful to Mr
Ravishankar Singh and all his colleagues at the state and district BEP offices for their patient sharing of
information. The authors are also very grateful to all the students, teachers, school ma nagement
representatives who spared time for the long interviews. Mr Rajesh Bhushan’s support for the facilitation
of this study and his constructive response to the findings of the survey has not only made this work
possible but also enriched it. The authors are grateful for comments from Dr Pronab Sen, Professor P P
Ghosh, and Professor Debdas Banerjee and others who responded to the study when it was presented in
London and Patna. Last but not the least, the authors acknowledge their debt to Mr Anvar Sa dath, the
anchor of the ICT@school programme in Kerala who spared invaluable time for discussion and his insights
have been very crucial to this study.
 2

I Background
Barret’s (2009) review of the international experience of using ICT to improve the
learning environment in schools has been demonstrative in identifying the conditions in
which ICT can be effectively used to enhance the quality of learning and create social
payoffs which would be conducive to sustainable growth and equitable development. It
is largely agreed in the literature that infrastructure, trained teachers, e-literacy or
stand-alone computer lessons though necessary are not sufficient. Integrating ICT tools
into the curriculum and tailoring pedagogy according to the social environment are
necessary for achieving qualitative improvements in learning (Kremer and Holla 2008;
Sreekumar and Sanchez 2008; Barret (2009); Gurumurthy 2009). Kremer and Holla
argue that pedagogical innovations that work around the distortions in educational
systems can improve student achievement at low cost. Technology-assisted learning or
standardized lessons can mitigate weaknesses in teaching and substantially improve
test scores.

In this process of integration of ICT into curriculum and pedagogy, the role of teachers
has been argued to be crucial as the ‘agents’ of change (OECD 2001; Semenov 2005).
Manchin et al examined the relationship in ICT investment and changes in educational
outcomes in the UK. They found that a change in the rules governing ICT funding led to
changes in ICT investment and subsequently changed educational outcomes. They
found a positive causal impact of ICT investment on educational performance in primary
schools primarily in English and to a lesser extent in science. But they could not find any
improvement in mathematics. They also observed that the effect of computer aided
learning is more effective in primary schools than in secondary schools. It was the joint
effect of large increases in ICT funding and a fertile background for making an efficient
use of it, that led to positive effects of ICT expenditure on educational performance.
Thus increase in ICT funding in itself is not enough to ensure the improvement in quality
of education.
 3

A case study of ICT-enriched school environment in Rishon Le-zion, Israel (OECD 2001)
which analysed the impact of innovation in teaching – learning methods implemented
in a school supports the argument that successful implementation of ICT depends
mainly on staff capability to assimilate ICT in teaching and learning processes. The
amount and variety of teachers’ training assured competency of staff regarding ICT
implementation in pedagogy. In addition, the wide range usage of ICT in all subject
matters, all grades and at all times made ICT a vital and essential means of learning.
Students were led by their teachers in an effort to improve their ICT skills and fostering
appropriate usage of technology in pedagogical practices. The study concluded that the
sustainability of the innovation depended on two factors: 1. staff and school
atmosphere and 2. political support from the community and the state institutions.

Synthesis from this literature shows that the effective uses of ICT to improve the
learning environment in schools catering to students from underprivileged social
backgrounds depends on cost-effective scalable delivery systems to meet challenges of:
1. Provision of basic infrastructure and teachers 2. Overall and sustained enhancement
of ICT skill-levels of teachers 3. Motivation of teachers, curriculum designers and other
stakeholders to integrate ICT into curriculum and pedagogy.

While the first two listed above are necessary conditions, it is the third which has been
found to make the critical difference in improvements in learning. These factors have
been studied within the different delivery processes of CAL in schools in various states
of India and a contested literature exists on the efficacy of delivery systems (public
delivery / PPP (Build-Operate-Own-Transfer or BOOT)/ private delivery models). The
most important lessons emerge in the experiences of Kerala and Karnataka, in terms of
experiences and subsequent willingness to make changes to policy, programme design
and delivery based on those experiences (Vidya Bhawan Society and Azim Premji
Foundation 2008; Gurumurthy and Vishwanath 2010).
 4

For example, Banerjee et al’s (2003 ) evaluation of a computer assisted learning and
remedial education programme to improve the quality of education in Vadodara
showed that computers were used effectively only in very few schools in 2002.
Pratham’s intervention with the help of local volunteers ran the programme with an
emphasis on learning Mathematics through computer games consisted of two hour
classes per week for children in the fourth standard with two students sharing one
computer. The design of the programme allowed children to learn as independently as
possible and the interactions between instructors and children were driven by the
child's experience with computer games. The results showed that the intervention led
to an increase in math scores by 0.37 standard deviations. Average scores on a 50-point
math test rose from 14.9 to 29.0 in the treatment group but only from 15.5 to 25.0 in
the control group. At the same time the programme did not have any visible changes in
the language competencies. It also suggested that the more interactive, computer-
based approach to learning might not have created a greater enthusiasm for learning
overall.

Another significant study on Computer Assisted Learning is the one done by Vidya
Bhawan Society and Azim Premji Foundation in 2008. This initiative began in 2001 in
rural Karnataka and was subsequently taken up in other states like Tamil Nadu, Andhra
Pradesh and Uttarakhand. The objectives of this study were 1. To study the
effectiveness of the implementation mechanisms and training s trategies in the
programme 2. To study the process of delivery in CAL classes and its impact in the
classroom. 3. To study the impact of CAL on children, teachers, community and other
stakeholders 4. To study the role of other stake-holders like the state government,
teachers, schools etc.

The study cited above found that in all the states and almost all the categories, more
than half the programmes were not functioning and among those that were functioning
majority turned out to be under the categories ‘average’ or ‘poor’. The programme
 5

seemed to function best in schools in Karnataka compared to other states. Most of the
centres are located in the school or within 2 kilometers of the school. The infrastructure
of almost all CAL centres was in place with all the safety measures needed. Each CAL
centre had 2 to 5 computers. Teachers from all states felt that computers and
technology are essential in today’s classrooms and that technology is not just meant for
private schools. At the same time, teachers across all states felt that computers do not
decrease the role of teachers in classroom process, classroom teaching does not get
diluted due to CAL and neither are learning opportunities reduced due to CAL. Also,
teachers felt that children get an opportunity for self learning throug h CDs. The general
feeling was that children are interested in learning subject matter through CDs and are
actually learning and understanding through the CDs. Teachers across all states feel that
CDs not only increase a child’s concentration but also their creativity and imagination.
Most teachers felt that the achievement levels of children have improved due to CDs.
Teachers in all states felt that CALP has increased regular attendance. Peer Group
Learning was observed in all states with children taking help from their friends to
understand content. A positive relationship was also seen between the teacher and
students during CAL sessions. In all the states children did not hesitate in asking
teachers questions.

However, these perceptions did not actually translate into effective practice. Teachers
had not actually used the CDs as an active tool or aid while teaching. All teachers who
stated that they have made changes in their teaching methods due to CAL, could not say
what these changes were. Children in all the states reached conclusions through trial
and error when it came to solving problem. Most teachers said that they had received
training for CAL but they felt that it was insufficient. One of the most important
conclusions was that effectively CAL was an add-on and not an integral part of the
teaching-learning process. The programme had not been able to engage with the
teachers at a deeper level. There was an indication in interaction with the team of a
feeling of very rapid expansion and a lot of expectations from the government. This
 6

expectation and the underlying assumptions had not somehow worked and this had
resulted in a large number of centres being dysfunctional.

Gurumurthy’s (2009) study based on policy reviews, theoretical explorations and

empirical evidence of delivery systems of CAL in Kerala and Karnataka points out that
the digital medium has the capacity to allow local knowledge construction and also
supports all the modes (text, audio, video). Hence its potential for revolutionising
teaching learning needs to be explored. However this exploration needs to be firmly
grounded in both educational aims/philosophies as well as educational contexts and
anchored by educationists to be successful.

Gurumurthy’s (ibid) study identifies the reasons for the failure of the BOOT model:

Typically the bids for the program tend to be highly

competitive. The winner is usually the organization who has offered
to implement the program at the 'least' cost. The very low margins also
mean that the computer instructor who is deputed to the school is a very
poorly paid person. The primary cause for the poorly qualified trainer is
that the monthly payment to the instructor is in the range of a few
thousands, far below what the teacher gets and what a competent
computer trainer would get elsewhere. Thus the breadth and depth of
understanding as well as skills in computers of the trainer is usually
inadequate (along with low motivation and job satisfaction) which
defeats the purpose of having an 'external expert'. More importantly, the
teachers and the school treat the program as an 'external' activity that is
not a part of the schools primary purpose nor of its mainstream work
flows. The responsibility of 'computer learning' is assumed to be entirely
that of the external trainer with the teachers having no role or
responsibility. Thus the program largely remains a standalone or 'special'
 7

venture, not integrated into the regular activities of the school... Thus
computer learning programs that bypass the processes of building the
active support of the teachers, both at a micro (school) and macro (the
teaching community) levels have all faced uncertain future, not
being able to figure out sustainability beyond the program.

While the stated goal of CLPS in Karnataka includes “Enrichment of existing curriculum
and pedagogy by employing ICT tools for teaching and learning”, the State's
Mahiti Sindhu and ICT@Schools programmes follow the BOOT model in keeping CLPS
distinct from the regular teaching-learning activities of the school, which has resulted in
little impact of the programs on the existing curriculum and pedagogy (Gurumurthy
2009).

Based on the failure of the BOOT model in Karnataka and the success of public delivery
of CAL in Kerala, the policy outline for ICTs and learning according to Gurumurthy (ibid)
are 1. focus on teachers and teachers’ training and not on direct student learning using
ICTs (ICTs are complex and need to be interpreted to young minds and hence teachers
need to build their own capacities before they can do such interpretation). The
successful models have all focused on teachers 2. focus on computer / ICT aided
learning and not on computer literacy 3. focus on systemic improvement rather than on
specific topics/subjects 4. focus on keeping 'public ownership' over knowledge
resources instead of privatising knowledge.

Two contrasting conclusions and policy prescriptions emerge from Banerjee and Duflo
(2011) and Gurumurthy (op.cit) based on differences in the interpretation of ‘failure’ in
different contexts of the effective use of digital media in learning and the factors
defining cost-effectiveness of delivery. While Banerjee and Duflo concentrate on
exploring the failure of pedagogical practices and ways to improve the state of
pedagogy in a given institutional context, Gurumurthy focuses on the institutional
 8

factors which shape the status of pedagogy in the public delivery system and specifies
ways for institutional improvement.

II Aim
Government of Bihar’s CAL (e-samarth) programme in learning centres and middle
schools has been experimenting with multiple delivery systems. A public delivery model
had been in place earlier under SSA which covers 234 middle schools (Model 1). Since
2005-06 a decentralised BOOT model has been covering 141 schools (Model 2). The
current initiative based on PPP implemented by a consortium of partner organizations
with IL&FS as the implementing partner has been introduced in 244 schools spread over
all 38 districts of Bihar (Model 3). In total, 175000 students and 2100 teachers in 619
centres spread over 375 blocks in all districts of Bihar are officially covered under the
programme. There are also private delivery initiatives by education, skill and software
providers. Thus Bihar presents a unique opportunity to study the relative efficacy of the
different delivery models in the same social environment in how far these have
succeeded in improving the quality of learning in elementary schools. In our discussions
with the Director, BEP, it emerged that the government is particularly interested to
know the 1) extent of improvement of learning in schools specially understanding the
hard spots in Language, Mathematics and EVS 2) impact of e-contents on motivational
level of learners in understanding technical skills, self-learning and self-evaluation. 3)
role of private partners and NGOs in delivering improvements in learning.

In this research, we had set out to do the following:

1. Design a set of simple indicators to measure the actual success and potential of
the current initiatives to provide scalable cost-effective and equitable delivery solutions
to schools catering to students coming from underprivileged backgrounds to overcome
the challenges of the digital divide in 1. Provision of basic infrastructure 2. overall and
sustained enhancement of ICT skill-levels of teachers 3. incentives of teachers and
 9

curriculum designers and the other agents to integrate ICT into curriculum and
pedagogy .
2. Conduct a field survey using multiple field methods (coded questionnaire based
interviews , participant-observer methods, participatory focus group discussions) of ---
schools and learning centres in ----districts of Bihar to collect data for documentation
and data for calculation of the indicators for Bihar.
3. Analyse the data based on qualitative and quantitative techniques
4. Outline and compare the documented results from the field survey in Bihar with
other states and particularly those in Kerala and Karnataka.
5. Cull out the significant and relevant policy inputs from the findings of the study
for Government of Bihar’s current initiatives around e-samarth.

III Method

What constitutes Learning Effectiveness?

There are no standard systems to measure learning effectiveness. A mix of three distinct
approaches to measure improvements in quality of learning - behavioural, cognitive and
technical, defines the indices of quality of learning used for the international monitoring
of what is now referred to as the Millennium Learning Goals (MLG). This mix is a
refinement in response to the critique that the Millennium Development Goals for
Education had not addressed the inequities arising due to differences in quality of
learning (Barrett 2009). The EFA Global Monitoring Reports (Various Years) cover
teacher supply and quality, finance, learning time, learning environments, school
resources as well as learning outcomes as measured by assessment of cognitive skills
(ibid). According to Barret (2009) of Edqual, a research programme consortium on
implementing quality education in low income countries, while the main emphasis has
to be on the measurement of cognitive learning outcomes, international monitoring
 10

agencies working on quality of basic education also report on measurable inputs like
supply and quality of teachers (e.g. academic and professional qualifications and
absenteeism). This framework has been extended by many researchers to study
effectiveness of CAL (for an example, see Taylor and Ku 2011).

While this debate has led to a technical framework combining qualitative and
quantitative parameters, in which quality of learning is measured through either self-
assessment (Banerjee and Duflo 2011; Vidya Bhawan Society and Azim Premji
Foundation 2008) or assessment by others through standardized ‘one size fits all’ tests
(e.g. Pratham’s test questions for ASER reports). Both approaches have limitations, and
a combination of the two have often been used to design frameworks e.g. World Bank
2007. However, these techno-managerial frameworks do not throw any light on: i) the
relationship between learning outcomes and socio-economic status of individuals ii)
implications of the relationship between learning outcomes and economic wealth iii) the
links between learning outcomes and the process of learning iv. links between process
of delivery and learning outcomes (Bowles and Gintis 1976; Goldstein 2004; Barret
2009; Tikley 2010).

This process centric debate of the last three decades has led to research initiatives
designed for a particular socio-economic context to estimate the impact of quality of
learning within a specific programme. These designs are based on four factors:
knowledge, skills, attitudes and process (Moody et al 2002). SACMEQ, a cross-country
survey of 42000 primary school teachers and pupils in Sub-Saharan Africa, which is one
of the largest studies on quality of education done so far, shows that programme
effectiveness (measured by programme context, design and delivery process) in
improving the four areas mentioned above is highly contingent on pupil background
(e.g. gender, age, socio-economic status), school context (e.g. school location, size,
average pupil socio-economic status), school process (e.g. facilities, procedures and
teacher characteristics). So ‘process’ has to be studied at multiple levels. Combining
 11

these two approaches, we designed a ‘learning effectiveness survey’ for CAL in Bihar
based on the following:
a. Defining the Learning Goals: The effectiveness of any particular educational
programme can only be assessed in the context of its learning goals defined as
“particular knowledge, skills or attitudes that participants should have at the end of the
learning episode” (Moody et al op.cit).
b. Designing Variables: A combination of latent, observable, and measurable
variables to measure Learning Effectiveness (Short Term Learning) to assess the overall
effect of the intervention on students enrolled in CAL for languages, mathematics and
environmental science to indicate:
1. Knowledge (K Variables): what was the effect of the intervention on increasing
knowledge in those particular subjects (e.g. did the student show any improvement in
acquisition, retention and reproduction of knowledge after enrolment in CAL; did the
teacher acquire and retain the knowledge disseminated through the training program)?
2. Skills (S Variables): what was the effect of the intervention on improving skills
(e.g. is there any demonstrable skill acquired by the student in ICT for course-work in
the subject areas under CAL – e.g. use of multimedia to improve vocabulary, or use of
ICT for EVS project work; has the teacher been able to use ICT tools for doing lesson
plans, designing assignments etc)?
3. Attitude (A Variables): what was the effect of the intervention on changing
attitudes (e.g. was any topic covered in the Mathematics curriculum perceived as easier
to understand by the student after introduction of CAL; does the teacher now aspire for
more time on computer, use of OHP facility for her/his teaching work)?
4. Long Term Learning (L Variables): what was the effect of the intervention beyond
the scope of the course itself (e.g. on choice of future courses for students, on future
design of courses/curriculum and approach to pedagogy)?
5. Process Variables (P variables): what would be the modifications necessary to
the existing process to strengthen the integration of the CAL programme in the existing
 12

curriculum and pedagogy (this will cover student background, school process, school
context, programme delivery context and delivery process).

The underlying assumptions are:

I) Learning Effectiveness in the short term (measured across cohorts at a point of time)
will be indicated by relative gains in knowledge, skill and attitude because learning
effectiveness is dependent on how well the learning goals set by the programme were
achieved and how much of these were attributable to the programme itself and how
much to other process variables.
II) Long Term Learning gain will be determined by Learning Effectiveness (Short Term
Learning) because how effectively students and teachers learn and utilise their skills
acquired during the programme will determine the usefulness of the learning beyond
the scope of the programme.

IV Design

1. Framing The Survey Questions: The survey used three different sets of schedules
(respectively for students, teachers and school management) covering coded
responses to questions for each of the five areas outlined above for design of
variables. It also included a few open ended questions to increase diagnostic
power.

2. Correcting Perception Bias: This design may be prone to ‘perception bias’ as the
schedules are intended to capture mainly student and teacher response.
However, it is established in the literature that perceptions of respondents in
learning programmes have a high correlation with their performance in the
programme (Feldman 1976; Cashin 1995). Thus, perception bias if any can be
captured through tests of correlation of perception and
achievement/performance. For this, data has been collected on the respondent
 13

students’ scores in examination and assessment exercises in languages,

mathematics and environmental sciences (involving use of computer aid) and
also on assessment of respondent teachers’ training performance in CAL.
However, this exercise has largely revealed that record keeping has been one of
the key weaknesses of the CAL programme in Bihar with inadequate
documentation of performance at various stages.

3. Validity and Reliability of Design: Inter-item correlation analysis and Cronbach’s

alpha reliability test have been carried out on the survey design outlined above
to test for construct validity and reliability for assessing learning effectiveness in
CAL. The results are summarized in Appendix I. In two different test designs,
testing major item correlations from the survey, the survey design is validated
and found to be reliable.

4. Sample design for survey: Sample design needed to ensure coverage of all
constituents of population to capture links between quality of learning and
factors like socio-economic status and regional economic wealth as well as
ensure fulfillment of randomness criterion necessary for statistical analysis. It
also had to be such as to enable comparison of the specific implications of
multiple delivery systems. To ensure these, a two-stage sampling technique was
used. Using the principal investigator’s earlier results of mapping of social
demography of districts to strength of public delivery systems (Das Gupta 2010),
out of the districts where all three models of delivery are operational, three
comparable districts (Bhojpur, Samastipur and Saran) were selected through
purposive sampling based on an index combining district level GDP, poverty, sex
ratio, female literacy and work-participation rates, maternal mortality,
vulnerability to poverty, share of dalit, adivasi and minority population, and
ranking of the district by public service delivery for six major interventions. On
this index, these are three comparable representative districts of Bihar but
 14

geographically spread over South West, North West and North Bihar
respectively. Two comparable districts on the index were selected as control –
Gaya to examine any variations due to a higher concentration of marginalized
castes and Muzaffarpur for its historical legacy of having a more developed
system of school education compared to other parts of Bihar . In each of these 5
districts, 5 schools/learning centres were selected based on simple random
sampling. It was estimated that around 25 schools/learning centres with an
average of 6785 students and 81 teachers would be covered through this sample
design based on enrolment records. The final survey covered 3960 students and
54 teachers which is an indication of percentage of attendance.

5. Results/output: The analysis of the survey data will lead to final results on
quantitative and qualitative indicators for each set of factors outlined (in the
sub-section ‘Designing Variables’) calculated for cohorts defined by gender and
social background to arrive at overall learning achievement for students. It will
also outline a similar set of indicators for the contribution of each of factors
listed under process variables for measuring effectiveness of teachers in
equipping them to integrate CAL in the curriculum and pedagogy. These results
have been analysed for all three models of delivery enabling comparison
between the models and across districts.

V Results
VA – School Level Analysis
1. Identification of hard spots in learning: One of the crucial aims of the survey was
to identify the subjects which students find difficult. This has been based on a
perception index along with a verification of inconsistent responses through a
matching of performance (examination scores).

Table 5.1: Subject wise hard spots

 15

Subjects Number Percentage Percentage of Percentage Percentage Percentage

of of Sample Students of students of Boys of Girls
Students having having among Among
having difficulty difficulty in those who those who
Difficulty only in one other have have
in subject or difficulties difficulties
more in in
along with
1-Hindi 92 2 54 46 54 46
2-Urdu 150 4 82 18 54 46
3-Eng 1026 26 75 25 42 58
4-Sans 2070 52 88 12 46 54
5-
Mathematics 388 10 76 24 35 65
6-Science 149 4 63 37 49 51
7- Other 77 2 … … 55 45

More than 82 percent students have a significant difficulty in one subject and in most
cases it is either English or Sanskrit which together account for 78 percent of hard spots.
Also, in English and Sanskrit, the gender gap in learning is significant (Table 5.1 above
and Table 2.1 in Appendix II). Thus by middle school itself, the gender disparities within
pedagogical approaches in school are leading to gender gaps in knowledge acquisition
and learning achievements (these have been noted in the literature for a discussion, see
Aslam and Kingdon 2011). The gendered pattern in mathematics accessibility in middle
school can be argued to be a clear predictor of gender-biased specializations in later life
(majority of boys will take up science and commerce as specializations at high school
and college while majority of girls will take up arts and humanities). Studies based on
NCERT data does not show any significant gap at Standard V though it does show overall
 16

low levels of achievement in both languages and mathematics at the primary school
level (World Bank 2009), but we find that in the next three years of middle school
(Standard VI-VIII), the gender gap turns significant in the case of Bihar.
2. Role of e-samarth in addressing the hard spots:

Table 5.2: Information and Perception gaps on Role of e-samarth

District level State BEP ILFS
authorities officials

Perception/ School
Performance Authority Teachers
Increased Interest No Presumed No
in learning 88 76 information information
Increase in No No No
attention span 76 65 information information information
Increase in No Presumed No
classroom information information
participation 88 48
Increase in No No No
classroom information information information
interaction 68 63
Correct No No No
answers/response 72 46 information information information
More clarity on Presumed Presumed Presumed
topics taught
through CDS 60 39
No No No
Improved information information information
examination
performance 64 44
Improved Presumed Presumed Presumed
understanding of
the subject 56 41
Increase in No No Presumed
enrolment information information
(students
changing schools) 15
 17

Table 5.3: Status of Trained Teachers in e-samarth

Trained
Trained Outside/
under self
CAL trained
Trained Teachers 85 15
Training Hours
Not sure 7
15 hours 7
25 hours 4
30 hours 54
35 hours 13
40 hours 2
50 hours 11
126 hours 2
Usage of Computer (Days in a week)
7 11

6 20

5 9
4 13
3 4
2 9
1 2
Sometimes 22
Never 11
Usage of Computer/Kyan
(computer aid) for Teaching
Yes 43
No 57
Note: All figures are in percentages

While school authorities held a very optimistic view on the perceived impact of e-
samarth, teachers’ perception was significantly different (Table 5.2). The perception of
district and state level officials and the personnel of the private partner were largely
presumptions with little or no information on the qualitative indicators of effectiveness
of the programme in positively intervening in the learning process. After the
 18

completion of the training, 57 percent of trained teachers were not using computer aids
for teaching (Table 5.3).

3. Perception-Performance Matching
Parallel analysis of examination scores of last two years have not been possible for most
schools due to unavailability of records of scores for the same batch. Based on the
comparable individual scores of students in five schools, Table 5.4 below shows that
subject-wise significant changes in exam performance vary from school to school. We
identify statistically significant change in exam performance using ANOVA techniques on
subject-wise exam scores distributions for same group of students in two consecutive
years before and after the implementation of e-samarth. This is in keeping with the
patterns reported in other studies (Manchin et al op.cit).

Table 5.4 Changes in Performance in Examination in 2011 compared to 2010

English Hindi Sanskrit Mathematics Science
School 1 Significant Significant Significant Significant Significant
Change Change Change Change Change
School 2 No Significant Significant No No
significant Change Change significant significant
change change change
School 3 No Significant No Significant Significant
significant Change significant Change Change
change change
School 4 No No No No No
significant significant significant significant significant
change change change change change
School 5 Significant Significant Significant Significant Significant
Change Change Change Change Change
Note: Based on results of Paired T test of examination scores of students in 2009 -10 and 2010-11

Here we present two extreme results from the analysis of scores from the exam score
records of five schools. In school 4, in which the paired T test showed no change in
performance in all subjects, the distribution of exam scores for the batch in the school
presented below shows that the top percentile show similar patterns of improved
 19

grades in English, Mathematics and Hindi while the bottom percentiles show similar
patterns of worsening. Thus unimodal distributions of scores in English, Mathematics
and Hindi have turned bimodal within a year which could be a dangerous trend if left
unchecked.
A similar pattern can be seen for the scores pooled across schools in Hindi for both male
and female students (Appendix III).

School 4
Fig 5.1A

Fig 5.1B
 20

Fig 5.1C

The impact of CAL on performance cannot be isolated out from these patterns as factors
like change of teacher teaching the subject as students move up from one grade to
another, difference in syllabi etc could also be causal factors as the before/after CAL
period coincides with two different years of schooling in different grades/classes.
However, the uniform patterns in all three subjects suggest something systemic in the
school environment which we have investigated further. Sixty percent of responses in
the students’ questionnaire based interviews revealed that teachers running the CAL
programme in the school offer private lessons (tuition/coaching) outside school hours.
Students who avail these tuitions constitute 80 percent of the top half in the class while
those who do not are crowded into the bottom half. 90 percent students in the bottom
half cannot afford the private lessons. Those who are availing the private lessons are
favoured inside the school in terms of learning access. In the case of CAL, it is only this
group of students who have access to the computer aided lessons in terms of access
 21

time and teacher supervision. This is the source of the systemic bias in the school
environment. This systemic bias has an underlying caste-bias evident from Table 5.5
below. In 2009-10, 11.1 percent of SC students in this school were part of the top half of
the class. In 2011-12, all SC students were in the bottom half of the bimodal distribution.
A similar pattern for 22.2 percent EBC students can be observed. It must be noted that
caste-discrimination in this case is not explicit but plays out in the caste-class
correspondence with respect to access to private tuitions and thus linked to issues of
access to the market for private tuitions in a context in which the market agents are
also the key delivery personnel (teachers) in the government school system.
Table 5.5: Caste-wise Percentage Distribution of Students According to Examination
Performance
Row No Caste Year 35-50 51-65 66-80 >80
1 General 2009-10 28.1 28.1 25.0 18.8
2 Backward 2009-10 26.7 33.3 33.3 6.7
3 Extremely
Backward 2009-10 55.6 22.2 0.0 22.2
4 Scheduled
Caste 2009-10 66.7 22.2 7.4 3.7
5 General 2010-11 23.8 33.3 23.8 19.0
6 Backward 2010-11 23.8 33.3 23.8 19.0
7 Extremely
Backward 2010-11 66.7 33.3 0.0 0.0
8 Scheduled
Caste 2010-11 100.0 0.0 0.0 0.0
Note: The i dentical figures in Rows 5 a nd 6 a re due to smal l s a mpl e s i ze. Sma l l va ri a ti ons a re there i n the thi rd
deci ma l whi ch i s not evi dent here due to roundi ng off.

Performance and perception of students in school 4 were highly correlated. While the
top 30 percent of students felt that CAL had helped them to understand lessons better
and thus helped in examination performance the rest of the students complained that
they did not get access to computer aided lessons in English, Mathematics and Hindi.
Very large proportion of students who scored below 65 percent felt that the CAL
sessions were useful for them. Students’ performance in the examinations and their
perception about the benefits of CAL is highly correlated confirming one of the initial
premises of our exercise (Table 5.6).
 22

Table 5.6: Perception- Performance Matching of Students in School 4

English Mathematics Hindi
Rank Correlation 0.81 0.79 0.91
coefficient
(Usefulness of CAL
(perception),
Examination Score
(performance)
Rank Correlation 0.87 0. 82 0.94
coefficient
(Usefulness of CAL
(perception –
negative),
Examination Score
(performance <65)
Rank Correlation 0. 78 0.71 0.86
coefficient
(Usefulness of CAL
(perception –
positive), Examination
Score (performance
>65)
 23

Table 5.7: School 5: Computer Aided Learning Access

Student Student
Access to Access to
ICT e-
Average outside samarth
Student Teacher actual Student school tools in
Attendance attendance Class Computer (%) school
(%) (%) size aid Ratio (%)
59.2 88.9 108 108: 1 4.3 21

On the other extreme, we have school 5 in our sample in which the significant change in
all subjects reported in Table 5.4 above has been uniformly negative for both male and
female students illustrated in Figures 5.2A to 5.2 J, once again suggesting a sys temic
problem in the school. Also, only three subjects English, Hindi and Mathematics are
being taught with computer aids. However, there is no difference in the outcomes on
performance among the subjects being taught with computer aids and those without
(Sanskrit and Science). This systemic problem was not evident from the student
responses. But the interviews with the teachers revealed tensions and conflicts among
the teachers in the school. In particular, one teacher whose computer skills are minimal
has dominated not only the running of the e-samarth programme but also ensured that
the more skilled and trained teachers cannot and do not have any access to the use of
the programme. The teacher’s role in creating an obstructive power structure in the
school has not been confined to e-samarth but also overall delivery of the different
subjects taught in the school. Two of the other teachers who divulged this information
requested anonymity for fear of retaliation. The interview with the teacher concerned
revealed a very contemptuous assessment of the other teachers and the principal of the
school. In particular, the teacher expressed doubts about the teaching abilities of
colleagues in the school. The headmaster denied any conflict but admitted that there
are ‘tensions’ between teachers. This finding is also similar to patterns observed in the
 24

literature which attributes such findings to causal connections to specificities in the

school environment (see Section 1).
But in Bihar, School 5 would rank as one of the best schools in terms of overall school
functioning using standard parameters like average attendance, and more relevant to
our study – regular functioning of e-samarth. However, there have been long periods in
which there have been no classes. Students could not explain why classes were
irregular. But the teachers attributed it to ‘problems’ arising from conflicts over class
schedules and distribution of workload. Both district level functionaries and ILFS
personnel, separately interviewed, when asked to name a school which in their
perception has operationalised CAL very well chose School 5. This also matched with the
field team’s observations that computer aided teaching being delivered in the school in
three subjects.

However, even in this school where delivery has been obstructed due to conflicts among
teachers, student perceptions matched with student computer ratios suggests that it is
students’ access to the computer aided learning tools which is vital to effective learning.

81 percent of student respondents surveyed across schools in response to the open

ended question on their observations on e-samarth stated that more computers are
needed in the school. The infrastructure design for the programme has been conceived
on the basis of conceptualizing e-samarth as a computer aided teaching programme as
opposed to computer-aided learning. For a computer-aided learning programme,
resource planning has to take on board social and physical outreach to students and
ensure their access to e-learning tools. This is a very basic lacuna in the current design of
e-samarth.
 25

School 5

Fig 5.2 A

Fig 5.2 B
 26

Fig 5.2 C

Fig 5.2 D
 27

Fig 5.2 E
 28

Fig 5.2 F

Fig 5.2 G
 29

Fig 5.2 H
 30

Fig 5.2 I

Fig 5.2 J

VB: Comparing outcomes across districts and delivery models

Having explored the physical and social specificity of quality issues at the school
level in the earlier section, this concluding section presents an analysis of
outcomes on knowledge (K), skills (S) and attitudes (A) of students. Each of these
has been measured on a scale of 1-5 where:
 31

1 – Much Below Average

2 – Below Average

3 – Average

4 – Above Average

5 – Much above Average

In this design, the variables K, S and A are a mix of performance and perception
variables. K has been measured on the basis of examination scores stated by
students in the survey after cross verification with the sample scores collected
from school authorities wherever it has been available. S has been measured
with pooling of two indicators: to what extent the student can use computer aids
for self or group learning; and the number of ‘difficult’ subjects stated by
students (indicator of learning skills). A has been measured by pooling two
indicators: whether the student likes classes taught with computer aids and to
what extent the student perceives computer aids as necessary to the learning
process.

The sample under consideration consists of students who are in schools in which
CAL is operational (Appendix II, Tables 2.2A and 2.2B).

Table 5.8 and 5.9 present the results of the mapping exercise of knowledge, skills
and attitude of students across districts and delivery models
 32

Table 5.8 : District level variations in percentage distribution of students classified by

outcome variables

Variables Scale Saran Samastipur Bhojpur Muzaffarpur Gaya Analysis of

(% of (% of (% of (% of (% of Variance Between
Students) Students) Students) Students) Students) Districts (F test at
5% significance
level)
Knowledge 1 20 22 23 10 28 There is significant
2 16 21 25 24 21 variation in
3 40 35 28 25 33 distribution of
4 10 14 16 25 12 students on the
5 14 8 8 17 6 knowledge scale
across districts
Skills 1 22 8 23 23 40 There is significant
2 46 14 56 49 8 variation in
3 1 14 7 4 5 distribution of
4 46 42 3 9 35 students on the
5 28 22 11 16 13 Skills scale across
Attitude 1 4 1 1 1 3 There is no
2 2 1 1 1 3 significant
3 1 1 1 1 1 variation in
4 2 1 1 1 3 distribution of
5 91 96 96 96 90 students on the
attitude scale
across districts

The distribution of students on the knowledge and skills scale significantly varies from
district to district but attitudes of students to CAL is uniform across districts. The
qualitative findings of our survey corroborate this result. More than 95 percent of
students were very positive towards CAL and the prime reason for their liking for
computer aided lessons was that they liked studying with the use of images (computer
graphics/animation). They felt that they enjoy and remember the lessons better when
taught through images. However, knowledge and skill levels showed much more
variations across districts. Also, there are no significant patterns of variation between
the students in Saran, Samastipur and Bhojpur and those in our control districts
Muzaffarpur and Gaya with respect to knowledge and skills (Appendix IV). Thus the
historical legacy of developed educational institutions in Muzaffarpur or the higher
 33

demographic concentration of oppressed castes in Gaya has no correlation with

students’ knowledge and skill levels.

The difference in outreach of the delivery models is significant. Model 3 (ILFS) has been
able to ensure that around 66 percent of regularly attending students are able to
attend lessons taught through computer aids. The corresponding figure for Model 1
(BEP) is 3 percent while for Model 2 (BOOT), it is 17 percent. Thus the ILFS model has
been successful in increasing the spread of students attending computer aided teaching
sessions in school. However, examination of the variations in distribution of knowledge
and skills across delivery models reveal that there are no differences in distribution of
knowledge and skill patterns of students across the three models of delivery namely,
BEP, BOOT and ILFS (Table 5.9). So while Model 3 has been successful in increase
outreach of computer aided teaching sessions session, it has not made any significant
difference to quality of learning compared to the other to models.

Table 5.9 : Delivery model wise percentage distribution of students classified by

outcome variables

Analysis of
Variance
Between
Delivery
models (F
test at 5%
Outcome significance
Variables Models 1 BEP 2 BOOT 3 ILFS level)
1 1.0 3.6 0.2 There is no
2 6.5 3.4 1.1 significant
difference
3 66.1 84.3 88.2
in
4 19.8 6.4 8.2
distribution
of students
on the
knowledge
scale
across
delivery
Knowledge 5 6.7 2.3 2.3 models
Skills 1 83.6 69.4 75.5 There is no
 34

2 9.3 18.5 14.7 significant

3 4.0 8.0 5.8 difference
4 1.3 2.1 2.1 in
distribution
of students
on the
skills scale
across
delivery
5 1.7 1.9 1.9 models
Percentage
share of
students
covered by
the
delivery
model who
Outreach
are
attending
Computer
Aided
Teaching
classes in
school 2.9 17.4 65.8

Table 5.10: Correlation Matrix of Outcome and Process Variables in Learning Effectiveness

Process Access to Project Work Access to Attendance of Ability to

Computer using Computers Computer run the e-
aided in Computers in Outside Aided Lessons samarth
Outcome Schools School School in school CDs
containing
lessons
Skill level 0.91 0.90 0.60 -0.03 -0.16
Knowledge 0.63 0.81 0.23 -0.24 0.01
level

Analysis of the relation between outcome and process variables reveal that s kill level is
highly correlated with students’ access to computer and project work done on
computers in school. Knowledge is also significantly correlated with project work done
with computer aids. Access to computers outside school, attendance of computer aided
 35

teaching lessons and ability to run the e-samarth CDs have no significant relationship
with knowledge and skill levels (Table 5.10).

Thus students’ access to computer aids and project work with computer aids are crucial
to levels of skill and knowledge acquired by the student. These two factors have been of
least importance in the existing programme design of e-samarth, while the
conceptualisation of the programme as a computer aided teaching delivery system has
led to emphasis on the teaching (through CDs in the current phase of the programme)
which have not made any significant impact on either skill or knowledge levels given the
larger constraints of the social impediments to learning effectiveness in schools in Bihar.

Based on these survey results in Bihar, we present the results of our comparative review
of CAL delivery systems in Bihar, Karnataka and Kerala focusing on design, process and
implementation issues in a policy brief for Government of Bihar which is annexed to this
paper. The policy brief provides inputs for possible and feasible changes to the
institutional structure of public delivery of e-samarth in Bihar to address the outcomes
reported in this paper.
References and Bibliography

o Aslam M and G Kingdon, 2011, What can Teachers do to Raise Pupil Achievement?,
Economics of Education Review, Elsevier, vol. 30(3), June.

o Banerjee A, Cole S, Duflo E and L Linden, 2003, Improving the Quality of Education in India:
Evidence from Three Randomized Experiments, March,
www.cepr.org/meets/wkcn/3/3515/papers/duflo.pdf

o Banerjee A V and E Duflo, 2011, Poor Economics: A Radical Rethinking of the Way to Fight
Global Poverty, PublicAffairs, New York

o Barret, A M, 2009, The Education Millennium Development Goals Beyond 2015, Prospects for
Quality and Learners, Edqual Working Paper No. 13, University of Bristol

o Bowles S and H Gintis, 1976, Schooling in Capitalist America: Educational Reform and the
Contradictions of Economic Life, Basic Books
 36

o Cashin, W E, 1995, Student Ratings of Teaching: The Research Revisited, IDEA Paper No 32,
Manhattan, Kansas State University

o Das Gupta C, 2010 ‘The Effectiveness of ‘Targeting’ in Bihar’, Proceedings of International

Seminar on ‘Poverty in Bihar: Patterns, Dimensions and Eradication Strategies’, Patna,
Published by Institute of Human Development, Delhi, with ADRI Patna and UNDP

o EFA Global Monitoring Reports (Various Years), UNESCO, www.unesco.org

o Feldman, K A, 1976, Grades and College Students’ Evaluation of their Courses and Teachers ,
Research in Higher Education, Vol 4.

o GESCI, 2007, Towards a National Policy on ICT in School Education in India – A Multi
Stakeholder Perspective – A collaborative effort of The Department of School Education &
Literacy (D/SE&L), Ministry of Human Resource Development (MHRD), Government of
India, along with Global e-Schools and Communities Initiative, Centre for Science,
Development and Media Studies (CSDMS), 2007, www.csdms.in

o Goldstein H, 2004, Education for all: the globalization of learning targets, Comparative
Education 40 (1), February

o Gurumurthy K, 2009, Computer Learning Programs in Schools: Moving from BOOT Models to
an Integrated Approach, Perspective Paper on ICTs in Education , IT for Change,
www.itforchange.net

o Gurumurthy K and K Vishwanath, 2010, ICTs programmes in schools in Yadgir district, IT for
Change Case Study, www.itforchange.net

o Hariharan V, 2010, Open Source: Modernizing India’s Education System,

http://www.opensource.com

o Kremer M and A Holla, 2008, Pricing and Access: Lessons from Randomized Evaluations in
Education and Health,
http://www.economics.harvard.edu/faculty/kremer/files/Pricing%20and%20Access_080803
.pdf

o Linden L, Banerjee A, and E Duflo, 2003, Computer-Assisted Learning: Evidence from a

Randomized Experiment, Poverty Action Lab Paper No. 5, October, http://econ-
www.mit.edu/files/804

o Machin S, McNally S and O Silva, New Technology in Schools: Is There a Payoff?, 2006, IZA DP
No. 2234, Institute for the Study of Labor, July.
 37

o Moody D L and G Sindre, 2002, Evaluating the Effectiveness of Learning Interventions: An

Information Systems Case Study, http://is2.lse.ac.uk/asp/aspecis/20030097.pdf

o OECD, 2001, A case study of ICT-enriched school environment at Cramim School, Rishon Le-
Zion, Israel, http://www.oecd.org/dataoecd/37/21/2738488.pdf

o SACMEQ III, 2010, (Southern and Eastern Africa Consortium for Monitoring Educational
Quality) Survey Reports, www.sacmeq.org

o Sreekumar T T and M R Sánchez, 2008 ICTs and Development: Revisiting the Asian
Experience, Science Technology & Society (13)

o Semenov, A, 2005. Information and communication technologies in schools: A handbook for

teachers or How ICT can create new, open learning environments, UNESCO.
http://unesdoc.unesco.org/images/0013/001390/139028e.pdf.

o T A PAI Management Institute, 2010, IT@School – 'Excellence & Quality in Education - No

Exceptions - No Excuses', Impact Study Report, https://www.itschool.gov.in

o Taylor, J. E. and Ku, H.-Y. (2011), Measuring active learning to predict course quality,
Performance Improvement Quarterly, 24

o Tikley L, Towards a framework for understanding the quality of education, 2010, EdQual
working paper No.27, University of Bristol, November.

o Vikramshila Education Resource Society, 2009 Information and Communication Technology

for Quality Education, Shikshak Sammelan Kolkata, www.vikramshila.org

o Vidya Bhawan Society & Azim Premji Foundation, 2008, A Study of the Computer Assisted
Learning Program (CALP), www.azimpremjifoundation.org

o World Bank, 2007, MDG Global Monitoring Report, www.worldbank.org

o World Bank, 2009, Secondary Education in India: Universalizing Opportunity, Human

Development Unit, South Asia Region

------
38