See

discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/242099018

Assessing the Quality of Democracy: Freedom

Competitiveness and Participation in Eighteen
Latin American Countries

Article in Democratization · September 2010

DOI: 10.1080/714000256

CITATIONS READS

100 124

2 authors, including:

Anibal Perez-Linan
University of Pittsburgh
105 PUBLICATIONS 941 CITATIONS

SEE PROFILE

All in-text references underlined in blue are linked to publications on ResearchGate, Available from: Anibal Perez-Linan
letting you access and read them immediately. Retrieved on: 10 August 2016
92dem05.qxd 02/05/02 12:00 Page 85

Assessing the Quality of Democracy:

Freedom, Competitiveness and Participation
in Eighteen Latin American Countries

D AV I D A LT M A N and A N Í B A L P É R E Z - L I Ñ Á N

This article explores the problem of conceptualizing and measuring the quality of
democracy in Latin America. The first part discusses the use of the concept and the
need for an operational measure. It explores three dimensions of the quality of
democracy: civil rights, participation and effective competition. The second part
develops an indicator of effective competition, one of the key dimensions of the
concept. The third part analyses the empirical relationship between all three
dimensions in 18 Latin American countries between 1978 and 1996. The study
constructs summary measures of the quality of democracy in several ways, and show
that the ranking of the cases is highly consistent no matter the procedure applied. The
final section tests the validity of the measure and discusses its limitations.

Introduction
After the third wave of democratization, students of comparative politics
have noticed a decreasing range of variance in their favourite dependent
variable: the political regime. Explaining the conditions for the emergence,
breakdown or survival of different regimes has been a classic goal of
comparative studies. Over the last decade, however, political democracy has
survived in many countries – meaning that the dependent variable has
shown no significant change. This situation has directed scholars towards
new and more subtle questions about preconditions for democratic
consolidation and the institutional features of new democracies. Moreover,
it is breeding a growing interest in the quality of democratic life, a factor
that clearly varies from country to country.

David Altman, Guest Research Assistant Professor at the Helen Kellogg Institute for
International Studies, University of Notre Dame and Aníbal Pérez-Liñán, Assistant Professor in
the Department of Political Science, University of Pittsburgh. The authors are indebted to Ana
María Bejarano, Daniel Brinks, Daniel Buquet, Miguel Centellas, Rossana Castiglioni, David
Collier, Michael Coppedge, Robert Fishman, Fito Garcé, Kim Quaile Hill, Nestor Legnani, Scott
Mainwaring, Gerardo Munck, María Rosa Olivera-Williams and Benjamin Radcliff for their
valuable comments. They also thank the Quality of Democracy Working Group of the Helen
Kellogg Institute for International Studies at the University of Notre Dame for its support.
Replication datasets for this study are available upon request.
Democratization, Vol.9, No.2, Summer 2002, pp.85–100
PUBLISHED BY FRANK CASS, LONDON
92dem05.qxd 02/05/02 12:00 Page 86

86 D E M O C R AT I Z AT I O N

This article explores the problem of conceptualizing and measuring the

quality of democracy in Latin America. The first part discusses the use of
the concept and the need for an operational measure. It begins by noting an
important distinction among polyarchies: even though all of them allow
(quasi) universal participation and legal opposition to the ruling party,
effective participation and competition vary from country to country. This
suggests that countries with similar levels of democratization might take
advantage of their democratic institutions to different degrees. Following
this observation, three dimensions of the quality of democracy are explored:
civil rights, participation and effective competition. Following Hill, these
three dimensions are conceived as the extension of Robert Dahl’s concept
of polyarchy.1 The second part then develops an indicator of effective
competition, one of the key dimensions in the study. The third part analyses
the empirical relationship between all three dimensions in 18 Latin
American countries between 1978 and 1996. Summary measures of the
quality of democracy are constructed in several ways, to show that the
ranking of the cases is highly consistent no matter the procedure applied.
The last section tests the validity of the measure and discusses its
limitations.

Approaching the Quality of Democracy

By quality of democracy is meant the extent to which any given polyarchy
actualizes its potential as a political regime.2 The assumption underlying
this view is that polyarchy is a necessary, yet not a sufficient, condition for
a high quality of democracy. Most students of democracy would probably
agree with this basic definition, if only because it is broad enough to
accommodate several perspectives on this issue.
Recent studies of democratization have increasingly dealt with this
question. Some authors have approached the topic as an extension of the
classic focus on regime change. A good deal of research has been done in
order to measure levels of democracy allowing scholars to trace fine
distinctions among the cases placed at the top of the scale.3 For instance,
Diamond and Coppedge conceived the quality of democracy as the relative
degree of democratization among countries that we already label as
polyarchies.4
The value of this perspective cannot be denied but there are two
potential problems. The first one is that instruments designed to grade
regimes in a wide range between full authoritarianism and full democracy
might lack sensitivity to discriminate within the pool of polyarchies
clustered at one extreme of the range. Second, the relevant criteria to
distinguish between authoritarianism and democracy are not necessarily the
92dem05.qxd 02/05/02 12:00 Page 87

ASSESSING THE QUALITY OF DEMOCRACY 87

same as the relevant ones to discriminate among polyarchies. For instance,
marginal improvements in political rights and civil liberties might be
relevant but do not constitute the only key to the quality of democratic life.
There is a substantial difference between addressing the quality of
democracy and the level of democratization of a political regime. Every
analysis of the quality of democracy should assume a minimum degree of
democratization, namely Dahl’s procedural minimum. When we compare
the quality of democracy among countries we are not comparing which
countries are more democratic (in the sense that Freedom House scores or
the Polity Index measures the level of democracy as opposed to
authoritarianism). Rather, we are analysing in which countries democracy
performs better given some normative standards. Much of the debate about
the quality of democracy is about the identification of these normative
standards.
For instance a second approach to the quality of democracy has
emphasized, often from a qualitative perspective, substantive flaws that
negatively affect democratic life in a given country or set of countries.5 To
deal with these cases of ‘reserved domains’, lack of ‘horizontal
accountability’, or ‘electoralism’, among other problems, scholars have
developed a whole array of diminished sub-types of democracy.6 This
perspective has been extremely lucid in identifying challenges for (and
flaws of) new polyarchies, but it has usually avoided a comprehensive
definition of the quality of democracy and has tended to ignore problems of
cross-national measurement.
In our view, some of the most interesting studies of the quality of
democracy have been done at the local level.7 Putnam identified the quality
of democracy with institutional performance – understood as some
objective measure of governmental responsiveness and output levels.8 Some
of Putnam’s indicators, however, are related to the performance of local
government per se, not necessarily local democratic government (even
though his universe of study was democratic). In addition, we suspect that
his indicators are too tailored to the Italian case (at the local level), and
would not travel well to Latin America (at the national level).
An operational definition of the quality of democracy anchored in Dahl’s
definition of polyarchy is employed here. It allows us to assess to what
extent different polyarchies transform legal opportunities for participation
and contestation into tangible patterns of citizen behaviour. Democracy
creates the potential for citizen participation and opposition to elected
officials, but in many countries citizen apathy or weak party competition,
among other possible reasons, hinder the development of this potential.
The approach adopted is similar to Hill’s study of the United States. Hill
proposed
92dem05.qxd 02/05/02 12:00 Page 88

88 D E M O C R AT I Z AT I O N

an empirical assessment of the extent of representative democracy in

the [American] states. That assessment is guided by empirical
democratic theory and its explication of the essential components of
such a governmental system: equal rights to vote in free and fair
elections, competitiveness among political parties contending to
control government through those elections, and the degree of mass
participation in elections.9

Because these three dimensions were derived from Dahl’s concept of

polyarchy, this approach is particularly useful for the purpose of cross-
national comparison.10
Following Dahl and Hill, therefore, the three dimensions of the quality
of democracy in Latin America can be summarized as:11

Effective civil rights: Dahl’s definition of polyarchy hinges on a set of

institutional conditions allowing mass participation and free opposition to
the ruling elite. The lack of such conditions (what we call effective rights)
determines the absence of polyarchy. But even if all conditions are present
to a good extent (making the country a member of the polyarchic set of
regimes) limited violations of civil rights may hinder the quality of
democracy. Countries in which some specific regions or social groups are
affected by political violence or electoral manipulation are clearly worse
than democracies in which the whole population effectively exercises its
rights – of course, no country has a perfect record. Freedom House scores
are used as an indicator of this dimension: the 2–14 scale was normalized to
range between 0 and 1 in order to facilitate comparison with the other
dimensions.12 Since all cases in our sample are polyarchies, the average
score is high: 0.73, with a standard deviation of 0.15.

Effective Participation: Dahl’s measure of participation reflected the right

to participate, not the actual rate of participation. Most scholars contend
(correctly) that voter turnout should not be part of a definition of democracy.
But many others have argued (also correctly, in our view), that turnout is an
important dimension of the quality of democratic life.13 Greater participation
– whether it is voluntary or encouraged by compulsory vote – makes
democratic governments responsive to a larger share of the population.14
The health of a democratic regime is particularly weak when some citizens
are effectively disenfranchised as a consequence of poverty, lack of basic
education or sheer apathy.15 Because low turnout in Latin America is
typically related to low levels of voter registration this study measures
turnout as the number of voters over the voting-age population (VAT).16
Data were gathered from a single source.17 In Latin America, turnout varies
92dem05.qxd 02/05/02 12:00 Page 89

ASSESSING THE QUALITY OF DEMOCRACY 89

from extremely low (15 per cent in 1994 Guatemala) to very high rates (95
per cent in 1989 Uruguay). The average for our sample was a VAT of 0.62,
with a standard deviation of 0.18.18

Effective Competition: Dahl’s definition of polyarchy allowed for the free

exercise of political contestation in – and between – elections, but it never
implied that effective competition had to occur. For instance, Japan between
1955 and 1993 is a classic example of democracy with low inter-party
competition. Nevertheless Hill is correct to argue that a more competitive
democracy is a better democracy.19 An indicator of competition useful for
our purposes must fulfil three requirements. First it should reflect the
opposition’s access to the legislative process, rather than mere electoral
outcomes (which can be distorted through disproportionality or fraud).
Second it should punish the excessive dominance of the ruling party in
policy making, but, third, without rewarding excessive dominance of the
opposition (which may create serious problems of governability). For
different reasons that are explained in the following section, traditional
measures of party competition do not serve our purpose. We therefore
develop an original index for cross-national comparison.

Assessing Effective Competition

Students of democracy and elections have developed different measures of
competition. For example, Powell measured competition as the frequency
of alternation in power over a 19-year period, and Ranney built a
multidimensional index of competition in the American states over several
decades.20 This long-term perspective is not very useful for new
democracies in which only a very few elections may have taken place.
Other students have measured competition as the winner’s percentage of the
votes, the per cent margin of victory, and the raw vote margin of victory in
elections.21 Such measures are closer to our purposes, but they are heavily
biased against two-party systems because margins of victory tend to be
smaller in multiparty democracies.
We measure the opposition’s access to power as a weighted difference
between the share of the seats of the government and the opposition parties
in the lower chamber.22 Being aware that it is important to penalize
fragmentation, we designed a measure to find the ‘typical party’ in the
opposition by weighting the shares of seats in favour of the largest parties:

O=
∑o 2
i

∑o i
92dem05.qxd 02/05/02 12:00 Page 90

90 D E M O C R AT I Z AT I O N

O is an indicator of the leverage of the opposition, where oi is the share of

seats for the i-th opposition party. The same procedure is followed to
estimate the size of the ‘typical party in government’ in the case of electoral
coalitions winning office:

G=
∑g 2
i

∑g i

Based on our previous assessment of the size of the ‘typical parties’ in

government and in opposition, we developed an index of competitiveness:

G−O
C = 1−
100
The value of C tends to zero whenever the government (or the opposition)
controls the whole legislature, and to one if there is balance between
government and opposition.23 For example, C equals 0.332 for 1984
Nicaragua, and 0.998 for 1990 Bolivia. The performance of this indicator is
well illustrated by the cases of Ecuador (1988) and Venezuela (1979). In
both cases, the ruling party (Izquierda Democrática and Copei, respectively)
controlled 42.2 per cent of the seats. In Ecuador, however, the opposition
was highly fragmented, therefore C=0.656. In contrast, Copei had to face
the powerful Acción Democrática in Congress: C=0.745.24 The average
value of C in the sample is 0.80 with a standard deviation of 0.14.25

Measuring the Quality of Democracy

The previous sections have argued that effective civil rights, participation
and competition are three dimensions of the quality of democracy inferred
from Dahl’s concept of polyarchy. Are these three dimensions independent,
or do they all reflect a latent variable? If these variables are independent
from each other, we can only conceive the quality of democracy as a
multidimensional phenomenon. If, on the contrary, all dimensions reflect an
underlying construct, it might be possible to develop a summary measure of
the quality of democracy in Latin America.
This section addresses the problem using factor analysis. The sample
includes all Latin American countries that were polyarchies at some point
between 1978 and 1996. The decision to include some cases in the sample
is debatable because the boundaries between democratic and non-
democratic regimes are sometimes contested – and, of course, it would not
make sense to measure the quality of democracy in non-democracies. One
way of dealing with this problem is to consider polyarchy a concept with
92dem05.qxd 02/05/02 12:00 Page 91

ASSESSING THE QUALITY OF DEMOCRACY 91

uncertain boundaries, not because the concept is ill-defined but because
there is uncertainty about some cases belonging to the polyarchic set.26 If we
adopt a ‘fuzzy set’ approach, we can think of countries as being part of this
set with different degrees of probability.27 For instance, the probability of
Cuba belonging to the set of democracies is virtually zero, the probability
of Costa Rica, virtually one. The probability of Mexico being a member of
the set in the early 1990s is likely to be less than one. Acknowledging the
problem of uncertainty, we wanted to include in our sample all cases with a
high likelihood of being members of the polyarchic set.
The decisions were based on two categorical measures of democracy,
the ACLP classification, and Mainwaring’s classification.28 The sample
includes all cases coded as democracies or semi-democracies in
Mainwaring’s classification (that is, cases coded as democratic by Alvarez
et al. but not coded as authoritarian by Mainwaring).29 Eighteen countries
entered the sample at different time points (see Table 1 below). Cuba and
Haiti were not classified as democracies at any point during the 1978–96
period.
The chosen units of analysis are democracies-after-each-election, a total
of 77 observations were included in the analysis (see Appendix). For the
few cases in which presidential and legislative elections were held in
different years, turnout at the presidential or legislative election was
measured and the competitiveness of the system calculated on the basis of
the configuration of the new government or legislature.
To the extent that the three dimensions (civil rights, competitiveness and
turnout) reflect an underlying construct like the ‘quality of democracy’, we
would expect them to be partially correlated across cases – they all map the
same latent variable. Indeed, the Pearson correlation between Freedom
House scores and turnout is 0.58, between turnout and the C index is 0.29,
and between Freedom House and C is 0.49 (all significant at 0.01 level).
These are not strong correlations and therefore it cannot be said with
certainty that they reflect a single latent variable without factor analysis.
Following a factor analysis is used to create a summary measure of the
quality of democracy.30 Factor analysis is useful to validate an index by
demonstrating that its constituent items load on the same factor. According
to Marradi, ‘factor analysis allows one to use statistical relationships
between several lower-level variables as empirical evidence for or against
the establishment of a semantic relationship of indication between these
variables and an abstract concept, which may thus be measured and
transformed into a variable with a high semantic extension and theoretical
importance’.31 Factor scores for each country are presented in Table 1 below.
The three dimensions were aggregated by estimating the unweighted
average of their standardized scores. The values, presented in the column
92dem05.qxd 02/05/02 12:00 Page 92

92 D E M O C R AT I Z AT I O N

TABL E 1
Q U A L I T Y O F D E M O C R A C Y I N L AT I N A M E R I C A ( 1 9 7 8 – 1 9 9 6 )

Country Dimensions Factor Alternative procedures

FH Turnout C Scores Average Product Z-
Scores

Uruguay (1984–96) 0.861 0.935 0.898 1.385 0.898 0.722 1.108

Costa Rica (1978–96) 0.983 0.807 0.848 1.338 0.879 0.672 1.027
Chile (1990–96) 0.833 0.840 0.980 1.301 0.884 0.686 1.064
Brazil (1985–96) 0.771 0.758 0.908 0.726 0.812 0.527 0.599
Argentina (1983–96) 0.821 0.807 0.790 0.681 0.806 0.526 0.525
Venezuela (1978–96) 0.833 0.684 0.806 0.479 0.774 0.460 0.361
Ecuador (1979–96) 0.781 0.627 0.870 0.358 0.759 0.425 0.289
Honduras (1982–96) 0.708 0.672 0.879 0.259 0.753 0.425 0.231
Dominican Rep. 0.764 0.548 0.893 0.185 0.735 0.382 0.158
(1978–96)
Bolivia (1982–96) 0.750 0.563 0.862 0.092 0.725 0.363 0.081
Panama (1990–96) 0.708 0.629 0.761 -0.156 0.699 0.334 -0.130
Nicaragua (1984–96) 0.556 0.752 0.677 -0.575 0.662 0.313 -0.445
Peru (1980–92, 0.625 0.631 0.681 -0.625 0.646 0.271 -0.504
1995–96)
El Salvador (1984–96) 0.597 0.468 0.796 -0.776 0.620 0.219 -0.597
Paraguay (1989–96) 0.625 0.470 0.732 -0.853 0.609 0.209 -0.679
Colombia (1978–96) 0.681 0.348 0.693 -1.066 0.574 0.175 -0.875
Mexico (1988–96) 0.528 0.525 0.557 -1.499 0.537 0.154 -1.212
Guatemala (1986–96) 0.479 0.328 0.724 -1.654 0.510 0.120 -1.292

labelled ‘Z-scores’ are highly correlated with the values for the common
factor. The problem with those two measures is that they are dependent on
the cases included in the sample. If, for instance, we add or delete some
observations from the sample, the standardized scores may change and the
relative order of cases may be affected. Therefore two forms of aggregation
that are free of this problem were tested: the average value
([FH+C+VAT]/3) and the product (FH*C*VAT) of all three dimensions.
The results are extremely consistent regardless of the aggregation
procedure, the only exception being the ranking of Chile and Costa Rica
(Chile’s ranking is discussed in more detail in the conclusions below).
Those two countries change positions if countries are sorted by Average,
Product or Z-Scores. The rest of the countries remain in the same position
when sorted by Factor or by any other measure.
For graphic parsimony a map of Latin America is also included (see
Figure 1) in which each country is shaded according to its factor coefficient.
On average, countries in the southern cone have shown a better quality of
democracy than countries in Central America or northern South America.
But Costa Rica is of course the most noticeable exception to this pattern.
92dem05.qxd 02/05/02 12:00 Page 93

ASSESSING THE QUALITY OF DEMOCRACY 93

Not only has Costa Rica been democratic for more than half a century, it is
also a good democracy. It is probable that the ‘white’ countries – Colombia,
Guatemala, Mexico and Paraguay – are, or have been for most of the period
under study, borderline cases. In other terms, they might be located under
the ‘cross-over point’ where membership in the set of democracies is more
than uncertain.32

Concept Validity
Besides the issue of aggregation, another important problem in concept
building is validation. Simply put, validity implies that our measure reflects
(our definition of) the quality of democracy, not just a small part of the
concept or some other theoretical construct.33 This section therefore focuses
on criterion-related validity (the extent to which our measure correlates to

F I GURE 1
Q U A L I T Y O F D E M O C R A C Y I N L AT I N A M E R I C A

Factor scores:
1.3 to 1.39 (3)
0.48 to 1.3 (2)
0.09 to 0.48 (5)
-0.78 to 0.09 (4)
-1.66 to -0.78 (4)

No data
92dem05.qxd 02/05/02 12:00 Page 94

94 D E M O C R AT I Z AT I O N

other indicators) and construct validity (the ways in which our measure
relates to broader theoretical assumptions about the quality of democracy).
Of course this is only a modest, first step in the validation process. As
Hubley and Zumbo noted, ‘the validation process is a form of disciplined
inquiry in which plausible, alternative inferences from the test scores or
observations are disproved’.34
Criterion-related validation is difficult in this case because, to our
knowledge, no other measures of the quality of democratic life are available
for Latin America. There are, however, available indicators of democracy
like the Polity III, 20-point scale.35 We expect a positive correlation between
our indicator of democratic quality and the (upper range) of the Polity score.
In fact, the average Polity III scores for countries in the sample correlate at
0.70 with our measure (n=18). This correlation is not very surprising, since
Polity is itself correlated to Freedom House scores, our indicator of civil
rights. Because our sample includes some cases (that is, countries in given
years) that Mainwaring classified as ‘semi-democratic’ the mean factor
score for this group was investigated as well. Cases labelled as semi-
democracies (n=28) have an average factor score of –0.92, while the ones
coded as democratic (n=49) have a mean of +0.53.
For construct validation, what Adcock and Collier called the AHEM
(Assume the Hypothesis, Evaluate the Measure) validity test was
employed.36 This is also a difficult task because the literature on the quality
of democracy is at an early stage, and there are no well-established
hypotheses about the causes (and consequences) of a ‘good’ democracy.
Three ideas that we considered (almost) uncontroversial were selected: 1) a
stronger democratic tradition is correlated with a better democracy; 2)
political violence has a negative impact on democratic life, and 3) public
satisfaction with democracy is related to the quality of the regime (probably
in a bi-directional way).
As an indicator of ‘democratic tradition’ the number of years of
democracy enjoyed by each country in the second half of the century was
counted, that is for the period 1950–96, the last year in the sample.37 The
correlation between this indicator and factor score is 0.79 (n=18). We also
measured democratic tradition in the long run using the Polity III country
average between 1900 and 1977 (the last year prior to our period). The
correlation between this indicator and our index was 0.62. Countries with
strong guerrilla movements in the 1978–96 period (Colombia, El Salvador,
Guatemala, Nicaragua and Perú) have a mean score value of –0.94, while
the average for the rest of the countries is +0.33. The inclusion of Mexico
in the first group would only help confirm the test.
Satisfaction with democracy at the mass level was measured by the
Latinbarometer (1997 wave) in all countries except for the Dominican
92dem05.qxd 02/05/02 12:00 Page 95

ASSESSING THE QUALITY OF DEMOCRACY 95

Republic. Satisfaction was measured as the difference between the
percentage of respondents saying that they are ‘very’ or ‘somewhat’
satisfied with the regime, and the ones declaring some degree of
dissatisfaction. The correlation between this indicator and our score is 0.65
(n=17). Table 2 below presents the relation between the three factors
(tradition, violence and satisfaction) and the quality of democracy index.
The regression model predicts 80 per cent of the variance in the dependent
variable, and all coefficients are significant and of the expected sign.

TABL E 2
P R E D I C TO R S O F T H E Q U A L I T Y O F D E M O C R A C Y

b Beta t

Democratic tradition (1950–96) 0.034** 0.539 4.25

Political violence –0.641* –0.311 –2.53
Satisfaction with democracy 0.013* 0.336 2.71
Constant 0.095 0.342
Adjusted R2=0.802

** Significant at 0.001 level, * 0.05 level. n=17

Conclusions
The major question underlying any assessment of the quality of democracy
is how to discriminate between better and worse democratic units. The
number of normative criteria that could be used to evaluate democracies is,
however, virtually unlimited. This study selected three dimensions that are
well-grounded in democratic theory: effective liberties, competitiveness and
participation.
These criteria have proved to be instrumental for comparing
democracies at different levels of analysis. For instance, Hill compared the
US states at two historical moments, Centellas assessed the evolution of
democracy over time in one country (Bolivia), and Altman and Pérez-Liñán
dealt with a cross-section of countries world-wide.38 This article has
compared 18 Latin American countries during the ‘third wave’, between the
late 1970s and the mid 1990s.
Those three aspects are not enough to fully describe the complexities of
democratic life. For instance, our indicator is not sensitive to the presence of
‘reserved domains’ – as the ones Valenzuela had in mind when describing
Chilean politics.39 This explains why Chile ranks so high in our study. Neither
is the measure very sensitive to the presence of ‘brown areas’ like the degree
of independence of the judiciary, accountability or institutional performance.40
Those limitations show that we are far from settling the question of how
to measure the quality of democracy. Although our index performs
92dem05.qxd 02/05/02 12:00 Page 96

96 D E M O C R AT I Z AT I O N

satisfactorily well, fitting our theoretical expectations, we consider this

measure a very modest contribution to the incipient debate on this issue.
Future research will be directed at addressing three topics: the validity of
this measure, its reliability in different contexts, and the addition of new
dimensions to this idea of democratic quality.

NOTES

1. Kim Quaile Hill, Democracy in the Fifty States (Lincoln: University of Nebraska Press,
1994).
2. Robert Dahl, Polyarchy: Participation and Opposition (New Haven, CT: Yale University
Press, 1971).
3. For example: Phillip Cutright, ‘National Political Development: Measurement and
Analysis’, American Sociological Review, Vol.28, No.2 (1963), pp.253–64. Kenneth A.
Bollen, ‘Issues in the Comparative Measurement of Political Democracy’, American
Sociological Review, Vol.45, No.2 (1980), pp.370–90. Kenneth A. Bollen, ‘Liberal
Democracy: Validity and Sources Biases in Cross-National Measures’, American Journal of
Political Science, Vol.37, No.4 (1993), pp.1207–30. Tatu Vanhanen, The Emergence of
Democracy: A Comparative Study of 119 States, 1850–1979 (Helsinki: Societas Scientiarum
Fennica, 1984). Ted R. Gurr, Keith Jaggers and Will Moore, ‘The Transformation of the
Western State: The Growth of Democracy, Autocracy, and State Power since 1800’, Studies
in Comparative International Development, Vol.25, No.1 (1990), pp.73–108. Michael
Coppedge and Wolfgang Reinicke, ‘Measuring Polyarchy’, in A. Inkeles (ed.), On
Measuring Democracy: Its Consequences and Concomitants (New Brunswick, NJ:
Transaction Publishers, 1990), pp.47–68. Raymond D. Gastil, ‘The Comparative Survey of
Freedom: Experiences and Suggestions’, in A. Inkeles (ed.), On Measuring Democracy: its
Consequences and Concomitants (New Brunswick, NJ: Transaction Publishers, 1991),
pp.21–46. Axel Hadenius (ed.), Democracy and Development (Cambridge: Cambridge
University Press, 1992). Dean E. McHenry Jr., ‘Quantitative Measures on Democracy in
Africa: An Assessment’, Democratization, Vol.7, No.2 (2000), pp.168–85. Joe Foweraker
and Roman Krznaric, ‘Measuring Liberal Democratic Performance: an Empirical and
Conceptual Critique’, Political Studies, Vol.48, No.4 (2000), pp.759–87. Gerardo Munck and
Jay Verkuilen, ‘Measuring Democracy: Evaluating Alternative Indexes’, Paper presented at
the Annual Meeting of the Midwest Political Science Association (Chicago, 2000).
4. Larry Diamond, Developing Democracy: Toward Consolidation (Baltimore, MD: The Johns
Hopkins University Press, 1999). Michael Coppedge, ‘Modernization and Thresholds of
Democracy: Evidence for a Common Path and Process’, in M. Midlarsky (ed.), Inequality,
Democracy, and Economic Development (Cambridge: Cambridge University Press, 1997),
pp.177–201.
5. Samuel Valenzuela, ‘Democratic consolidation in Post-transitional settings: Notion, Process
and Facilitating Conditions’, in Guillermo O’Donnell, Scott Mainwaring and Samuel
Valenzuela (eds), Issues in Democratic Consolidation: The New South American
Democracies in Comparative Perspective (Notre Dame, IN: University of Notre Dame Press,
1992), pp.57–104. Guillermo O’Donnell, ‘On the State, Democratization and Some
Conceptual Problems: a Latin American View with Glances at Some Postcommunist
Countries’, World Development, Vol.21, No.8 (1993), pp.1355–69. Guillermo O’Donnell,
‘Delegative Democracy’, Journal of Democracy, Vol.5, No.1 (1994), pp.55–69. Guillermo
O’Donnell, ‘Illusions about Consolidation’, Journal of Democracy, Vol.7, No.2 (1996),
pp.34–51. Terry Lynn Karl, ‘The Hybrid Regimes in Central America’, Journal of
Democracy, Vol.6, No.3 (1995), pp.72–86.
6. David Collier and Steve Levitsky, ‘Democracy with Adjectives: Finding Conceptual Order
in Recent Comparative Research’, World Politics, Vol.49, No.3 (1997), pp.430–51.
Diamond, op. cit., Chapter 2.
92dem05.qxd 02/05/02 12:00 Page 97

ASSESSING THE QUALITY OF DEMOCRACY 97

7. William L. Miller, Irrelevant Elections? The Quality of Local Democracy in Britain (Oxford:
Clarendon Press, 1988).
8. Robert Putnam, Making Democracy Work: Civic Traditions in Modern Italy (Princeton, NJ:
Princeton University Press, 1993).
9. Kim Quaile Hill, Democracy in the Fifty States (Lincoln, NA: University of Nebraska Press,
1994), p.xii.
10. Ibid., pp.13–15.
11. Robert Dahl, op. cit.; Hill, op. cit.
12. See Miguel Centellas, ‘The Consolidation of Polyarchy in Bolivia, 1985–1997’, 57th Annual
Meeting of the Midwest Political Science Association (Chicago, 1999).
13. For instance E.E. Schattschneider, The Semisovereign People: A Realist’s View of
Democracy in America (New York: Holt, Rinehart and Winston, 1960); Olavo Brasil De
Lima, ‘Electoral Participation in Brazil (1945–1978): The Legislation, the Party Systems and
Electoral Turnouts’, Luso-Brazilian Review, Vol.20, No.1 (1983), pp.65–87; Ruy Texeira,
Why Americans don’t Vote: Turnout Decline in the United States, 1960–1984 (New York:
Greenwood Press, 1987); Miller, op. cit.; Hill, op. cit.; Arend Lijphart, ‘Unequal
Participation: Democracy’s Unresolved Dilemma’, American Political Science Review,
Vol.91, No.1(1997), pp.1–14.
14. Admittedly, this is a controversial issue. David Altman and Aníbal Pérez-Liñán explore this
problem in greater length in ‘Más allá de la poliarquía: midiendo la calidad de las
democracias’, Revista Uruguaya de Ciencia Política, Vol.11 (1999), pp.83–105.
15. Some analysts contend that low participation may reflect high satisfaction with the political
regime. We dispute this idea based on the fact that most studies have shown that the less
educated people (the ones with fewest opportunities in the system) are the less inclined to
vote. For instance see R. Wolfinger and S. Rosenstone, Who Votes? (New Haven, CT: Yale
University Press, 1980).
16. Centellas, op. cit., developed a subtler Index of Effective Participation (IEP) which accounts
for blank-null votes and votes for parties not represented in congress. We rely on the classic
VAT indicator due to lack of reliable information on ‘ineffective’ parties.
17. International IDEA, Voter Turnout from 1945 to 1997: A Global Report on Political
Participation (Stockholm: IDEA: Institute for Democracy and Electoral Assistance, 1997).
18. For the advantages (and some problems) of the VAT measure, see Hill, op. cit., pp.135–9. We
are aware that political participation is not limited to voting in elections. But casting ballots
is the basic form of participation, and the one that is equally relevant in all democracies.
Other forms of participation may be too idiosyncratic to facilitate comparison across
countries.
19. Ibid., Chapter 3.
20. G.B. Powell, ‘American Voter Turnout in Comparative Perspective’, American Political
Science Review, Vol.80, No.1 (1986), pp.17–43; Austin Ranney, ‘Parties in State Politics’, in
Herbert Jacob and K. Vines (eds), Politics in the American States (Boston: Little, Brown,
1965), pp.51–92. See also: Kim Q. Hill and Jan Leighler, ‘Party Ideology, Organization, and
Competitiveness as Mobilizing Forces in Gubernatorial Elections’, American Journal of
Political Science, Vol.37, No.4 (1993), pp.1158–78; Hill, Democracy in the Fifty States.
21. Samuel Patterson and Gregory Caldeira, ‘Getting Out the Vote: Participation in
Gubernatorial Elections’, American Political Science Review, Vol.77, No.3 (1983),
pp.675–89. Gary Cox and M. Munger, ‘Closeness, Expenditures and Turnout in the 1982
House Elections’, American Political Science Review, Vol.83, No.1 (1989), pp.217–31. Gary
Cox, ‘Closeness and Turnout: a Methodological Note’, Journal of Politics, Vol.50, No.3
(1988), pp.768–75.
22. We operationalize opposition as all parties with seats in the lower chamber that are not the
ruling party or that are not part of the electoral coalition that elected the president. By
government, on the other hand, we understand the party (or the coalition of parties)
controlling the executive as a result of a given election. By ‘government coalition’ we refer
to any set of parties that had formed an explicit electoral front with the ruling party, that is
parties that presented a single presidential candidate under a common label. We are aware
that the majority of government coalitions in Latin America are post-electoral, but post-
92dem05.qxd 02/05/02 12:00 Page 98

98 D E M O C R AT I Z AT I O N

electoral coalitions are not relevant for our measurement. The fact that the executive has to
negotiate the formation of a government coalition with the opposition parties shows the
presence of a competitive party system rather a non-competitive one. See David Altman,
‘Coalition Formation and Survival under Multiparty Presidential Democracies in Latin
America: Between the Tyranny of the Electoral Calendar, the Irony of Ideological
Polarization and Inertial Effects’, Latin American Studies Association (Miami, Florida,
2000).
23. The ratio between O and G yields another measure called Index of Effective Opposition, or
IEO, see Altman and Pérez-Liñán, op. cit. The IEO rewards a larger opposition bloc, rather
than the balance between government and opposition. For a use of this measure, see Altman,
op. cit.
24. The index assumes party cohesion, which was presumably higher in Venezuela than in
Ecuador. Although lack of discipline may create some noise in our indicator, we presume
that, even in cases of low cohesion, party labels are yet ‘somewhat meaningful predictors of
how legislators would vote’. See Scott Mainwaring and Aníbal Pérez-Liñán, ‘Party
Discipline in the Brazilian Constitutional Congress’, Legislative Studies Quarterly, Vol.22,
No.4 (1997), pp.452–83.
25. For a more detailed discussion on the performance of C, see Centellas, op. cit. and Altman
and Pérez-Liñán, op. cit. The C index is a measure of potential contestation, rather than an
indicator of closeness in the races. C would reflect any consociational agreement dividing the
legislative seats as effective power sharing, even if the distribution of votes is not that even.
26. Uncertainty about the classification of borderline cases may originate in insufficient
information, measurement error, or the fact that democracy is a continuous, rather than a
dichotomous concept. Our fuzzy-set approach does not require us to take a position in the
debate between dichotomous and continuous measures of democracy, see David Collier and
Robert Adcock, ‘Democracy and Dichotomies: A Pragmatic Approach to Choices about
Concepts’, Annual Review of Political Science, Vol.2 (1999), pp.537–65.
27. Michael Smithson, Fuzzy Set Analysis for Behavioral and Social Sciences (New York:
Springer-Verlag, 1987). Charles Ragin, Fuzzy-Set, Social Science (Chicago: University of
Chicago Press, 2000). Charles Ragin, ‘Conceptualizing Complexity: A Fuzzy-Set
Approach’, in Annual Meeting of the American Political Science Association (Atlanta,
1999).
28. Mike Alvarez, José A. Cheibub, Fernando Limongi and Adam Przeworski, ‘Classifying
Political Regimes’, Studies in Comparative International Development, Vol.31, No.2 (1996),
pp.3–36. Scott Mainwaring, Democratic Survivability in Latin America, The Helen Kellogg
Institute for International Studies, University of Notre Dame, Working Paper No.267 (1999).
Scott Mainwaring, Daniel Brinks and Aníbal Pérez-Liñán, Classifying Political Regimes in
Latin America 1945–1999, The Helen Kellogg Institute for International Studies, University
of Notre Dame, Working Paper No.280 (2000).
29. We suspect that the ACLP dichotomous measure of democracy is too inclusive (that is, the
risk of type-II error in the selection process is high). For instance, Alvarez et al. coded the
late years of military dictatorship in Brazil (1979–85) as democratic (see Mainwaring et al.,
‘Classifying’, op. cit. On the other hand, Mainwaring’s definition of democracy may be too
demanding for our purposes (therefore increasing the risk of type-I error).
30. Hill, op. cit., pp.100–102. Principal component analysis yielded a single factor (with an
eigenvalue of 1.92) accounting for 64 per cent of the variance in the component dimensions.
Communalities (h2) were 0.52 for C, 0.62 for VAT, and 0.78 for Freedom House Scores.
31. Alberto Marradi, ‘Factor Analysis as an Aid in the Formation and Refinement of Empirically
Useful Concepts’, in David J. Jackson and Edgar F. Borgatta (eds), Factor Analysis and
Measurement in Sociological Research. A Multi-Dimensional Perspective (Beverly Hills:
SAGE, 1981), p.13.
32. Ragin, op. cit., p.4.
33. Gerardo Munck, ‘Canons of Research Design in Qualitative Analysis’, Studies in
Comparative International Development, Vol.33, No.3 (1998), pp.18–45.
34. Anita Hubley and Bruno Zumbo, ‘A Dialectic on Validity: Where we Have Been and Where
we are Going?’, The Journal of General Psychology, Vol.123, No.3 (1996), p.214.
92dem05.qxd 02/05/02 12:00 Page 99

ASSESSING THE QUALITY OF DEMOCRACY 99

35. Gurr, Jaggers and Moore, op. cit., pp.73–108.
36. Robert Adcock and David Collier, ‘Connecting Ideas with Facts: The Validity of
Measurement’, Annual Meeting of the American Political Science Association (Washington,
DC, 2000).
37. Of course, this test created the problem of choosing an operational definition of democracy
again. In this case, we just followed Mainwaring’s ‘Democratic Survivability in Latin
America’ and its restrictive definition.
38. Hill, op. cit.; Centellas, op. cit.; Altman and Pérez-Liñán, op. cit.
39. Valenzuela, op. cit.
40. O’Donnell, ‘On the State’, op. cit., Putnam, Making Democracy Work, op. cit.

APPENDIX

Country Year FH Effective C Year Turnout

Legislative Number Index Elections
of Parties

Argentina 1984 0.83 2.220 0.882 1983 0.775

Argentina 1986 0.92 2.327 0.831 1985 0.778
Argentina 1988 0.92 2.491 0.889 1987 0.801
Argentina 1990 0.83 2.536 0.789 1989 0.825
Argentina 1992 0.75 2.684 0.740 1991 0.894
Argentina 1994 0.75 2.705 0.734 1993 0.781
Argentina 1996 0.75 2.776 0.667 1995 0.798
Bolivia 1983 0.75 3.997 0.815 1980 0.591
Bolivia 1986 0.75 4.313 0.852 1985 0.652
Bolivia 1990 0.75 3.917 0.998 1989 0.508
Bolivia 1993 0.75 3.715 0.782 1993 0.500
Brazil 1986 0.83 3.294 0.923 1986 0.704
Brazil 1989 0.83 6.914 0.827 1989 0.794
Brazil 1990 0.75 7.079 0.905 1990 0.766
Brazil 1994 0.67 7.870 0.977 1994 0.769
Chile 1990 0.83 4.382 0.996 1989 0.863
Chile 1994 0.83 4.948 0.964 1993 0.817
Colombia 1978 0.75 2.064 0.837 1978 0.34
Colombia 1982 0.75 1.985 0.824 1982 0.431
Colombia 1986 0.75 2.450 0.836 1986 0.421
Colombia 1990 0.58 2.183 0.652 1990 0.356
Colombia 1991 0.67 3.007 0.552 1991 0.256
Colombia 1994 0.58 3.348 0.456 1994 0.284
Costa Rica 1978 1.00 2.366 0.911 1978 0.753
Costa Rica 1982 1.00 2.269 0.671 1982 0.79
Costa Rica 1986 1.00 2.209 0.884 1986 0.801
Costa Rica 1990 1.00 2.209 0.884 1990 0.851
Costa Rica 1994 0.92 2.300 0.890 1994 0.842
Dominican R. 1978 0.83 1.994 0.946 1978 0.628
Dominican R. 1982 0.92 2.249 0.850 1982 0.656
Dominican R. 1986 0.83 2.531 0.867 1986 0.607
Dominican R. 1990 0.75 3.053 0.938 1990 0.473
Dominican R. 1994 0.58 2.434 0.878 1994 0.306
Dominican R. 1996 0.67 2.434 0.878 1996 0.616
Ecuador 1979 0.83 3.944 0.712 1979 0.416
Ecuador 1984 0.83 6.659 0.973 1984 0.58
Ecuador 1986 0.75 7.490 0.921 1986 0.649
92dem05.qxd 02/05/02 12:00 Page 100

100 D E M O C R AT I Z AT I O N

A P P E N D I X (Cont’d)

Country Year FH Effective C Year Turnout

Legislative Number Index Elections
of Parties

Ecuador 1988 0.83 4.465 0.656 1988 0.671

Ecuador 1990 0.83 6.732 0.943 1990 0.647
Ecuador 1992 0.75 6.208 0.994 1992 0.7
Ecuador 1994 0.75 5.830 0.862 1994 0.663
Ecuador 1996 0.67 6.579 0.896 1996 0.691
El Salvador 1984 0.42 3.163 0.860 1984 0.581
El Salvador 1985 0.67 2.097 0.837 1985 0.483
El Salvador 1988 0.67 2.406 0.790 1988 0.435
El Salvador 1989 0.58 2.410 0.790 1989 0.394
El Salvador 1991 0.58 3.011 0.754 1991 0.44
El Salvador 1994 0.67 3.064 0.743 1994 0.472
Guatemala 1986 0.50 2.980 0.644 1985 0.477
Guatemala 1990 0.58 4.435 0.916 1990 0.38
Guatemala 1994 0.42 3.480 0.701 1994 0.145
Guatemala 1995 0.42 2.716 0.634 1995 0.31
Honduras 1982 0.67 2.164 0.837 1981 0.519
Honduras 1985 0.75 2.122 0.943 1985 0.778
Honduras 1989 0.75 2.000 0.875 1989 0.757
Honduras 1993 0.67 2.031 0.860 1993 0.635
Mexico 1988 0.58 3.039 0.602 1988 0.417
Mexico 1991 0.50 2.214 0.477 1991 0.5
Mexico 1994 0.50 2.287 0.593 1994 0.659
Nicaragua 1984 0.33 1.787 0.332 1984 0.74
Nicaragua 1990 0.67 2.054 0.850 1990 0.757
Nicaragua 1996 0.67 2.737 0.849 1996 0.759
Panama 1990 0.67 3.315 0.855 1989 0.556
Panama 1994 0.75 4.149 0.667 1994 0.702
Paraguay 1989 0.58 1.888 0.592 1989 0.547
Paraguay 1993 0.67 2.451 0.873 1993 0.394
Peru 1980 0.75 2.472 0.693 1980 0.585
Peru 1985 0.75 2.314 0.600 1985 0.691
Peru 1990 0.58 4.042 0.915 1990 0.628
Peru 1995 0.42 2.912 0.514 1995 0.621
Uruguay 1985 0.83 2.923 0.877 1984 0.939
Uruguay 1990 0.92 3.350 0.846 1989 0.949
Uruguay 1995 0.83 3.299 0.970 1994 0.919
Venezuela 1979 0.92 2.646 0.924 1978 0.745
Venezuela 1984 0.92 2.422 0.650 1983 0.769
Venezuela 1989 0.83 2.828 0.751 1988 0.728
Venezuela 1994 0.67 4.650 0.902 1993 0.492