Research publications

Contents

• Executive Summary
• 1 Introduction: Why Measure Democracy?
• 2 Defining and Measuring Democracy
• 3 Freedom House’s Index
  • 3.1 Canada’s Results
  • 3.2 Possible Limitations of Approach
• 4 The Economist Intelligence Unit’s Index
  • 4.1 Canada’s Results
  • 4.2 Possible Limitations of Approach
• 5 Varieties of Democracy’s Index
  • 5.1 Canada’s Results
  • 5.2 Possible Limitations of Approach
• 6 Conclusion
• Notes

Executive Summary

Measuring Democracy examines three widely used democracy indexes: those produced by Freedom House, the Economist Intelligence Unit, and Varieties of Democracy (V‑Dem). This HillStudy explains how these indexes work, how each index evaluates the state of democracy in Canada, how they differ from each other and some possible limitations of their approaches.

The paper shows that democracy is a broad and varied concept that is not easily defined. Efforts to measure democracy therefore involve necessarily subjective decisions about how to define democracy, how to find evidence of it on the ground and how to compile that evidence into a numerical score.

Different measurement choices produce different results. Canada tends to rank closer to the top in some indexes than in others, although it ranks consistently well relative to most other countries. The countries that tend to be ranked highest – and, to a lesser extent, lowest – are not the same for each index. For example, in 2023, no country ranked in the top three of each index. However, this variation should not be overstated: there is loose agreement across the three indexes as to which countries are most and least democratic.

This HillStudy also shows that political scientists and other observers have identified possible limitations with each index. Notably, the most methodologically sophisticated of the indexes– V‑Dem’s – is highly transparent in some regards but opaque in others.

While it may not be possible to achieve a perfect measure of democracy, democracy indexes nonetheless provide benchmarks that allow for relatively rigorous comparisons of the state of democracy from one year to the next, both globally and at the individual country level.

1 Introduction: Why Measure Democracy?

In recent years, a growing chorus of observers has argued that the world is becoming less democratic. According to Freedom House’s 2024 Freedom in the World report, global freedom declined in 2023 for the 18^th consecutive year.1 Concerns have even been raised about the state of democracy in longstanding democracies with robust institutions, serving as a reminder that democratic institutions are not self⁠-⁠sustaining. Canadians – like people everywhere – have good reason to want to know how democracy is faring at home and abroad. But how do we get a reliable answer to such an enormous and complex question?

A democracy index is a tool designed to address this question. Democracy indexes attempt to measure democracy in an objective, transparent and rigorous way. The most ambitious of them aspire to collect data on every country in the world and to aggregate that data into individual scores or classifications, or both. Because the same method is used to evaluate the state of democracy in each country, scores can be used to compare countries and even to rank countries from most to least democratic. Some democracy indexes are published annually, allowing users to track increases or decreases in levels of democracy across time.

While democracy indexes are meant to measure democracy, they can also influence it. In a broad sense, an index might be said to put pressure on governments by publicizing their successes and failures. As an indication of their influence, in March 2024, India announced the release of its own democracy index, reportedly out of concern that negative assessments from the three indexes discussed in this HillStudy might hurt its credit rating.2 Democracy indexes can also inform foreign policy decisions: for example, Freedom House’s Freedom in the World reports have been used by government bodies in the United States (U.S.) to guide decisions about foreign aid allocation and to assess democracy promotion efforts, as noted below.

Measuring democracy is complex and requires analysts to make a series of subjective decisions. Different approaches can and do produce different conclusions about individual countries and the state of democracy globally. In what follows, three widely used democracy indexes are examined: those of Freedom House, the Economist Intelligence Unit (EIU) and Varieties of Democracy (V‑Dem). This HillStudy explains how each of these indexes work, how they evaluate the state of democracy in Canada, how they differ from one another and some possible limitations of their approaches.3

2 Defining and Measuring Democracy

Etymologically, democracy is a combination of the ancient Greek words “people” (demos) and “power” (kratos) – but how the power of the people might best be realized in democratic institutions is as much a moral and philosophical question as it is an empirical one. What follows is a schematic presentation of some of the important choices faced by an individual or organization attempting to measure democracy, as well as a brief consideration of the potential consequences of those choices.4

At the outset, the compilers of democracy indexes must decide what they are looking for: they must come to some conclusions about what democracy is and what makes a country more or less democratic. Some attributes are likely to be uncontroversial: free and fair elections and freedom of the press are widely considered fundamental attributes of democracy.5 Other possible attributes are more contentious. For instance, one index deems mandatory voting pro⁠-⁠democratic, while another deems it anti‑democratic.6 Initial decisions about what are and are not features of a democracy will inevitably guide the rest of the measurement process.

In the second stage, the compilers of democracy indexes must decide how they will measure the attributes they have identified as fundamental. They must move from a concept like free and fair elections to a series of real⁠-⁠world “indicators” – like voting laws and the presence of genuine opposition parties – the presence or absence of which “indicate” the existence of free and fair elections. There is much room for variation here. In 2018, a group of political scientists reviewing the situation wrote that “nearly all researchers who have attempted to measure democracy have used different indicators. The situation is confusing.”7

Thirdly, the compilers of democracy indexes must decide how to combine each indicator into an overall score. For instance, Freedom House gives each of its indicators equal weight and adds the indicators together to produce a total score out of 100. By contrast, V‑Dem gives its indicators unique weights and produces a total score by both adding and multiplying the indicators together. Different decisions about how to aggregate indicator scores will also produce different results.

Finally, if a democracy index wishes to assess the global state of democracy, it faces several additional choices, one of which is whether to assess by country or by population. If the index uses population⁠-⁠based assessment, then the state of democracy in a very large country like India will have a major influence on the conclusions it will reach about the state of global democracy.8

3 Freedom House’s Index

Freedom House – a non⁠-⁠profit organization based in Washington, D.C. – began offering year⁠-⁠end reviews of freedom around the world in the 1950s. These reviews became more comprehensive in 1972, when Freedom House launched an annual Comparative Study of Freedom. Now called the Freedom in the World report, the annual review has a wide reach and is cited in newspaper articles, think tank reports, academic papers, speeches by politicians and the like. It should be noted that the Freedom in the World survey is intended to measure freedom, not democracy, but that Freedom House’s index is nevertheless often used as a measure of democracy. Moreover, Freedom House considers there to be a correlation between a country’s freedom rating and its level of democracy.9

The Freedom in the World report ranks freedom on a scale of -4 to 100, where -4 is the least free and 100 is the freest. Countries and territories are given a score of zero to four for each of 25 indicators for an overall score out of 100. The 25 indicators are divided into two main categories – Political Rights and Civil Liberties – and seven subcategories. One additional discretionary question addressing forced demographic change allows for up to four points to be subtracted from a country or territory’s overall score, creating the possibility of a score below zero.10

Each indicator takes the form of a question: for instance, “Does the government operate with openness and transparency”? or “Is there an independent judiciary”? According to Freedom House, the indicators used to score each country and territory are derived from the United Nations’ 1948 Universal Declaration of Human Rights.

Each indicator is assessed by a team of analysts drawing on a range of sources, including news articles, academic analyses, reports from non⁠-⁠governmental organizations and on⁠-⁠the⁠-⁠ground research. Analysts then defend their proposed scores at a series of review meetings before a final score is determined. Scores are determined based on the events that took place in a country within a given year. However, Freedom House usually only changes a country’s score from the previous year in response to major events, thus favouring score continuity.11

In addition to a numerical score, each country is classified as “Free,” “Partly Free,” or “Not Free,” based on an aggregation of their political rights and civil liberties scores when equally weighted.

3.1 Canada’s Results

As Table 1 indicates, for 2023, Canada received a Political Rights score of 39 out of 40. Canada received a four out of four on nine of this category’s 10 indicators, and a three out of four on an indicator that measures whether the government operates with openness and transparency. According to Freedom House’s assessment, “Canadians requesting information from the federal and provincial governments about public policies and government decisions face long delays and excessive fees, and often receive documents that are heavily redacted.”12 Moreover, a 2019 law amending the Access to Information Act was deemed to have been “completely ineffective,” with no announcements made in 2023 about plans for further reform.13 The change in score on this indicator represents a drop from 2022, when Canada received a four out of four.

In 2023, Canada received a Civil Liberties score of 58 out of 60. This corresponds to a score of four out of four in 13 of this category’s 15 indicators, and a three out of four on two indicators. The first of these two indicators measures whether individuals are free to practise and express their religious faith or non⁠-⁠belief in public and private. To explain why Canada received only three out of four, Freedom House cited Quebec’s 2019 passage of Bill 21 on state secularism, which bans some public sector employees from wearing certain religious symbols while at work.

The second Civil Liberties indicator for which Canada received a score of three out of four measured whether laws, policies, and practices guarantee equal treatment of various segments of the population. Freedom House claims that “Black and Indigenous Canadians remain subject to widespread discrimination, struggle with food insecurity, and have unequal access to education, health care, public services, and employment.”14 Freedom House noted that there has been progress on this issue, but that it has been slow.

Table 2 shows Canada’s Freedom House overall score and inferred ranking (tied with four other countries) for 2023, alongside the top three and bottom three scores in 2023.

Canada scores very highly on the Freedom House Global Freedom ranking. As of the 2024 edition of the report (which covers events in 2023), only Finland (100), Sweden (99), New Zealand (99) and Norway (98) score higher than Canada (97).15

3.2 Possible Limitations of Approach

One criticism of Freedom House’s approach charges that the organization’s rankings reflect a “pro⁠-⁠American ideology.”16 Some political scientists have claimed that Freedom House tends to rank U.S. allies higher, and non⁠-⁠allies lower, than other democracy indexes.17 Observers have also noted the existence of numerous informal ties between Freedom House and the U.S. government, as well as the U.S. government’s use of Freedom House rankings to determine the allocation of some international assistance and to evaluate some democracy promotion efforts abroad.18 However, it is worth noting in this regard that Freedom House’s score for the U.S. has declined in recent years, from a score of 92 in 2014 to a score of 83 in 2024.19

Relatedly, observers have suggested that Freedom House’s criteria are closely aligned with an American conception of democracy that weights certain liberal rights (namely, those related to freedom from state interference) more highly than other potentially important characteristics of democracy, like social justice, economic equality and political participation.20 However, Freedom House’s Yana Gorokhovskaia claims that “Freedom House staff make a concerted effort to find individuals who are living in, working in, studying in, or hailing from the countries and territories that they are assessing for the survey.” In 2022, 72% of Freedom House’s analysts were based outside the U.S.21

Political scientists Jan Erk and Wouter Veenendaal have argued that Freedom House tends to evaluate microstates – like Palau and the Seychelles – too uncritically. Erk and Veenendaal argue that this is because Freedom House’s approach privileges the “more formal aspects of democracy,” like a constitutional provision for restraints on the executive branch, at the expense of “real power relations and pathways of influence,” like the role of kinship networks in determining political appointments. Because there tends to be less data available for microstates than for larger states, Erk and Veenendaal argue that coders rely even more on the existence of formal institutions, leading to an incomplete and overly charitable evaluation of some microstates.22

Another line of criticism claims that Freedom House is not as scientific or transparent as other democracy indexes.23 An influential evaluation of democracy indexes published in 2002 by political scientists Gerardo L. Munck and Jay Verkuilen singled out Freedom House for criticism. The main issue identified by Munck and Verkuilen was Freedom House’s approach to aggregating the indicator scores into a total score; they concluded that Freedom House did not give due attention to the complexities involved in aggregating various measures of democracy into an overall assessment.24

According to Freedom House’s Yana Gorokhovskaia, Freedom House periodically invites academic researchers to review its methodology. She notes that the organization “remains open to collaboration and discussion with other researchers.”25

4 The Economist Intelligence Unit’s Index

The Democracy Index, first published in 2006, is a product of the EIU, the London, United Kingdom‑based research and analysis division of The Economist Group, the sister company to The Economist magazine.

The index uses 60 questions to create indicators that are grouped into five categories: Electoral Process and Pluralism (12 indicators); Functioning of Government (14 indicators); Political Participation (9 indicators); Political Culture (8 indicators); and Civil Liberties (17 indicators).

The indicators use a combination of a dichotomous and three⁠-⁠point scoring system; that is, some indicators are limited to “yes” or “no” answers, with either one or no points given, while other indicators are scored on a three⁠-⁠point scale consisting of 0, 0.5, or 1 point.

Below are examples of each scoring system:

4. Is there universal suffrage for all adults?
   Bar generally accepted exclusions (for example, non⁠-⁠nationals; criminals; members of armed forces in some countries).
   1: Yes.
   0: No. …

29. Women in parliament.
    % of members of parliament who are women.
    1 if more than 20% of seats.
    0.5 if 10–20%.
    0 if less than 10%.26

Each indicator is scored through a combination of experts’ assessments and public opinion surveys. According to the EIU, its use of surveys is “a crucial, differentiating aspect” of its approach.27 The principal survey used by the index is that of the World Values Survey (WVS), a Stockholm⁠-⁠based “global network of social scientists studying changing values and their impact on social and political life.”28 However, the EIU also uses other polls and surveys.29 They note that “in the case of countries for which survey results are missing, survey results for similar countries and expert assessment are used to fill in gaps.”30

In essence, each category receives a score equal to the sum of the indicators in that category converted into a 0⁠-⁠to⁠-⁠10 scale. However, the EIU also notes that a country’s failure to score one point in four “critical areas for democracy” can result in “adjustments to the category scores.”31 Those four areas are:

• whether national elections are free and fair;
• the security of voters;
• the influence of foreign powers on government; and
• the capability of the civil service to implement policies.32

The overall Democracy Index score is the average of the scores in the five categories of indicators.33

The index places countries, based on their overall scores, within one of four regime types: Full Democracies (a score of 8 out of 10 and above); Flawed Democracies (scores between 6 and 8); Hybrid Regimes (scores between 4 and 6); and Authoritarian Regimes (scores of 4 and below).34

4.1 Canada’s Results

Table 3 shows Canada’s scores in each of the five categories in the EIU’s index, as well as its overall score and global ranking, over the last five years.

Table 3 – Economist Intelligence Unit Index: Canada’s Scores and Global Rankings, 2019⁠–⁠2023

Year	Electoral Process and Pluralism (Out of 10)	Functioning of Government (Out of 10)	Political Participation (Out of 10)	Political Culture (Out of 10)	Civil Liberties (Out of 10)	Overall Score (Out of 10)	Global Ranking (Out of 167)
2019	9.58	9.64	7.78	9.38	9.71	9.22	7
2020	9.58	8.93	8.89	9.38	9.41	9.24	5
2021	10.00	8.21	8.89	8.13	9.12	8.87	12
2022	10.00	8.57	8.89	8.13	8.82	8.88	12
2023	10.00	8.21	8.89	7.50	8.82	8.69	13

Sources:

Table prepared by the Library of Parliament using data obtained from Economist Intelligence Unit (EIU), Democracy Index 2019: A year of democratic setbacks and popular protest  (4.4 MB, 73 pages); EIU, Democracy Index 2020: In sickness and in health?  (3.6 MB, 75 pages); EIU, Democracy Index 2021: The China challenge  (2.2 MB, 85 pages); EIU, Democracy Index 2022: Frontline democracy and the battle for Ukraine  (4.8 MB, 84 pages); and EIU, Democracy Index 2023: Age of conflict  (5.3 MB, 84 pages).

Table 4 compares Canada’s overall score and global ranking in the 2023 index to those of the three best- and worst⁠-⁠performing countries.

Table 4 – Economist Intelligence Unit Index: Selected Countries’ Overall Scores and Global Rankings, 2023

Country	Overall Score (Out of 10)	Global Ranking (Out of 167)
Norway	9.81	1
New Zealand	9.61	2
Iceland	9.45	3
Canada	8.69	13
North Korea	1.08	165
Myanmar	0.85	166
Afghanistan	0.26	167

Source:

Table prepared by the Library of Parliament using data obtained from Economist Intelligence Unit, “Table 2: Democracy Index 2023,” Democracy Index 2023: Age of conflict  (5.3 MB, 84 pages), pp. 9–13.

While Canada ranked 13^th out of 167 (165 independent states and two territories) in the EIU’s 2023 index, it ranked fifth as recently as 2020 (as shown in Table 3). In the 2021 index, Canada’s ranking dropped seven spots to 12^th, driven by a significant decline in the Political Culture category. The 2021 Index emphasized that Canada’s performance was considerably lower compared to 2020 in large part because of the 2017⁠–⁠2020 WVS results, released in July 2021.35 The biggest impact of the WVS data, the index further demonstrated, was felt in the Functioning of Government and Political Culture categories. As Table 3 shows, Canada declined in both categories in 2021 – from 8.93 to 8.21 in the Functioning of Government category and from 9.38 to 8.13 in the Political Culture category. The index noted that the WVS data

captured a souring of public sentiment amid the coronavirus pandemic, particularly as the country grappled with a second wave of infections during that month. The results also reflected frustration concerning the reimposition of pandemic containment measures, after these were rolled back during the summer months, as well as reported difficulties accessing the federal government’s coronavirus relief benefits.36

4.2 Possible Limitations of Approach

Some criticism of the EIU’s index relates to the transparency and validity of its data. The Spanish academic Jordi Mas Elias, for instance, has noted that the index does not provide data at the indicator level, which makes it difficult to understand variations in scores, from one year to the next, at the category level.37 Concerning the indicators themselves, others have criticized the index for not including indicators related to social protection, social welfare or outcome⁠-⁠based economic equality.38

The analyst Peter Tasker has criticized the anonymity of the experts upon whom the EIU relies,39 and others have indicated that the EIU index consistently rates some countries, such as Russia and Kazakhstan, higher than Freedom House because these countries “do relatively well in terms of political participation.”40

For its part, the EIU itself acknowledges, “There is no consensus on how to measure democracy. Definitions of democracy are contested, and there is a lively debate on the subject.”41 It also recognizes that the Democracy Index does not include elements that “some authors argue are also crucial components of democracy – such as levels of economic and social wellbeing.”42 It concludes, nonetheless, that the index “respects the dominant tradition that holds that a variety of social and economic outcomes can be consistent with political democracy, which is a separate concept.”43

5 Varieties of Democracy’s Index

V‑Dem (Varieties of Democracy) is a research project administered at the V‑Dem Institute, which is based at the University of Gothenburg’s Department of Political Science in Sweden. V‑Dem has been publishing annual democracy reports since 2017.44

These reports are based on V‑Dem’s database, which is “the largest global dataset on democracy.”45 The database contains 367 indicators for 179 countries, making it more granular than either the Freedom House index or the EIU’s index.46 Furthermore, for approximately 80 countries, the database extends its measurement of some indicators back to 1789.47 The database, therefore, provides a much longer time horizon for comparison than the Freedom House or EIU indexes.

V‑Dem uses both experts and research assistants to produce a score for each country: research assistants code easily observable (or “factual”) indicators, like the existence or non⁠-⁠existence of legally enshrined constraints on the executive. Experts code difficult to observe (or “subjective”) indicators, like the ability of the legislature to constrain the executive in practice.48 Each subjective indicator is assessed by a minimum of five experts,49 who code the indicator by choosing from a given set of answers, which are often presented on a five⁠-⁠point scale from 0 to 4.

Once the indicators have been coded, they are aggregated at several levels. As part of the aggregation process, indicators are given unique weights, and the relations between indicators are taken into consideration.50 Moreover, V‑Dem uses an advanced statistical model to aggregate expert judgments in a manner intended to moderate expert bias and account for uncertainty about estimates.51 This approach provides a range of scores for each indicator, along with a best estimate, based on deviations among experts, experts’ evaluations of their own uncertainty and other factors.

At the highest level, indicators are aggregated into five different indices (or “varieties”) of democracy: the Electoral Democracy Index, the Liberal Democracy Index, the Participatory Democracy Index, the Egalitarian Democracy Index, and the Deliberative Democracy Index.52 This means that each country receives five scores rather than one. V‑Dem measures five different varieties of democracy in an attempt to reflect the varied ways the term “democracy” is used – although they consider electoral democracy to be fundamental to the other four, and a country’s electoral democracy score influences its score in the other four categories.

V‑Dem scores are also used by an affiliated Regimes of the World index, which classifies countries as either Closed Autocracy, Electoral Autocracy, Electoral Democracy or Liberal Democracy. These classifications are used in V‑Dem’s annual democracy reports but are not officially endorsed by V‑Dem. To be classified as a liberal democracy, a country must meet the criteria for electoral democracy while also exceeding a certain threshold on the Liberal Component Index – a sub⁠-⁠index of the Liberal Democracy Index – and three other indicators.53

5.1 Canada’s Results

Table 5 shows Canada’s score (which is V‑Dem’s best estimate of the expert assessments) on the Liberal Democracy Index, its regime classification and its global ranking, over the last five years. Here, the focus is on the Liberal Democracy Index because it forms the basis of V‑Dem’s democracy reports.54

Table 5 – Varieties of Democracy: Canada’s Scores, Regime Classifications and Global Rankings, 2019–2023

Year	Liberal Democracy Index (best estimate)	Regime Classification	Global Ranking
2019	0.78	Liberal Democracy (-)	21
2020	0.74	Liberal Democracy (-)	28
2021	0.75	Liberal Democracy	24
2022	0.74	Electoral Democracy	24
2023	0.76	Liberal Democracy (-)	25

Note:

The minus (-) symbol indicates that the regime categorization is statistically ambiguous and might be better classified as the regime below, i.e., Electoral Democracy. Note that the Regimes of the World typology is not officially endorsed by the V‑Dem (Varieties of Democracy) Steering Committee.

Sources:

Table prepared by the Library of Parliament using data obtained from V‑Dem Institute, Democracy Report 2020: Autocratization Surges – Resistance Grows  (17.0 MB, 40 pages), March 2020, pp. 26 and 30; V‑Dem Institute, Democracy Report 2021: Autocratization Turns Viral  (4.3 MB, 52 pages), March 2021, pp. 31 and 34; V‑Dem Institute, Democracy Report 2022: Autocratization Changing Nature?  (4.5 MB, 60 pages), March 2022, pp. 45⁠–⁠46; V‑Dem Institute, Democracy Report 2023: Defiance in the Face of Autocratization  (4.8 MB, 56 pages), March 2023, pp. 39 and 44; and V‑Dem Institute, Democracy Report 2024: Democracy Winning and Losing at the Ballot  (6.8 MB, 64 pages), March 2024, pp. 17 and 62.

Table 6 situates Canada with respect to the highest and lowest⁠-⁠performing countries in the Liberal Democracy Index.

Table 6 – Varieties of Democracy Scores and Global Rankings: Selected Countries, 2023

Country	Liberal Democracy Index Score	Global Ranking
Denmark	0.88	1
Sweden	0.85	2
Estonia	0.84	3
Canada	0.76	25
Myanmar	0.02	177
North Korea	0.01	178
Eritrea	0.01	179

Sources:

Table prepared by the Library of Parliament using data obtained from V‑Dem (Varieties of Democracy) Institute, Democracy Report 2024: Democracy Winning and Losing at the Ballot  (6.8 MB, 64 pages), March 2024, pp. 17 and 62.

As tables 5 and 6 indicate, Canada ranked 25^th globally on V‑Dem’s Liberal Democracy Index for 2023. This is a lower ranking than that produced by Freedom House’s index (tied for fifth in 2023) or the EIU’s Democracy Index (13^th in 2023). That said, it should be noted that Canada performs better (14^th) on V‑Dem’s Electoral Democracy Index than on its Liberal Democracy Index.55

Because V‑Dem offers only quantitative assessments of each country, it can be difficult to understand why a country has been scored the way it has. It is necessary to consult V‑Dem’s Variable Graph database in conjunction with V‑Dem’s Codebook to discern which indicators are at play and to track them.56 Even then, it is not possible to say why Canada was coded the way it was on a given indicator because V‑Dem’s expert coders do not provide their reasoning.

As Table 5 indicates, Canada’s regime classification – as determined by the unofficial Regimes of the World typology – has fluctuated between Liberal Democracy and Electoral Democracy. As noted above, countries must meet a certain threshold across several indices and indicators to be classified as liberal democracies. Canada regularly meets or exceeds most of these thresholds but has been scored around the threshold on the indicator measuring access to justice for women. Variations in this score have caused Canada’s regime classification to fluctuate. V‑Dem does not offer an explanation for Canada’s score on this or any other indicator.57

5.2 Possible Limitations of Approach

While V‑Dem is often praised for its methodological sophistication, observers have identified some potential shortcomings.

One possible shortcoming of V‑Dem’s democracy reports is the difficulty of interpreting V‑Dem’s findings.58 This is perhaps a necessary corollary of V‑Dem’s pursuit of a high level of social scientific rigour, and it should be noted that V‑Dem is probably the democracy index most widely endorsed by political scientists. One consequence of this rigour, however, is that many of V‑Dem’s resources and publications target an audience with a high degree of statistical literacy.59

Several other criticisms of V‑Dem pertain to its use of expert coders. Some political scientists note that V‑Dem’s reliance on experts might cause it to miss trends – like the erosion of support for democracy – that might be detected by opinion surveys.60 Others have suggested that reliance on expert evaluations could lead to the reproduction and confirmation of biases held by experts.61 However, V‑Dem has argued that there is no evidence of particular biases being widely held among its expert coders.62

According to political scientists Steven Levitsky and Lucan Way, another downside of expert coding of the kind undertaken by V‑Dem is that “only the expert coders know which specific events and factors motivated their coding decisions,” making the scores “impossible to replicate or falsify.” This difficulty is especially apparent when V‑Dem experts make “dubious scoring decisions,” like registering a decline in Ukrainian democracy following the toppling of Victor Yanukovych’s autocratic regime in 2014.63

Furthermore, Levitsky and Way note that reliance on expert evaluation of subjective measures may lead to “divergent benchmarks for democracy” among country experts covering different parts of the world. They cite as an example the fact that V‑Dem had placed Malaysia and Russia in the same category – Electoral Autocracy – even though Malaysia had experienced three democratic turnovers since 2018, while opposition figures in Russia were routinely killed, jailed or barred from running in elections.64

In defence of their use of experts to code subjective indicators, members of the V‑Dem team have argued that “eschewing expert judgment while thoroughly measuring conceptually relevant aspects of democracy is difficult,” and they have observed that even “objective” indicators of democracy require some degree of subjective human judgment to code.65

6 Conclusion

This overview of three influential democracy indexes – those produced by Freedom House, the EIU and V‑Dem – shows that democracy is a broad and contested concept that eludes straightforward measurement. Efforts to measure democracy involve inescapably subjective decisions about how to define democracy, how to find evidence of it on the ground, and how to compile that evidence into a score that allows one to track changes within a country and across the globe.

Different measurement choices produce different results. Canada fares better in some indexes than in others, though it fares consistently well relative to most other countries. The countries at the very top – and, to a lesser extent, at the very bottom – of the rankings are different for each index. For 2023, no country ranked in the top three for all indexes. Interestingly, Denmark, which was not ranked in the top three by either Freedom House or the EIU, ranked first across three of the five indices used by V‑Dem.66

While these indexes may be imperfect tools, they nonetheless provide benchmarks that allow for rigorous and quantifiable comparisons of the state of democracy – both globally and at the individual country level – and from one year to the next. Democracy indexes might also be used to identify trends and diagnose problems before they become irreversible. In short, measuring democracy is a complex but crucial endeavour.

Notes