Law Review
Northwestern Law

Northwestern University Law Review : Colloquy : 2009 : Symposium: The Future of Law and Development, Part II

Symposium: The Future of Law and Development, Part II

Mariana Prado, Susan D. Franck, & John Cioffi

[download pdf]

Should We Adopt a "What Works" Approach in Law and Development?

Mariana Prado[*]

My comments will be mostly connected to Tom's third question, relating to the future,[1] and I would like to start by responding to Salil Mehra.[2]  I generally agree that there is an excessive focus on replicable "tools" and "best practices" in Law and Development studies, but I think there are a lot of questions to be asked regarding Mehra's suggestion that the way forward involves addressing embedded cultural practices and institutions.  In fact, there are currently a number of scholars (myself included) emphasizing the importance of looking at the interaction between so-called informal institutions (such as cultural practices, social norms, and historically entrenched attitudes and values).  The problem is that although most of these analyses are very helpful in understanding what went wrong and why the "toolkit" did not work in a given context, they do not tell us how to improve our efforts going forward.  So, like the "blueprint" Law and Development scholars, the "context matters" Law and Development scholars are not helping the field move forward.

The problem becomes even more pressing when we move out of specific legal fields, such as best practices for antitrust or telecommunications regulators, to something like the rule of law.  There seems to be some palpable and measurable outcome in the former (for example, we can assess whether the telecommunications sector is operating with lower rates or greater competition), but not in the latter.  Brian Tamanaha argues that this explains why rule of law projects continue to receive generous financial support, despite their poor results:

The "rule of law" is intangible and cannot be pointed at like a broken factory. . . .  [T]he rule of law has no blueprint, no standard structure, and is not something that can be constructed on demand.  There is no timetable . . . .  Judgments about the value and effectiveness of rule of law projects thus can be postponed indefinitely.[3]

One could argue that measuring success in the fields of antitrust and telecommunications regulation also presents difficulties.  As Daniel Sokol suggests in his contribution to this Symposium, we can always ask if rates could have gone even lower than they did, or if the benefits of regulatory reforms could have been better distributed among society.[4]  Indeed, such questions make it hard to define what constitutes success in antitrust and telecommunications regulation.  Those issues notwithstanding, in these fields there does seem to be a threshold line between failures and non-failures.  There are outcomes (such as efficiency, prices, and competition) that allow us to assess whether reforms improved—at least in some respect—the utility services provision, or if regulation had any impact on market structures.  In other words, although it might still be hard to define absolute success, it seems easier to identify a "broken factory" in these sectors than in rule of law reforms.

Katharina Pistor tries to tackle the lack-of-measurable-outcomes problem in rule of law reforms by suggesting that we should identify our ends (and reject the idea that rule of law is an end in and of itself) and "begin to disentangle the relation between specific legal reform efforts and [ ] social and economic indicators . . . ."[5]  In other words, she suggests a "what works" approach.  This seems like a healthy shift of focus, because it allows us to consider alternative legal and institutional arrangements, avoiding the "toolkit" approach.  Also, defining our ends seems to be what was missing in the idea that "context and difference matters": by knowing what we are trying to achieve, it will be easier to search for successful examples and change or adapt them to devise similar solutions in other contexts.  Recovering a bit of Tamanaha's 1995 optimism, "the end result may be the achievement around the world of successful, indigenous permutations of the rule of law, maintaining its core elements yet altering it to fit local circumstances."[6]

There are a number of issues, however, that need to be considered before we follow this path.  First, although the "what works" approach might open our eyes (and minds) to ingenious solutions developed by insiders and locals, it can also quickly turn into a race around the world, wherein legal reformers try to find innovative solutions to replicate elsewhere.  The experience of participatory budgeting in Porto Alegre, Brazil,[7] seems to be a clear example: when replicated in other Brazilian cities and throughout Latin America, it did not always work as well as it did originally.[8]  This transfer-failure routine will not happen if we "disentangle the relationship between specific legal reform efforts and [ ] social and economic indicators," but until that happens, there could be a strong temptation to simply replicate successes.

Second, the "what works" approach can create an obsession with social and economic indicators that might generate the wrong incentives for developing countries.  For instance, those who do not want reforms to happen might manipulate the numbers.  Such practices might become even more pernicious if these indicators become the measurement used to determine if developing countries should receive loans, garner support for rule of law reforms, or attract foreign direct investment.

Third, I take Katharina Pistor[9] and Daniel Kaufmann's[10] point about the gap between law in the books and law in reality, but there is a conceptual question about ends and means here.  How much can we tell our ends and means apart when we are concerned, for instance, with the issues raised by Katharina Pistor: discrimination and the status of women in society?  Are women's rights just a means to reach equality, or an end in itself?  Can we set up a linear path in which we say that women's rights is a means and de facto equality is an end, or is it the case that the concern with de facto equality is embedded in the idea that there is an effective legal system to protect women's rights when necessary?  Sometimes it might not be clear what the end point looks like.

Fourth, the "what works" approach may create the temptation to adopt shortcuts to achieve immediate ends or goals, without necessarily considering the broader institutional implications of these strategies.  One example is investment treaty arbitration (ITA).  As Susan Franck shows, there is a great deal of empirical evidence that allows for a cautioned optimism about the integrity of ITAs in solving investment disputes.[11]  However, investors may use ITAs because they want to avoid corrupt judiciaries or do not want to rely on a dysfunctional and inefficient court system.  If this is the case, then the availability of ITAs will eliminate some of the political pressure that could mobilize powerful interest groups to press for judicial reforms.  Thus, the goal of solving disputes and attracting investments needs to be considered within context; these benefits may bring with them a risk of reduced judicial reform, a reform that could benefit not only investors, but also the population at large.

Finally, if the "what works" approach becomes predominant among Law and Development institutions, what is the role that scholars can and should play?  On the one hand, they can "engage with the projects while providing serious scholarly analysis and critique," as John Ohnesorge suggests, in an attempt to avoid the abovementioned problems.[12]  On the other hand, there is always the risk that "toolkit" people will misappropriate any scholarly critical work, exacerbating the problems noted above.  A careful analysis disentangling the relationship between specific legal reform efforts and well-identified social and economic indicators, for example, can always be misread as suggesting an empty emphasis on social and economic indicators.  The same analysis, focusing on one specific case of successful legal innovation, can be misused as a suggestion that this particular experience should be transplanted into other contexts.  In sum, "toolkit" people can always convert a scalpel into an axe.  The question, then, is whether scholars can do anything to prevent such misuse.

Investment Treaty Arbitration and Law & Development

Susan D. Franck[**]

Tom Ginsburg's initial post raises a series of fascinating questions about the future of Law and Development.  These questions encourage us to think about the utility of different methodological approaches, basic definitions, and the implications for institutional change.  Investment treaty arbitration (ITA)—which gives foreign investors the right to arbitrate directly against a host government for arguable violations of their substantive investment right—provides a unique opportunity to explore these issues in a tangible manner.  So I beg your indulgence as I reflect upon a specific application (namely, ITA) to highlight some of the themes raised by my fellow bloggers.

As the legitimacy of ITA becomes a matter of heated debate, who wins the dispute has become a matter of particular interest.  Suggesting that ITA is unfairly tilted toward the developed world, various countries have withdrawn from World Bank dispute resolution bodies or are considering the elimination of arbitration.[13]  Rather than relying on anecdotal evidence, supposition, or political rhetoric, it is vital to provide systematic data to aid stakeholders in the assessment of the ITA process and consider the implications for international development.  Ideally, a mixed-methods approach that both (1) capitalizes on the strengths of the individualized, sociological, and qualitative approaches advocated by Katharina Pistor[14] and John Ohnesorge,[15] and (2) contextualizes specific experiences within the framework of a larger puzzle offered by broader quantitative research, could provide particularly useful insights.[16]  Given the availability of data from public arbitration awards, this Essay focuses upon quantitative aspects of a mixed-methods approach.

Previous research has shown that although both investors and governments won investment treaty arbitration cases, the respondent states were more likely than investors to win (57.7% for states as compared with a 38.5% win rate for investors).  In cases where there was a violation of the underlying international investment agreement (IIA), tribunals awarded amounts that were smaller than what investors claimed.  More particularly, while investors claimed an average of $343 million in damages, the average award was in the order of $10 million.  In sum, far more investors lost than won, and when investors did win, they usually received far less than they originally claimed.[17]

The open question was whether the outcomes reported were somehow related to variables such as the parties' or arbitrators' development background.  Finding the answer to this question raises an issue echoed by John Cioffi, namely that "large-n quantitative analyses of law related variables . . . [are] deeply problematic" when "these studies lump together highly developed countries" and developing countries.[18]  Given the possible disparate impact on the developing world, it is critical that research consider the difference in outcomes between these two groups.  If outcome is reliably associated with—let alone causally influenced by—the development background of the respondent state or presiding arbitrator, serious questions about the integrity of ITA could arise.

But even recognizing the difference between developed and developing countries raises a question: what is "development status"?  In the context of quantitative research, operationalizing terms properly and establishing measurement validity is one of the thorniest issues.  "Development status" can mean different things to different people in different contexts.  In order to benefit from standard terms and begin the process of creating more nuanced analysis, my own research started by using pre-existing measures and categories for defining development.  In particular, the research considered "development status" in two ways, namely by analyzing categories, including: (1) membership in the Organisation for Economic Co-operation and Development (OECD), and (2) World Bank classification as a High Income, Upper-Middle Income, Low-Middle Income, or Low Income country.[19]

Although this was a relatively straightforward metric for defining the development status of respondent states, defining the "development status" of arbitrators was more difficult.  Arbitrators' development status might have been measured in various ways, including pure nationality of origin, country of residence, country of legal training, number of advanced degrees, membership in professional organizations, average annual income, or some combination thereof.  For the purposes of this initial, limited study, development status for arbitrators was defined as a function of arbitrator nationality, in part because of the historical focus upon arbitrator nationality and the belief that nationality is a proxy for adjudicative neutrality.[20]  A presiding arbitrator's status was, therefore, measured by considering the OECD or World Bank classification of his/her country of origin.

Bearing in mind how "development" was operationalized in this research, and recognizing that different constructions may require different methodological approaches and definitions, the newest generation of research considered whether there was a reliable statistical link between development status and ITA outcome.[21]  One study considered only the impact of a respondent's development status on outcome.  The results of statistical analyses demonstrated that there was no statistically significant relationship between a government's development background and the outcome of ITA.[22]

A second study considered the relationship among outcome, the development status of the respondent state, and the development status of the presiding arbitrator's country of origin.  The results generally showed that outcome was not reliably associated with the development status of the respondent, the development status of the presiding arbitrator, or some interaction between those two variables.  This lack of relationship held true for both: (1) winning or losing investment treaty arbitration, and (2) amounts tribunals awarded against governments.[23]  There were, however, two statistically significant simple effects—found in one sub-set of potentially non-representative cases[24]—that suggested tribunals with presiding arbitrators from Middle Income countries awarded different damages in cases against High Income countries. Specifically, if the presiding arbitrator was from a Middle Income country, High Income countries received statistically lower awards than either: (1) Upper-Middle Income respondents, or (2) Low Income respondents.  Awards by Middle Income presiding arbitrators for High Income and Lower-Middle Income respondents were statistically equivalent.[25]

The overall results cast doubt on the arguments that: (1) ITA is the equivalent of tossing a two-headed coin to decide disputes, (2) the developing world is treated unfairly in ITA, and (3) arbitrators from the developed and developing world decide cases differently.  The evidence creates a basis for cautious optimism about the integrity of ITA and suggests radical overhaul, rejection, or rebalancing of procedural rights in International Investment Agreements (IIAs) is not necessarily warranted.  Although the follow-up tests and limitations of the data suggest optimism must be tempered properly, a sensible approach would involve creating targeted solutions to address particularized problems and enacting targeted reforms to redress perceived concerns about the international investment regime.

In the context of ITA, Salil Mehra's point that the "way forward for Law and Development [ ] involves embracing and addressing differences rather than seeking a universal solvent"[26] buttresses this need for individualized solutions.  Individualized solutions, in turn, may help address the concerns raised by Mariana Prado.  It suggests that scholars can and should play a vital role in the Law and Development debate.  Part of that role could be encouraging the discourse to move beyond a dichotomy polarized by "unilateral blueprint" versus "context matters" models.[27]  In the context of the resolution of international investment disputes, this means encouraging scholars to: develop methodological insights that operate on multiple levels, consider different definitions of terms like "development," recognize the limitations of inferences based upon specific methods and definitions, provide interpretive guidance about the policy implications, and—in light of those points—develop theories, subject to the research loop, that respect and reflect the complexities of variation within the population.

Law & Development: Past Performance Is Not Indicative of Future Results

John Cioffi[***]

I come to the subject of Law and Development as something of an outsider.  My work in law and comparative political economy focuses on the advanced industrial countries,[28] where rule of law issues are generally less salient and pressing (at least until recently).  That perspective, however, has informed my reaction to much of the Law and Development literature.  First, as a conceptual and methodological matter, I note that the trend towards large-n quantitative analyses of law-related variables and economic outcomes is deeply problematic; often, these studies lump together highly developed countries with a well-established rule of law with those in which the rule of law is as undeveloped as their economies.  Second, much of the literature builds on questionable but common assumptions about the rule of law, conceptually rooted in neo-liberalism and its presumptively beneficent relation to development in general, and certain classes of economic outcomes in particular.  These assumptions elide critical questions about (1) the arguable necessity of the rule of law, at least in its neo-liberal guise, to successful economic development, and (2) the relation of the rule of law to the historically defined social and political context of legal rules and institutions.

As an initial matter, I regard the relation of law to economic and social development as a matter of enormous import and one in dire need of sustained thoughtful inquiry.  Yet, I also join Anna Gelpern[29] in avoiding the subject of whether Law and Development is a "field" or not.  That question turns on definitional matters, on what constitutes a field, or a sub-field, or a discipline, etc.—definitions that are seldom resolved satisfactorily, let alone conclusively.  Such debates often provide academics with solid exercise, but ultimately lend themselves more to professional boundary policing, ontological combat (or stalemate), and methodological battles without end, rather than research leading to improved policy proposals.  Academic disciplines are littered with these types of disputes; they are seldom productive or edifying.

Perhaps a more useful way to view Law and Development is from the perspective of why it has been resurrected, after the dispiriting failures of the original Law and Development movement of the 1960s, to become such a prominent and a seemingly inevitable subject of intellectual inquiry and policy activism.  This return strikes me as the product of a particular historical moment.  The renewed emphasis on rule of law and law reform issues in the developing world reflects the decline of more statist industrial policies, such as import substitution industrialization policies and state guided (or controlled) allocation of credit, common to the Japanese-inspired East Asian developmental model.  Statist and corporatist development strategies were often successful.[30]  However, they were hardly law-centric in their governance, nor did they use law to convey broad discretionary authority to public and/or private actors.  Statist strategies, even in successful industrialized countries like France and Japan, confronted diminishing returns over time.  Corruption and rent-seeking on a grand scale, though varying considerably cross-nationally, helped to discredit statism further[31]—whether this discrediting was warranted across the board may now be a moot point.  One important casualty of the global economic crisis may well be export-driven growth models such as those deployed with notable success in East Asia.

The diminution—and increasing distrust—of state activism, coupled with burgeoning globalization of trade and finance, left policymakers ever more reliant on markets as primary allocative mechanisms.  And markets require at least a basic legal infrastructure of rules, enforcement mechanisms, and functional legal institutions to operate efficiently.  To paraphrase Steven Vogel, freer markets require more rules.[32]  A brutal lesson of the current global economic crisis is that the increasing complexity and development of a national economy compels the expansion and maintenance of more complex and well-administered legal, regulatory, and institutional frameworks.

Conceptions of the rule of law rooted in neo-liberalism were poised to fill the vacuum left by what Susan Strange called "the retreat of the state."[33]  This liberal vision of law and its role in ordering economic affairs has much to recommend it, but it suffers from at least two glaring problems.  First, the ideological biases of neo-liberalism toward regulatory minimalism introduce a tension at the center of the law reform project.  Fears of legally-enabled rent-seeking and efficiency-destroying juridification are at cross-purposes with the need to develop strong autonomous legal institutions, codes, and standards suitably tailored to different national settings.  Second, and more important, is the neo-liberal tendency to frame law reform as something of a one-size-fits-all set of policies that can be imported without regard to extant socio-political conditions.  Establishing the rule of law, as a means of both constituting and delimiting state power, and as a means of establishing the ground rules for market-led growth, is intensely politicized and notoriously difficult.  (Indeed, much political and constitutional theory is devoted in some way to this core problem of law and politics.)  It is also an inherently case-specific process.[34]  Hence, large-scale comparative studies may give us some valuable information, but miss the central political economic dynamics that determine the juridical and material outcomes that concern us.

The problem of determining "what works," as Katharina Pistor has called on us to do,[35] is complicated by the paucity of reliable data.  Even in the United States, where legal and regulatory issues are politically sensitive and intensively studied, legal variables are notoriously difficult to measure, as are their effects on behavior.  The problem grows exponentially worse in less developed countries.  Accordingly, studies that are more qualitative and case-specific are likely to be most useful in informing us about the relationships between law and economic behavior and the process by which the rule of law and reasonably efficient legal frameworks can be developed.  In this sense, I agree with John Ohnesorge that detailed sociological investigation is unavoidable.[36]  I would add to this the necessity of finer-grained institutional analysis that bridges economic and sociological theoretical approaches.  But, of course, what we gain in vividness and concrete understanding, we lose in generalizability.  I fully recognize how unsatisfying this conclusion is, but in a domain of scholarly inquiry and policymaking that so often seems in search of magic bullets (or perhaps magic weapons systems), a certain degree of disenchantment is inevitable.

Finally, "what works" is subject to substantial doubt and debate, even when substantive issues of social development (such as women's rights) are left aside and we focus with hard head and shriveled heart on narrower metrics of economic growth.  For example, I speculated above that the export-led growth model may be a casualty of the current global economic crisis and the likely long-term reduction in U.S. consumer demand (the global consumer of last resort for over two decades).  Hopefully, the bubble-driven growth model of the United States and many other developed countries in recent decades will also take its place on the scrap heap of history.  A further victim of the crisis is the finance-centered vision of the firm and economy.  The vast literature on the legal requisites of financial globalization and its beneficial developmental role is premised on analytical assumptions regarding the long-term efficiency and optimizing influence of financial markets.  It also relies on economic data produced by an international financial and economic order that is collapsing around us.  Caveat emptor, caveat legislator: Past performance is not indicative of future results.

————

Copyright 2009 Northwestern University    

Cite as: 104 Nw. U. L. Rev. Colloquy 174 (2009), http://www.law.northwestern.edu/lawreview/colloquy/2009/38/LRColl2009n38SympLaw&DevPartII.pdf.

Persistent URL: http://www.law.northwestern.edu/lawreview/colloquy/2009/38

(Comments)