RESOURCES FOR COMPARATIVE INSTITUTIONAL RESEARCH
| This two-part special issue of American Behavioral Scientist (that is, this volume and the one that preceded it in June 2002) emerged from the work of the Social Science Research Council's (SSRC) Committee on Research on Philanthropy and the Nonprofit Sector. The purpose of the committee is to help research on philanthropy and the nonprofit sector to achieve greater visibility, greater coherence, intellectual momentum, and intellectual direction. A central part of this field-building initiative is to promote the development of intellectual and material resources for studying philanthropy and the nonprofit sector. Toward this end, the committee decided early in its work to examine the quality and availability of data on nonprofit organizations and philanthropy by commissioning the essays included in this volume. |
OBJECTIVES
Several factors commended data resources as a focus of the committee's attention. First and most important, the committee identified the need for research on the following three of the fundamental questions about the nonprofit sector: (a) How do nonprofit organizations compare with governmental and forprofit organizations that perform similar tasks? That is, what are the strengths and weaknesses of nonprofits relative to other organizational forms in general, in specific subsectors such as health or social services, and in particular contexts such as highly industrialized versus developing countries? (b) What are the ways in which nonprofit organizations are held accountable to the various constituencies that have an interest in their performance? How can the effectiveness of nonprofit organizations be gauged so as to make them more accountable? and (c) What are the relationships between nonprofit organizations and democratic processes, political activism, policy initiatives, and political culture? To address fundamental questions like these, a variety of research methods and approaches are needed. One critical component that might enhance many streams of research would be the creation of intelligently designed, high-quality data sets covering large, comprehensive samples of nonprofit organizations.
Like all academics, we began with the view that we have had too little good data. It has been hard, and in some parts of the sector impossible, to answer even basic questions such as how many nonprofit entities exist and at what rates are they going out of business or coming into being. (Some observers have worried that even the best government statistics vastly underestimate the actual number of nonprofit entities.) It is difficult to find adequate data for comparing and understanding the differences among health care, arts, and educational nonprofits. It is more difficult still to find data useful for comparing nonprofit organizations to their counterparts in the for-profit and public sectors. Given the importance of assessing relative performance across sectors, this is lamentable. However, rather than simply assume the data shortfall, the committee chose to explore the data resources that are available to scholars of the nonprofit sector.1
This exploration had several ambitions. A careful look at available data might reveal many sources and kinds of data that have not yet been fully exploited. Data resources developed for one purpose might prove to be useful to researchers with other interests. Researchers with expertise in one subsector of the nonprofit world (such as day care or mental health agencies) might discover comparable data about other subsectors that might enrich their theoretical insights. The committee hoped to encourage empirical scholarship on philanthropy and the nonprofit sector. In particular, information about good data might be especially attractive for younger scholars, who ordinarily have few resources of their own for original data collection.
Reflecting on their experience, the authors of a comprehensive study of the nonprofit sector warned that "doing empirical work in the nonprofit domain requires knowledge of databases, definitions, and conventions which are largely inaccessible to neophytes" (Bowen, Thomas, Turner, & Duffy, 1994, p. 184). The essays in this special issue ameliorate this problem by offering a wealth of timesaving information for any beginning (and many established) researchers who contemplate work in this area.
Questions about the kinds of high-quality data that are available, and the kinds that are not, are of scholarly as well as practical interest. Information is an important type of public good (Lessig, 2001). Public goods are goods (or services) that, once purchased by one individual or group, can be enjoyed by others at no cost. (Common examples are fresh air or lighthouses.) It is inefficient (and usually infeasible) to exclude those others from enjoying the benefits. Ordinary markets fail to produce the right amount of public goods because individuals are tempted to "free ride"-that is, to hold back their contributions in the hope that someone else will pay the freight. In light of this market failure, it usually falls to government to provide public goods through the instruments of majority voting and compulsory taxation.
As Weisbrod (1988) has shown, however, nonprofit organizations themselves play a role in providing public goods. When a majority of voters desires a good (e.g., public schools or fire departments), government supplies it, using the power to tax to ensure that everyone pays a share. Nonprofit organizations produce public goods under the following two conditions: when some people want more of a public good than voters will pay for, and are willing to cover the difference (for example, when community educational foundations raise funds for public schools in districts in which tax revolts have stripped away public support); and when policy makers believe that government can supply public goods more effectively by paying nonprofit organizations to produce them than by producing them itself (Weisbrod, 1988).
High-quality, readily accessible, empirical data are public goods. The presence of such data improves the quality and quantity of research (by lowering the cost of undertaking it). In that way, accessible, high-quality data contribute to the production of knowledge about the sector and to the recruitment of scholars to a field. And, as is the case of other public goods, government and nonprofit organizations have often taken the lead in data production.
Data about nonprofit organizations may also be private goods, however. They are often produced by the organizations themselves or interest groups with a clear goal in mind of improving the management of organizations, increasing the funding for certain purposes, monitoring behavior of interest to key constituencies, or holding organizations accountable for performance on specified dimensions, and so forth. The costs of data collection and analysis are incurred by these groups in the expectation that the data will serve their immediate purposes. These data often do not become available to researchers, although they offer important windows into the functioning and performance of significant groups of nonprofit organizations.
In other words, not only are nonprofit data crucial to nonprofit scholarship but questions about their supply lead to a better understanding of the political economy of the sector. We invited the contributors to these two issues not just to describe the data that are and are not available in their fields but to try to explain why the gaps in the data exist. The identification of missing streams of evidence is helpful; it allows us to understand more fully the ways in which our knowledge of the sector is incomplete. It is also a first step to repairing the most critical omissions.
We hope that the articles in this issue, which inventory existing data and set priorities for collecting new and more useful information, can contribute to the long-term project of improving the data to which scholars and policy makers have access. We also hope that these volumes will be useful reference guides for the scholarly community, stimuli to the field of research in nonprofit studies, and contributions of their own to scholarship in the political economy of information.
CONTENTS
Defining the field of nonprofit research is elusive. Any scholarly area that organizes itself around a set of institutions and practices, rather than around a theoretical perspective, is inevitably heterogeneous to the extent that it faithfully reflects the heterogeneity of the institutions that are the defining focus. To address the multidimensional character of the nonprofit sector, the editorial team itself is multidisciplinary, with training in economics, psychology, and sociology. Contributors to the volumes come from these disciplines and from history and political science as well. Most of the authors have studied more than one nonprofit subsector and have used data ranging from industry-wide sample surveys to paper archives to large-scale federal databases. This ecumenical character, typical of SSRC projects, is a strength of this special issue and of the nonprofit field as a whole.
The present volume includes four articles. Whereas the first volume of the special issue focused on particular industries, the articles in this volume address data that cross-cut industry boundaries. As David C. Hammack describes, historical data can be found in a wide range of physical documents (court cases, proceedings of government bodies, organizational records, personal memoirs) scattered across libraries and other collections all over the world. Historical research is challenging not only because of the elusiveness of materials but also because issues of boundary-setting and comparability are complicated by overtime change in the legal definitions and popular understanding of philanthropy and nonproprietary enterprise (Hall, 1992; Hammack, 1998). Hammack both reviews this history and describes the kinds of sources available to scholars who wish to contribute to its study.
A major problem vexing students of contemporary nonprofit organizations who wish to do comparative work across sectors has been lack of data sufficiently similar in content and sample design to be suitable for comparison. Even researchers with the wherewithal to conduct their own surveys have had difficulty finding organizations to compare through means sufficiently standardized that samples drawn for different industries would be comparable. Two articles address this issue by focusing on particular sources of data that may help to make comparative research more fruitful.
Linda M. Lampkin and Elizabeth T. Boris describe the availability of data from the Internal Revenue Service's (IRS) Form 990, filed by all charitable entities incorporated under section 501(c)(3) of the Internal Revenue Code and having revenues of greater than $25,000 per year. (Private foundations are required to file similar forms.) Except for churches, which are not required to file, these forms cover all the familiar "nonprofit" types, from street-level advocacy groups to mammoth research universities. Students of the nonprofit sector have long looked to the IRS data as a basis for generalization about basic financial parameters of the sector and for creating sample frames for more detailed studies. But until recently, data from 990s were often plagued by such problems as inconsistent reporting, misclassification of organizations by mission, and failure to identify defunct entities. (In part, these deficiencies reflected the purposes for which the data were collected, as such errors would have neither tax nor enforcement consequences.) In recent years, owing to the efforts of IRS and the National Center for Charitable Statistics (efforts in which the authors have been deeply involved) and to the dramatic increase in public scrutiny of 990 data (and the recognition by nonprofits of the importance of accuracy that this scrutiny has inspired), the quality and availability of these data have much improved so that they now offer considerable advantages for certain kinds of research. The authors describe the forms in which data from IRS 990s are now available, the uses to which they have been put, and the difficulties that remain.
In their article, Lester M. Salamon and Sarah Dewees describe the advantages of using an entirely different source of data for describing the scope, growth, and function of the nonprofit sector: employment data. Like government, most nonprofit organizations are labor-intensive. This makes data on payments to workers not a bad approximation for the economic concept of "valueadded" to national income. (To be sure, nonprofit organizations use many volunteers, but the omission of volunteers does not weaken employment data any more than it does 990 data.) Among the advantages of basing measures of nonprofit activity on employment, the authors argue, is that the magnitude of activity for an organization can be divided by establishment and does not have to be aggregated for the entire organization. For nonprofits with numerous offices, this will be significant. Among other things, this approach lessens the bias that arises when all activity is ascribed to each entity's headquarters city. Another advantage offered by employment data is the enhanced ability it offers through parallel data to compare nonprofit to government and for-profit entities in the same sectors.
A critically important step in comparative research is identifying a population of organizations from which to sample. To do this, researchers need solid data on the virtues and limitations of different approaches to creating a sample frame, to develop cost-efficient portfolios of sources that complement one another's strengths. Gathering data that are sufficiently complete to use as a yardstick against which to measure the effectiveness of various approaches requires Herculean effort. Few such studies have been undertaken (but see DiMaggio, Kaple, Rivkin-Fish, Louch, & Morris, 1998; Gronbjerg, 1989; Kalleberg, Marsden, Aldrich, & Cassell, 1990). In the last chapter, Kirsten GrOnbjerg offers one such effort-a thorough assessment of the adequacy of conventional data sources, using her state of Indiana as the test case. Employing a variety of data-sniffing techniques to uncover, compare, and verify information on individual organizations, she produces some surprising-and in some cases shocking-revelations about how good the current sources are. Perhaps most notable is her finding that a researcher relying on IRS Form 990 data alone would have missed fully 40% of all nonprofit organizations in Indiana. Through an ingenious combination of approaches and painstaking attention to detail, she is able to provide the most complete picture of one state's nonprofit sector. In so doing, she suggests strategies for users of current data as well as directions for the further development of data sources.
CHALLENGES
WHAT ACADEMIC RESEARCHERS WANT
In some impossible dream world, scholars of the nonprofit sector might wish for data about the nonprofit sector that meet many criteria. The data would be relevant to the questions of concern to scholars. They would be accurate. They would be available immediately after the events they record. They would be easy to link with other statistics or administrative records about related phenomena. They would be easy to disaggregate to small units of analysis or to aggregate to consider larger units. They would be fully comprehensive of all corners of the nonprofit sector, and they would be cheap and simple to access. In this section, we consider the gaps between the research community's wishes and the reality of the data about the sector that are now available.
Although scholars of nonprofit organizations range across a very broad agenda of theoretical and empirical questions, some of the most fundamental questions of the field can be suggested (DiMaggio & Anheier, 1990). First is the question of origins. Why do nonprofit organizations exist in the first place? One can ask this in general, as do Hansmann (1980) and Weisbrod (1988), or with respect to particular national societies, industries, or service areas. Historians tend to seek origins in long data series that support narrative explanations based on archival sources (Hall, 1984). By contrast, many social scientists prefer comparative data (from different countries or industries or states) that permit one to test hypotheses about the conditions under which nonprofit sectors flourish and decline (James, 1993; Salamon & Anheier, 1998).
A second issue that has intrigued academic researchers is whether we observe behavioral differences between nonprofit entities and government or for-profit entities that do much the same work (DiMaggio & Anheier, 1990; Weisbrod, 1988). Such research debates whether for-profit firms are more efficient than their nonprofit counterparts, whether (and under what conditions) nonprofits produce higher quality services, and when responsibility for service to various groups shifts from public to nonprofit service providers (Smith & Lipsky, 1993).
Third, many academics have an interest in the impact of nonprofit organizations on democratic institutions. Through their various functions, nonprofit organizations play several important, albeit sometimes subtle, political roles. Nonprofits are an important source of voice and diversity in a democratic polity (Brody, 1997; Douglas, 1983; Simon, 1978) and have historically played an important role in social movements struggling for social change. Some nonprofits provide public goods, acting as quasi-governmental entities (Weisbrod, 1988). Nonprofits have also played a role as a means by which the wealthy circumvent the democratic order or co-opt dissent (Jenkins, 1998; Odendahl, 1990).
Related to questions about the nonprofit sector's impact on the distribution of power and participation are questions about its impact on the distribution of income or collectively produced benefits. Although social scientists have addressed such distributional questions using data on the revenues and expenditures of nonprofits (Clotfelter, 1992), the available data are far from adequate to answer these questions satisfactorily. With regard to the sector's effect on power in society, measurement is much more difficult still.
Finally, academic researchers ask questions about the nature and extent of change over time in the size, forms, and function of the sector. As the economy and public sector have changed, as the population grows and shifts in composition, as technology has altered the kinds of services offered by nonprofit organizations, scholars seek to monitor the changes in the sector itself. These involve tracking and explaining both the emergence of new forms and functions for nonprofit organizations and the increased blurring of boundaries across sectors.
WHAT RESEARCHERS GET
Many factors frustrate the quest for good data on nonprofit organizations that might meet the dream-world criteria articulated above. First is the difficulty of agreement on definitions that are consistent, comparable across domains within the sector, comparable across sectoral boundaries, and comparable over time and place. Institutional factors make it challenging to collect representative data on some activities of nonprofit organizations at all because anomalies of categorization and organizational form render certain service providers invisible, or nearly so.
Second, data are costly and funds are scarce. Moreover, it is most expensive to gather data on the nonprofit organizations and activities we know least about because their staff are least well equipped to provide it. The government agencies and private interests who are most willing and able to pay the expense of data collection do so for their own reasons. Overlap between those interests, questions, and concerns and those of nonprofit researchers is often purely coincidental.
Third, for each mode of collecting data, some organizations and individuals will be unwilling to cooperate in providing full and accurate answers. Resistance to data collection limits the data that nonprofits are willing to provide and the accuracy of much data that are collected (Weiss & Gruber, 1987). We discuss each of these problems in turn.
PROBLEMS OF DEFINITIONS AND COMPARABILITY
Academic researchers are constitutionally inclined to worry about how to define the "nonprofit organization." Whereas policy makers are content to define a "nonprofit organization" as an entity that qualifies as tax-exempt under section 501 (c)(3) of the Internal Revenue Code, academics may choose to define nonprofit on the basis of function, structure, or public perception. Other definitional debates of interest to academics include the question of whether the "nonprofit sector" is a coherent subject of inquiry at all or the extent to which "nonprofit sectors" can be compared in a meaningful way across countries with different political, legal, and economic arrangements (Salamon & Anheier, 1998).
Other definitional challenges include asking questions, whether about funding, programs, or structure, in ways that define exactly what is to be measured and how, getting agreement on measures that can mean the same thing across astonishingly heterogeneous organizations, and providing unambiguous guidance about the time frame for generating responses. When the entities reporting data fail to use comparable measures, aggregate in comparable ways, or use the same time frame, then the information generated is neither valid nor reliable.
Standardized data systems rely on commonality of responses from different actors, and all comparability is grounded in underlying systems of social classification. When social categories are ambiguous, or when the same categories mean different things in different fields, collecting standardized, comparable data becomes difficult. Take for example the distinction between "public" and "private," which is both fundamental and seems as if it should be clear and transparent. In the field of higher education, nonprofit colleges and universities are usually called "private" (in distinction to government-supported public institutions). In day care and social services, nonprofit organizations are often classified as "public" (in distinction to for-profit providers). Such differences wreak havoc with comparison across data sources.
Systems of social classification, including the ones we use to design research instruments, thrive on clarity and find anomalies repulsive (Zerubavel, 1997). Yet, the industries in which nonprofits operate are filled with organizational hermaphrodites: entities with characteristics of organizations from more than one sector, such as for-profit hospitals owned by nonprofit chains, public universities with nonprofit foundations, or museums in which public agencies own the building and private nonprofits control the collections. (Before the Depression, for example, the Metropolitan Opera was not one entity at all but rather a collaboration between a nonprofit producing organization and a for-profit real-estate company, both of which were run by more or less the same cast of characters.) Mixed types were typical in 19th-century America (Hall, 1992), and they have proliferated again at the end of the 21 st century. Mixed cases render comparison treacherous and unreliable.
Much research interest in the nonprofit sector revolves around comparing the characteristics and performance of nonprofit and for-profit firms in the same industries. Yet, the definition of "industry" is to some extent a matter of convention, and convention tends to dictate the way in which data are collected. For that reason, existing data make it easier to compare prestigious nonprofit art museums to government-sponsored aquariums to small historic-house museums than it is to compare any of them to for-profit theme parks, although many observers would contend that the latter comparison is at least as germane.
Collection of comparable data is complicated even further by the fact that firms in different sectors may undertake the same functional activities using substantially different organizational forms. For example, artists in for-profit performing arts are ordinarily connected to venues by short-term contract (such as jazz bands that perform in nightclubs), whereas artists in the nonprofit sector are more frequently salaried employees (such as musicians in symphony orchestras). Higher education in the government and nonprofit sectors takes place primarily in colleges and universities; higher education elsewhere in America often takes place in corporate conference facilities or vocational programs that lack the label of "higher education" or separate institutional status.
Indeed, many kinds of comparison are rendered difficult by systematic differences in the ways in which functions clump together into organizations in different sectors. Universities, for example, do many things that for-profits do in the outside world. For example, they present concerts and film series, serve meals, run golf courses, administer student loans, and offer health care services. None of these functions are central purposes of universities, even though they often do them on a very large scale. Yet universities are typically missing from the sample frames in studies of performing-arts organizations, food-service providers, economic development agencies, banks, and health care organizations.
Related to this is an additional problem: You cannot count organizations until you institutionalize them, that is until they fit into a commonly recognized category with standardized attributes, a service organization or trade association, and a largely known membership. Take, for example, the case of for-profit art museums. Not counting corporate art collections open to the public, there are probably a couple of hundred of these museums (the best known being South Dakota's Wall Museum of Western Art). Yet, they are statistically invisible because, in the late 1970s, the Federal Institute for Museum Services endorsed the American Association of Museum's definition of a museum as a "nonprofit organization" dedicated to public exhibition (DiMaggio, 1987). Also largely invisible are emergent organizational forms, such as birthing centers or freestanding hospices in the health care sector. Finally, in some largely nonprofit fields, productive units are too small for incorporation to be cost-efficient: Such entities may operate as unincorporated partnerships, such as chamber trios, neighborhood block watch groups, home schooling networks, or baby-sitting cooperatives. All have fluid boundaries and lack clear demarcations of their organizational birth or death. Or small programs may form under the shadow of larger entities such as community centers. Such entities may do the same kind of work as their larger incorporated brethren, but they fly under the radar of systematic data collection.
Such institutional realities make comparison difficult, if not impossible, in many fields. They also render data sets on many nonprofit industries chronically biased and incomplete. Obviously, these problems are much more serious in fields such as the arts, social advocacy, or social services, where entities are relatively small, programs diverse, and organizational arrangements heterogeneous, than in fields such as philanthropic foundations, where definitions are legally prescribed and organizations highly regulated. And they are less burdensome on researchers studying nominally defined organizational types (for example, universities or hospitals) than for researchers interested in the roles played by nonprofits in functional areas (for example, higher education or health care). But it is important to be aware of the issues. Under a surprising number of scenarios, such problems render data sets distorting mirrors that track changes in cultural representations more accurately than they track change in organizational populations themselves.
PROBLEMS OF COST
A second challenge to the collection of good data on nonprofit organizations is the cost. Data collection is costly in at least three ways. First, collecting data is expensive, especially if one attends to the accuracy, replicability, comprehensiveness, and timeliness of the data. Many data-collection efforts require difficult trade-offs between costs and data quality. For example, collecting data on the entire population of nonprofit organizations providing a service is far more expensive than collecting data on the 40% of the organizations willing to answer requests for data the first or second time they are asked. Getting initial nonrespondents to change their minds and cooperate is essential to the research value of the data but labor-intensive and expensive. Second, the marginal cost of increasing the validity and reliability of data that are collected may be high because it may require that researchers spend time helping respondents answer questions that they do not fully understand or that do not fit their circumstances in an obvious way. This is especially the case for small nonprofit organizations, which seldom employ staff with specialized financial or managerial skills. (This is one reason why many studies systematically underrepresent small organizations.) Third, quality control-for example, cross-checking results and recontacting respondents whose responses lack plausibility-is also costly.
The scarcity of funds endemic to any research enterprise is exacerbated in work on nonprofit organizations. Especially when research is paid for by government or private foundations, nonprofit organizations may perceive it as competing for resources with service provision and resist it on those grounds. Many research sponsors are sensitive to such criticisms and, in some parts of the sector, often unsophisticated about research method. The result is chronic underinvestment in quality so that many research dollars produce data sets so flawed as to be of virtually no value to the research community or to policy makers desirous of reliable information to inform their decisions.
Moreover, data collection imposes reporting costs on nonprofit organizations, which must devote staff time to maintaining information systems capable of providing mandated data (Gormley & Weimer, 1999; Weiss & Gruber, 1984). These costs fall most heavily on smaller organizations, with less capability and limited expertise in information systems. When data collection is mandated by government, these costs can be a significant burden on respondents, leading to nominal compliance but, in truth, sloppy or incomplete reporting. When data collection is initiated by a professional association, a university researcher, or an advocacy group, voluntary data provision is likely to depend in part on the costs of compliance with the requests for data. Requests that entail significant costs for respondents are unlikely to yield data from many respondents. The more comprehensive and timely the data, the more likely that the data will be costly to collect.
Because of the costs involved, many of the data sets described in these two volumes are collected by government agencies or professional associations. Aside from a handful of federal statistical agencies (such as the Census Bureau or the National Center for Education Statistics), government data gatherers usually collect information for legal or policy purposes of monitoring and control (Weiss & Gruber, 1984). The IRS requires the submission of information on Form 990 not out of any general curiosity about the differences between organizational forms but because the data are intended to provide accountability for compliance with the tax code. Nonprofit professional and trade associations gather data about the income-producing activities or human-resources policies of their members not to answer scholarly questions but to serve the management or political needs of their dues-paying members. Such data are shared to establish benchmarks for administrators' salaries, for example, or to track the demand for government aid.
Indeed, it is probably fair to say that government's primary motivation for collecting data on nonprofit organizations is the desire to make them accountable for the benefits that they receive and the quality of the services they provide. At the extreme (as when states' attorneys general collect information on fundraising practices), the motive is to prevent abuse of citizens. By the same token, some of the impetus for data gathering within the nonprofit sector is probably the avoidance of accountability, or at least the avoidance of regulatory intervention.
The result of this dynamic is that data collection is shaped to a significant degree by the governmental agenda. In health care, data collection is overwhelmingly focused on costs, with some attention to treatment efficacy for undertaking cost-benefit analysis. In higher education and the social services, in which treatment efficacy is harder to measure, policy makers have been concerned with efficiency (such as caseload per staff member) and access to services (such as tuition costs). In the arts, in which decisions about quality are often made by peer review, policy makers have ordinarily sought financial or managerial data to assess whether institutions are financially sound and can sustain their service provision over time.
In some cases, data are created to serve the needs of paying clients or beneficiaries of nonprofit organizations. Examples include guides to undergraduate education sold to prospective students and their families, consumers' guides to nursing home quality, or report cards on health maintenance organizations for the benefit of employers purchasing health insurance. In these cases, the costs of data collection are willingly incurred by clients to guide significant investments they make in the services of the nonprofit organizations. Such data-collection enterprises, naturally, focus on those aspects of nonprofit organizations of interest to clients, such as ease of access, prices charged, complaints, and client outcomes. Nonprofit organizations may cooperate with this kind of data-collection enterprise in part because participation helps them to attract clients (Gormley & Weimer, 1999).
Scholars can find common ground with other data collectors when research questions coincide with the interests that led to data collection. The interests of government, clients, and interest groups are not completely irrelevant to the questions that scholars care about. In particular, issues of cost and efficiency are most likely to surface in data collected for many different reasons. Many datacollection efforts are also concerned in one way or another with the question of client outcomes or satisfaction. However, the bulk of data about the nonprofit sector available to researchers are the by-product of data collected to suit the agendas of groups that are able to pay the costs of data collection.
PROBLEMS OF RESISTANCE
Even if financial constraints did not limit our ability to produce adequate data on the nonprofit sector, political resistance would make doing so difficult (Starr, 1987). Two kinds of political considerations have an impact on the data that become available to scholars and policy makers. First, the fact that information often threatens vested interests produces a countermobilization aimed at restricting the collection of information. Second, even when participants are supportive of data-collection efforts, they must find the political means to overcome the collective action problem that inheres in coordinating multiple actors to support and contribute to the production of usable data.
At least two kinds of political interests are at stake in efforts to collect organizational data. First, all bureaucracies (and almost all contemporary organizations, even nonprofits, are bureaucracies in the broadest sense) attempt to control information about their own internal processes as completely as they can, letting it out only when they must or when they feel that it serves their interests. This is especially the case for organizations whose activities are regulated, and for organizations that face policy decisions (for example, choices about eligibility for federal contracts) that may have a decisive impact on their well-being.
When organizations cannot prevent data collection, they try to control it. One way to control it is by misreporting data in a way to make the organization appear as though it is in compliance with the preferences of its stakeholders, even when it is not. This tactic, when widely used, leads to serious deterioration in the validity of data systems. Another way to control data collection is by exerting political pressure, usually through their own trade associations, on elected officials or government agencies, to avoid questions about the most sensitive data and to ensure that other information will be treated in the manner most flattering to the industry they represent.
Nonprofit organizations often have enough clout to shape the data they provide about themselves in ways that undermine the intention to hold them accountable. To ensure that policy makers end up with the information they want, government data collectors need to ensure that the data they collect are accurate, complete, and timely. Because those nonprofit managers who control the information may have incentives to distort or conceal their reports, the collection of information nearly always needs some provision for monitoring to ensure that information is provided as intended. Monitoring, auditing, and checking of information collection may involve antagonizing those who provide the information and, hence, may raise political opposition. In the United States, political values legitimize resistance to government information collection; this has led to extensive legal and budgetary restrictions on direct collection of information by government (Weiss, 1989). These political constraints may make it impossible to get the best possible information for the policy purpose and force policy makers to rely on information that is less costly to obtain but also less valuable for policy purposes.
Second, political interests outside the nonprofit sector may also impede the data-collection process. Government officials with a strong ideological agenda (for example, for or against government funding of religious nonprofits) may be highly motivated to avoid collecting information that could assist their opponents in policy contention. Collection of data on nonprofit organizations may be constrained, as well, by apparently unrelated political agendas. For example, political sensitivities combined with the concern of the Congress and the White House's Office of Management and Budget with "paperwork reduction" have inhibited statistical agencies in health and education from gathering data necessary to distinguish between secular and religious nonprofits.
Such political considerations may prevent government from establishing systematic data-collection systems; or, if such systems are established, they may constrain data collection to those elements less threatening to influential political groups. What is left may be information that, although useful for some purposes, is basically incapable of addressing the most pressing questions with which policy makers are concerned. In an essay on federal data-collection efforts on elementary and secondary education, Weiss and Gruber (1987) referred to that system's "managed irrelevance," an apt phrase that may apply equally well to some data on the nonprofit sector.
Even if nonprofits and funders share an interest in producing useful information, how can they organize themselves to find the money and voluntary cooperation to produce it? Where state legislatures or Congress have mandated the provision of information, as in the IRS 990s and some health and education data, cooperation is less problematic, though data accuracy and incentives for misreporting become urgent concerns. This is true of the two sectorwide data systems described in this volume (the IRS 990s and the state employment data that Salamon and Dewees discuss), and also of major information systems in health, higher education, and the arts.
Where government has not taken the lead, potential users face a collective-- action dilemma, leading to chronic undersupply. The proprietary solution, in which specialized businesses collect data for which there is demand and charge users for access to tightly controlled databases, is unattractive to academics. As Starr (1987) noted, it leads to highly selective data availability based on the interests of those who can pay. In some cases, private foundations have undertaken an essentially statelike role, funding and motivating major data-collection efforts. In other cases, the collective-action dilemma is resolved when nonprofits band together defensively in associations (often in the face of government regulation) and produce their own data, using a combination of moral suasion and side payments (for example, special reports comparing respondents to fieldwide benchmarks) to induce cooperation. This latter is a second-best solution as such efforts often avoid asking questions that might yield embarrassing answers, usually elicit only imperfect cooperation, and often restrict access to the data that are collected.
It should be noted that political processes can generate, rather than discourage, the collection of more data. Many of the data sets described in the articles in this special issue have been collected by nonprofit associations and advocacy groups, often in response to perceived political threat. Government datacollection efforts can provoke competing data-collection efforts by groups with differing interests and agendas. The more groups with a stake in the issues, the more potential data collectors. In significant respects, the availability of data about the nonprofit sector is attributable to the expansion of the modern state and to the proliferation of identities and interests that state expansion produced.
TOWARD BETTER DATA
During the first part of the 20th century, American political and intellectual elites associated with the Progressive Movement developed an optimistic view of the capacity of research to improve government policy and practice (Lindblom, 1990). This position rested on two major premises. First, problems were soluble. Political dispute was over the means to their solution rather than over the ends to be sought. Second, social science could serve the quest for solutions by undertaking surveys of relevant facts and analyzing the results. Facts, and the categories into which researchers placed them, were good, and social scientists were understood to be primarily interested in the application of knowledge to social ills.
Drawing on the contributions to this special volume, we have identified a number of ways in which data about the nonprofit sector deviate from this rosy scenario. Even among themselves, researchers do not agree about the facts and ask different questions of the same body of data. Researchers, government officials, and nonprofit leaders interpret data in different ways, as they ask different questions about the sector. Cost considerations limit the amount and quality of information available. The allocation of money, time, and cooperation to data collection is inextricably enmeshed in political agendas.
Despite these observations, we are not pessimistic about the prospects for improving our knowledge of the nonprofit sector in ways that enhance both our scientific understanding and the capacity of other actors to play their own roles in improving performance in the nonprofit sector. Students of social policy have known for a long time that applying data to policy dilemmas is a lot harder than the Progressives thought it would be (Lindblom & Cohen, 1979). In the articles in this special issue (both this volume and the preceding one), we see reasons for optimism.
First of all, the articles in the previous volume demonstrate improvements at the industry level. After many years of deliberate neglect, high-quality databases on religious organizations, for example, became available for the first time in the 1990s, ushering in a renaissance of research on churches and congregations. In the arts, we see more than 20 years of effort by the National Endowment for the Arts Research Division finally culminating in a "Unified Database" on nonprofit arts organizations far more inclusive than anything imaginable a decade ago. And in both of these domains, new digital initiatives are placing a wide range of documented databases online, thus dramatically increasing the availability of data and reducing barriers to entry into the research field. Even in the more mature fields of health care and higher education, we see continuing progress as data designers grapple with the challenge of finding more valid and discriminating measures of activities and performance.
Second, as the articles in this volume demonstrate, we are experiencing marked improvement in the processes by which regularly collected administrative data may be transformed into information of value for scholarship. Years of collaboration between IRS and the National Center for Charitable Statistics have led to improvements in the collection and processing of Form 990s, thus turning a huge set of administrative records into a usable database of incomparable value to researchers. With modest prodding and improvement, other administrative data (for example, state employment records) promise to fill additional gaps in our ability to track change in the composition and activities of the nonprofit sector.
Third, as the multidisciplinary field of nonprofit studies continues to develop, we see efforts to harness research and theory to develop better strategies for increasing the amount and quality of policy-relevant data and for making the most of the data that are available. Gronbjerg reports on a truly remarkable study of the strengths and weaknesses of different sources of data for identifying the population of nonprofit organizations in the state of Indiana. Although it highlights shortcomings of conventional data, such work is valuable for scholars attempting to construct inclusive sample frames and for data specialists concerned with making their own systems more accurate and comprehensive.
In this introduction, we have attempted to contribute to the study of the ways that economic, political, and institutional factors shape the nature and quality of publicly available data (Alonso & Starr, 1987). In doing so, we have drawn on, and attempted to provide a framework for, the more specific analyses presented in the chapters in these volumes. We have explored the ways in which institutional factors complicate the collection of data even when the will and means are present. Although presenting formal hypotheses about the relationship between such factors as industrial organization and regulatory regimes, on one hand, and data availability, on the other, is beyond the scope of this article, doing so would be a natural next step. It stands to reason that a sophisticated, theoretically grounded understanding of such factors would help policy makers and scholars anticipate (and overcome) the obstacles they confront in designing more effective strategies for improving data systems.
In sum, then, nonprofit scholars have much data with which to produce research, albeit not the data of their dreams. The nonprofit sector is continually changing in ways that definitions embedded in data-collection systems do not reflect. Many data systems are inconsistent in their definition of the sector, and many are vague or unreliable in their measurement of organizational form. At the same time, data resources are richer than most scholars recognize. There is more scope for making comparisons across subsectors than researchers have thus far exploited. And there are unrealized opportunities for exploring the division of labor across sectors and across different kinds of institutions.
This special issue is a step in the direction of building the field of nonprofit studies. Those who collect and assemble data may be enticed to measure sectoral status in more useful and flexible ways. We hope that the articles will make scholars in several academic disciplines more aware of the provocative questions and opportunities for research in the nonprofit sector. We hope that the debates will give impetus to conversations about gaps in the data record, and steps to fill them in. Perhaps, best of all, some new participants may choose to join us in this enterprise.
| [Footnote] |
| NOTE |
| [Footnote] |
| 1. Owing to the efforts of a few pioneers, valuable data on the nonprofit sector does exist, and they have proven valuable for scores of empirical studies. The best known compendium of aggregate data is Hodgkinson, Weitzman, Abrahams, Crutchfield, and Stevenson (1996). Combining data from government, trade groups, and household surveys, these compilations are largely responsible for our understanding of the scope and diversity of what scholars have come to understand as "the nonprofit sector." |
| [Reference] |
| REFERENCES |
| [Reference] |
| Alonso, W., & Starr, P. (EDs.). (1987). THe politics of numbers. New York: Russell Sage. |
| Bowen, W. G., Thomas, I. N., Turner, S. E., & Duffy, E. A. (1994). The charitable nonprofits. San Francisco: Jossey-Bass. |
| Brody, E. (1997). Charitable endowments and the democratization of dynasty. Arizona Law Review, 39, 873-948. |
| Clotfelter, C. (Ed.). (1992) Who benefits from the nonprofit sector. Chicago: University of Chicago Press. |
| [Reference] |
| DiMaggio, P. (1987). Nonprofit organizations in the production and distribution of culture. In W. W. Powell (Ed.), The nonprofit sector: A research handbook (pp. 195-220). New Haven, CT: Yale University Press. |
| DiMaggio, P., & Anheier, H. K. (1990). The sociology of nonprofit organizations and sectors. Annual Review of Sociology, 16, 137-159. |
| DiMaggio, P., Kaple, D., Rivkin-Fish, S.. Louch, H., & Morris, L. (1998). Comparing sample frames for research on arts organizations: Results of a study in three metropolitan areas. Journal of Arts Management, Law and Society, 28, 41-66. |
| Douglas, J. (1983). Why charity: The case for a third sector? Beverly Hills, CA: Sage. |
| Gormley, W. T., & Weimer, D. L. (1999). Organizational report cards. Cambridge, MA: Harvard University Press. |
| Gronbjerg, K. A. (1989). Developing a universe of nonprofit organizations: Methodological considerations. Nonprofit and Voluntary Sector Quarterly, 18, 63-80. |
| Hall, P. D. (1984). The organization ofAmerican culture, 1700-1900. New York: New York University Press. |
| Hall, P. D. (1992). Inventing the nonprofit sector and other essays on philanthropy, voluntarism, and nonprofit organizations. Baltimore: Johns Hopkins University Press. |
| [Reference] |
| Hammack, D. C. (Ed.). (1998). Making the nonprofit sector in the United States: A reader. Bloomington: Indiana University Press. |
| Hansmann, H. (1980). The role of nonprofit enterprise. Yale Law Journal, 89, 835-901. Hodgkinson, V., Weitzman, M. S., Abrahams, J. A., Crutchfield, E. A., & Stevenson, J. S. (1996). The nonprofit almanac: Dimensions of the independent sector, 1996-1997. San Francisco: Jossey-Bass. |
| [Reference] |
| James, E. (1993). Why do different countries choose a different public-private mix of educational services? Journal of Human Resources, 28(3), 571-592. |
| Jenkins, J. C. (1998). Channeling social protest: Foundation patronage of contemporary social movements. In W. Powell & E. Clemens (Eds.), Private action and the public good (pp. 206216). New Haven, CT: Yale University Press. |
| Kalleberg, A., Marsden, P. V., Aldrich, H. E., & Cassell, J. W. (1990). Comparing organizational sampling frames. Administrative Science Quarterly, 35, 658-688. |
| Lessig, L. (2001). The future of ideas: The fate of the commons in a connected world. New York: Random House. |
| [Reference] |
| Lindblom, E. E. (1990). Inquiry and change: The troubled attempt to understand and shape society. New Haven, CT: Yale University Press. |
| Lindblom, E. E., & Cohen, D. K. (1979). Usable knowledge. New Haven, CT: Yale University Press. Odendahl, T. (1990). Charity begins at home: Generosity and self-interest among the philanthropic elite. New York: Basic Books. |
| Salamon, L., & Anheier, H. (1998). Social origins of civil society: Explaining the nonprofit sector cross-nationally. Voluntas, 9, 213-248. |
| Simon, J. (1978, Summer). Charity and dynasty under the federal tax system [Special issue]. The Probate Lawyer. |
| [Reference] |
| Smith, S. R., & Lipsky, M. (1993). Nonprofits for hire: The welfare state in the age of contracting. Cambridge, MA: Harvard University Press. |
| Starr, P. (1987). The sociology of official statistics. In W. Alonso & P. Starr (Eds.), The politics of numbers (pp. 7-57). New York: Russell Sage. |
| Weisbrod, B. A. (1988). The nonprofit economy. Cambridge, MA: Harvard University Press. Weiss, J. (1989). The powers of problem definition: The case of government paperwork. Policy Sciences, 22, 97-121. |
| Weiss, J., & Gruber, J. (1984). Deterring discrimination with data. Policy Sciences, 17, 49-66. Weiss, J., & Gruber, J. (1987). The managed irrelevance of federal education statistics. In W. Alonso & P. Starr (Eds.), The politics of numbers (pp. 363-391). New York: Russell Sage. |
| Zerubavel, E. (1997). Social mindscapes. Cambridge, MA: Harvard University Press. |
| [Author Affiliation] |
| PAUL J. DIMAGGIO |
| Princeton University |
| JANET A. WEISS |
| University of Michigan |
| CHARLES T. CLOTFELTER |
| Duke University |
| [Author Affiliation] |
| Authors' Note: For support and assistance in this project, the editors are grateful to the other members of the Committee on Research on Philanthropy and the Nonprofit Sector and to its chair. Burton Weisbrod; to Steven Heydemann, Aaron Beebe, and Ellen Perecman of the Social Science Research Council; to the committee's first chair, Ellen Lagemann; and to Laura Lawrie of the American Behavioral Scientist |
| [Author Affiliation] |
| CHARLES T. CLOTFELTER is Z. Smith Reynolds Professor of Public Policy Studies and professor of economics and law at Duke University. He is also director of the Center for the Study of Philanthropy and Voluntarism at Duke and is a research associate of the National Bureau of Economic Research. His research has covered the economics of education, public finance, the economics of gambling and state lotteries, tax policy and charitable behavior, and policies related to the nonprofit sector. Currently, he is working on a study of school desegregation since 1954, the market for and allocation of teachers in public schools, and the excise taxation of gambling. |
| [Author Affiliation] |
| PAUL J. DIMAGGIO is professor of sociology at Princeton University and research director of the Princeton University Center for Arts and Cultural Policy Studies, which he cofounded with Stanley Katz. A former executive director of the Yale University Program on Non-Profit Organizations, he has written extensively on nonprofit organizations, organization theory, and the sociology of |
| [Author Affiliation] |
| culture. His current research is on patterns of cultural conflict and consensus in the United States since 1965 and on inequality in access to and use of new information technologies. |
| [Author Affiliation] |
| JANET A. WEISS is associate provost for Academic Affairs at the University of Michigan. She is the Mary C. Bromage Collegiate Professor of Organizational Behavior and Public Policy at the University of Michigan Business School. She is also professor of public policy at the Gerald R. Ford School of Public Policy, and the founding director of the Nonprofit and Public Management Center at the University of Michigan. Her research interests focus on the roles of information and ideas in the public policy process and on the relationships among the public, private, and nonprofit sectors. |
No comments:
Post a Comment