November 1, 2002 Special Issue: A Look At Innovation Indices & Report Cards

Copyright State Science & Technology Institute 2003. Information in this issue of the SSTI Weekly Digest was prepared under a cooperative agreement with the U.S. Department of Commerce, Economic Development Administration. Redistribution to all others interested in tech-based economic development is strongly encouraged — please cite the State Science & Technology Institute whenever portions are reproduced or redirected. Any opinions expressed in the Digest do not necessarily reflect the official position of the U.S. Department of Commerce.

Subscription to the SSTI Weekly Digest is free. If you are reading a forwarded copy of this issue and would like to receive your own copy each week directly, please subscribe at: http://www.ssti.org/Digest/digform.htm Requests to unsubscribe should be sent to sstiwd@ssti.org.


Why A Special Issue on Innovation Indices?
One of the more common topics in the information requests SSTI receives regards how to do or what are good examples of innovation indices or S&T report cards — compilations of statistics and data presenting a geographic area's standing against similar areas across several metrics thought to be useful guides in assessing economic health or potential for growth.

Why is that? Can it simply be dismissed as another example of how America's fascination with superlatives and comparatives pervades nearly every aspect of society? Claims for the biggest, the fastest, the richest, the first, the strongest, the top 100, etc., and the counter arguments against the claims can be found throughout our culture. Are indices the muscle-flexing machismo element of tech-based economic development?

Probably not.

When done well and often, innovation indices and science and tech report cards actually can serve several beneficial ends for the tech-based economic development community. They can:

Through past issues of the SSTI Weekly Digest, we have highlighted more than 40 various indices created by organizations, states, and local agencies. The indices vary as to content and factors that are accounted for but similarities exist. This issue is dedicated to reviewing several indices that exist at different spatial scales as well as briefly discussing the uses of these tools. We close with SSTI's recommendations for good indices and several links for more information or data for developing an index for your own state or region.

return to the top of this page

A Couple of Cautionary Notes
An important aspect of all indices, regardless of their geographic orientation, is that each is created with a different purpose or goal in mind. As a result, each index has an inherent or implicit bias toward the data collected, the manner in which the data is manipulated and presented, and the conclusions drawn or policy recommendations made.

When considering the various indices that have been prepared, it is important to be aware that differences do exist and not to think all the indices are interchangeable. Indices will have some common elements and measurements to them; however, subtle and substantial differences may also exist.

For instance, since California holds one-quarter of the nation's population, it can be expected to be number one among the states in most categories that use raw values such as population, gross state product, number of business starts, number of college graduates, etc. If the goal is to present California in such light or to highlight the difference between California and another state with less population, then use of raw numbers can be very powerful. Only using raw numbers does, however, open the index to considerable criticism.

A technique that paints a stronger and usually more useful picture is to standardize the raw values. Looking at a comparison of the percentage of college graduates in the state's workforce, the proportion of the gross state product that is represented by R&D investments, or the ratio of business starts to business failures can provide a more meaningful starting point for interpreting the results and outlining a policy of corrective action.

Indices also can include or exclude use of a weighting system within and across categories. The purpose of the weights is to recognize the relative importance of a particular metric or category toward the region's or state's technology-based economic development goals.

For instance, is the number of SBIR awards received by small businesses in a state in a given year as important to the state's economy as the increase in total technology employment? Is an area's median income equal to the total research funding received per scientist or engineer?

These are obviously subjective matters. As a result, weighting systems, or lack thereof, can account for substantial differences in the results between two otherwise similar indices. Weighting systems in themselves can be arbitrary and less useful than desired. When looking at indices, it is important to note the method used to arrive at the indicators included in the study and the manipulation of the data once included.

return to the top of this page

The Nation-wide Indices
Seven indices introduced below are followed by a sample of the similarities and differences that exist among them.

Progressive Policy Institute (PPI)
The Progressive Policy Institute provides one of the most widely used barometers of states' relative positions in technology-based economies. State New Economy Index: Benchmarking Economic Transformation in the States offers an innovation-oriented public policy framework for the states to foster success in the New Economy. PPI contends states that overhaul traditional approaches to economic development and replace them with a new approach focused on boosting skills, entrepreneurship, technology and quality of life are best prepared to prosper in the New Economy.

The 2002 index includes 21 indicators and more finely tuned measurements to assess state progress since PPI's first report in 1999. The indicators are distributed across five categories: knowledge jobs; globalization; economic dynamism and competition; the transformation to a digital economy; and technological innovation capacity.

PPI's 2002 State New Economy Index is available online at http://www.neweconomyindex.org/states/. Print versions are available through SSTI's bookstore at http://www.ssti.org/Publications/publications.htm

Office Of Technology Policy (OTP)
With the goal of assembling “a consistent set of state-level data that approximates the ‘technology infrastructure’ of the states,” the Office of Technology Policy (OTP) has released, The Dynamics of Technology-based Economic Development: State Science and Technology Indicators. OTP, part of the Technology Administration of the U.S. Department of Commerce, cautions repeatedly throughout the report that no attempt is made or intended to analyze, benchmark, or assess any state's performance along any of the 37 metrics included. Dynamics is meant to serve as a neutral reference guide for policy makers and researchers as they develop, implement, and evaluate state science and technology policies and programs. The 37 metrics are divided among five major groups: funding in-flows (10 indicators); human resources (9); capital investment and business assistance (5); technology intensity of the business base (5); and, outcome measures (8). The guide draws upon state-level data "that approximates the 'technology infrastructure' of the states, or, at the very least, compiles information about those factors that clearly affect states' capacity to generate new enterprises and high quality jobs, and sustain economic growth."

The Dynamics of Technology-based Economic Development: State Science and Technology Indicators is available at http://www.ta.doc.gov/reports/TechPolicy/StatesIndicators2.pdf

Corporation for Enterprise Development (CFED)
The Development Report Card of the States is issued annually by the Corporation for Enterprise Development (CED). CED uses more than 70 measures and data sets as indicators of the effectiveness of each state’s economic development policies in three broad categories: performance, business vitality and development capacity. Each category has a number of subunits, which consist of several individual measures. S&T indicators and those more generally related to tech-based economic development are scattered throughout the report card. The uniqueness of the CED report card is the selection and aggregation of the particular indicators, the assignment of letter grades to indicate a state’s performance for each of the three broad categories, the ease of online access to the information, and the amount of public attention the report card receives annually.

CED's Development Report Card of the States is available at http://drc.cfed.org/

Beacon Hill
A study released by the Beacon Hill Institute at Suffolk University in Boston is one of the more recent efforts to examine all aspects of U.S. states and their economies. Entitled State Competitiveness Report 2001, the study defines competitiveness as the ability to ensure and sustain a high level of per capita income and its continued growth. The BHI report combines more than three dozen variables into nine subindexes: government and fiscal policy, institutions, infrastructure, human resources, technology, finance, openness, domestic competition and environmental policy. Using the nine subindexes, each of which the Beacon Hill Institute says represents an element of competitiveness, the authors made an overall index and ranked the states according to their overall competitiveness.

The State Competitiveness Report is available at http://www.beaconhill.org/MasterDocumentSCI_A1.pdf

Milken Institute
Commissioned by TechVentures Network and the California Technology Trade and Commerce Agency's Division of Science Technology and Innovation, the Milken Institute set about to develop a series of indicators to measure the performance of California’s high technology-based economy against the rest of the country in State Technology & Science Index: Comparing and Contrasting California. The result is a set of 73 indicators arrayed across five composites: R&D inputs; risk capital and infrastructure; human capital investment; technology and science workforce; and technology concentration and dynamism. The study offers a detailed look at California’s technology position among the other 49 states. While not part of the freely available report, a separate set of Institute rankings for all 50 states shows the inventory of each state’s technology and science assets.

State Technology and Science Index: Comparing and Contrasting California is available at: http://www.milkeninstitute.org/poe.cfm?point=pub03

Southern Growth Policies Board
The Southern Innovation Index, a strategic plan created with the governments of 13 Southern states and Puerto Rico to promote innovation, entrepreneurship and economic growth in the South, was prepared by the Southern Growth Policies Board, a bipartisan public policy group based in Research Triangle Park, North Carolina. The index identifies 56 benchmarks and 10-year targets for each of the Southern Growth member states — Alabama, Arkansas, Georgia, Kentucky, Louisiana, Mississippi, Missouri, North Carolina, Oklahoma, South Carolina, Tennessee, Virginia, West Virginia and the commonwealth of Puerto Rico — to track the progress of technology and innovation initiatives in the region.

The Southern Innovation Index is available at: http://www.southern.org/main/stc/projects/2002/SII.shtml

Small Business Survival Committee
The annual Small Business Survival Index, released in July 2002 by the Small Business Survival Committee, ties together 20 major government-imposed or government related costs that impact businesses and entrepreneurs. These factors include personal income tax, health care costs, and minimum wage among many others.

The index suggests that those states scoring best in the index should be havens for small business and entrepreneurship growth. Measures of actual business start-ups and survivability, incorporated in most of the other indices listed above, suggest there is much more that goes into small business survival than selected tax rates and regulatory policies (see the August 24, 2001 issue of the SSTI Weekly Digest for a discussion of taxes and entrepreneurship). Inclusion of such information most likely runs counter to the Committee's purpose in preparing the index.

The Small Business Survival Index 2002 can be found at http://www.sbsc.org/Media/pdf/SBSI2002A.pdf

return to the top of this page

Similarities and Differences in the National Indices
Even when indices are trying to measure the same activity or characteristic this may be done utilizing different measures and some indices may include indicators not used in others. A matrix presenting the common indicators among the national indices is available here. Examples of these differences and similarities follow: (Please note that not all indicators for all indices are mentioned.)

Education
All indices use some measure of the educational attainment of the workforce or population. OTP and Milken include measures of standardized test scores. Milken and CED use measures of spending on education, while OTP, Milken, CED and Beacon Hill look specifically at students and graduates in science and engineering fields. For example, OTP uses a measure of the percent of the civilian workforce with a recent bachelor’s, masters or Ph.D. in science and engineering. CED and Beacon Hill look at the overall number of science and engineering graduate students.

Workforce Composition
All indices except Beacon Hill and CED measure the percent of employment in high-tech jobs. CED and Milken use as a measure the number of Ph.D. scientists and engineers in the workforce, while Beacon Hill and PPI look at all scientists and engineers in the labor force. PPI specifically looks at employment in information technology (IT) jobs in non-IT industries, as well as measuring managers, professional and technical jobs. Beacon Hill and CED include the unemployment rate in their indices, while OTP includes the labor force participation rate.

Business Environment
All indices include a measure of patent activity. OTP and Milken look at the number of business incubators, while CED looks at university spin-offs. All indices use a measure of the number of high-tech companies in the economy as well as measures of new businesses. OTP and Milken include measures as to the number of Technology Fast 500 and Inc. 500 companies. CED specifically includes sectoral diversity through a score based on the Herfindahl index.

R&D and Funding
All indices include a measure of industry R&D, while all but PPI and Beacon Hill look at federal and university R&D dollars. Milken includes specific measures on R&D expenditures on engineering, physical sciences, environmental sciences, math and computer science and life sciences. Milken and Beacon Hill include NSF funding while OTP, Milken and CED include SBIC and SBIR awards. Beacon Hill includes NHI support.

Venture Capital
All indices include measures concerning venture capital. PPI’s measure is venture capital invested as a percentage of gross state product, as is OTP's. Beacon Hill’s measure is venture capital on a per capita basis while Milken uses a measure of the number of companies receiving VC investment and venture capital investment growth. CED looks a venture capital dollars per worker.

Connectivity
OTP, Milken, CED and Beacon Hill include a measure for the number of households with computers. PPI includes a measure for the percentage of population with Internet access while OTP, Milken and Beacon Hill look at a percent of households with Internet access. PPI also includes a measure for broadband deployment.

Quality of Life
Beacon Hill and CED include numerous quality of life factors in the measurements. Some examples are CED including poverty rates, infant mortality, teenage pregnancy, crime rates, and homeownership rates. Beacon Hill includes crime rates, travel time to work, infant mortality rate and percent of population without health insurance.

return to the top of this page

Indices & Report Cards Created by Individual States
Led by Massachusetts, which released its first innovation index in 1997, states have been creating indices in an attempt to track changes and transformations in their own economies. Many of these indices look at some of the same characteristics such as investment capital, industrial productivity, technology workforce development, business development and many others.

The following outlines some of the similarities and differences in state indices. For the purpose of this comparison, indices from (Alaska, Maine, Maryland, Massachusetts, Mississippi and Washington) are used. A matrix presenting the common indicators among the state indices is available here.

All indices factor high-tech employment into their indices in some way. Washington, Maryland, Mississippi, and Massachusetts measure company openings while company closings are included in the index by Washington. Fast growth or “gazelle” firms are included by Washington, Massachusetts, and Mississippi.

Some sort of patent activity measure is included by all of the states except Mississippi. Mississippi, however, includes an incubator measure in their index. Massachusetts and Mississippi include measures of industrial clusters. Measures of Internet connectivity are included by Maine and Alaska. All indices except Washington include measures relating to annual average pay, wages or income overall or in specific industry sectors.

Each index also looks at SBIR awards received within the state, and Maine and Maryland also include STTR awards. Washington and Maryland include ATP awards as well. Funding for R&D is included in some form in all indices whether looking at total funding for R&D or breaking it down into federal, university, or industrial R&D. The availability of venture capital or investment is also measured in one or more ways by the indices.

Some indices include quality of life measures in their indices. Washington includes such measures as traffic congestion, housing affordability and crime rates among others. All indices include measures of educational attainment as well. Some look at science and engineering graduate students, Ph.D.s in the workforce, standardized test scores, degrees specifically awarded in science and technology, and other measures of educational progression.

Some unique measures included in indices include ISO 9000 certification in Mississippi, FDA approval of medical devices in Massachusetts, tax burden in Washington, Alaska Gross State Product, and federal funding for not-for-profit labs in Maine. Many other measures are used in each indices other than the ones mentioned above. To review these and other state indices, follow the links provided below.

return to the top of this page

Local and Regional Indices Guide Policy, Too
Indices also are created at the regional and local level as well. Once again, while there is some crossover in measures, there are also significant differences that reveal some regional variation and shifts in priorities. This section highlights: regional indices for Washington D.C. (Potomac) area, Silicon Valley, Tri-Cities area in Washington State, the Roanoke region and Philadelphia; and national metro area studies compiled by the Progressive Policy Institute, the Milken Institute, the Brookings Institution and AEA. A matrix presenting the common indicators among the local indices is available here.

The Regional Indices
All regional indices include some measure of workforce make-up. All but the Potomac Index measure “gazelle” firms in some manner. Silicon Valley and Tri-Cities include measures of average income and all indices look at education of the workforce or the educational system in some manner. Venture capital or investment is included by all indices as well as patent activity. All but Philadelphia include crime indices and some measure of air and water quality. All indices include various measures in reference to transportation and transportation access.

There are many differences in local and regional indices as well. Tri-Cities includes weather and tax burden, Philadelphia includes their “creative class” ranking and broadband availability, Silicon Valley includes volunteer rates and housing near rail centers or major bus corridors, and Potomac includes a voter registration measure and intergovernmental cooperation.

Local or regional indices can be very different depending on the goal and what each region is trying to measure. To view each of these and other local indices, follow the provided links below:

Progressive Policy Institute
Offering an assessment of the progress made by the nation's 50 largest metropolitan areas toward becoming high-tech communities, as well as providing policy recommendations to help cultivate and encourage New Economy businesses, the Progressive Policy Institute (PPI) and Case Western Reserve University's Center for Regional Economic Issues prepared The Metropolitan New Economy Index: Benchmarking Economic Transformation in the Nation's Metropolitan Areas. These metro areas account for approximately 60 percent of the nation's workforce.

The index uses 16 economic indicators grouped in five subcategories — knowledge jobs, globalization, economic dynamism, transformation to a digital economy, and technological innovation capacity — to rank the metro areas on the extent to which they have made the transformation to the New Economy. Some of the indicators include number of initial public offerings (IPO's), workforce education levels, broadband telecommunication providers, dot-com domain names, patents issued, and the number of science and engineering graduates from area colleges and universities.

Along with the rankings, the report recommends a public policy framework that metro areas should follow to succeed in the New Economy. In particular, the report argues it is time to replace the Old Economy goal of simply getting bigger with the New Economy goal of becoming more prosperous. To do this, metropolitan areas need to shift focus from providing businesses tax breaks and other subsidies to investing in the skills of the workforce, a vibrant infrastructure for technological innovation, and quality of life.

The index can be found at http://www.ppionline.org

The Milken Institute
The earliest and still one of the best indices on metro economies is the America's High Tech Economy: Growth, Development, and Risks for Metropolitan Areas, prepared by the Milken Institute in 1999. Using a series of applied econometric approaches, the Milken Institute examined the role of high technology industries in explaining the relative economic growth of metropolitan areas. They found that high-tech activity (measured by movements in the relative metro real high-tech output index and the 1990 location quotient in high-technology activity) can explain 65 percent of the growth differential between metros in the 1990s.

The authors also developed a composite, standardized measure of high-tech spatial concentration combining the location quotient with the metro area's share of the national high-tech output. The result was the identification of "Tech-Poles" — metro areas that have some degree of relative technology gravitational pull or attraction.

The report is available online at: http://www.milken-inst.org. A print version is available through SSTI's bookstore at http://www.ssti.org/Publications/publications.htm

The Brookings Institution
Looking at 51 metropolitan areas, the Brookings Institution report Signs of Life: The Growth of Biotechnology Centers in the U.S identifies those areas possessing the greatest concentration of the U.S. biotechnology industry and potential for growth. Factors going into the assessment of which metro areas constitute biotechnology centers include: biomedical research infrastructure; NIH funding for medical schools; biotechnology related patents; venture capital for biopharmaceuticals; pharmaceutical/biotechnology alliances; life science and pharmaceutical research employment; number of biotech companies with more than 100 employees; number and share of biotech firms by establishment date; market capitalization of biotech companies; and, memberships in the biotechnology industry association.

Signs of Life: The Growth of Biotechnology Centers in the U.S. is available on the Brookings Institute's Center on Urban and Metropolitan Policy website: http://www.brookings.edu/urban/

AEA's Cybercities
Another large scale index is the Cybercities: A City By City Overview of the High-Technology Industry available from http://www.aeanet.org/. Like the annual Cyberstates report, Cybercities only looks at employment in the information, electronics and communication sectors.

return to the top of this page

Characteristics of Good Indices
Some characteristics that STTI has found to represent a good index would contain some, if not all, of the following:

return to the top of this page

Web Resources for Developing an Index
One of the more popular sections of the SSTI Weekly Digest — according to our annual surveys of readers — are the periodic useful stats, which point to or present comparative statistics across cities or states. Fortunately, because we try to standardize the information for more meaningful interpretation, several of these tables provide useful data for developing a local or state innovation index. The following webpage provides a list of all 56 useful stats articles and tables included in SSTI Weekly Digest since the series began in May 2000: http://www.ssti.org/Digest/Indices/usefulstats.htm

Additional links that can be helpful in compiling data for innovation indices and that are frequently used in the indices cited in this issue are:.

General Data

Education

Business Environment

R&D and research funding

Venture Capital

Connectivity

return to the top of this page

State Science & Technology Institute
5015 Pine Creek Drive
Westerville, OH  43081
Phone: (614) 901-1690
Fax: (614) 901-1696
Email: ssti@ssti.org

© 2003 State Science and Technology Institute. All rights reserved.