SSTI Analysis: The Value of Statistics for TBED: Part Two

SSTI looks at paper on research parks

Last week's issue of the SSTI Weekly Digest included a brief review of a report which, in our opinion, included the use of statistics to potentially advance the discussion of what comprises effective tech-based economic development (TBED). Perhaps econometricians and other academic researchers may take issue with particular elements of the Milken Institute's approach, data set, equations or conclusions, but the model makes sense logically even to those of us without advanced degrees in statistics.

This week, SSTI turns its editorial attention to a recent working paper by Scott Wallsten and issued jointly by the American Enterprise Institute (AEI) and the Brookings Institute,Do Science Parks Generate Regional Economic Growth? An Empirical Analysis on their Effects on Job Growth and Venture Capital. This paper in our opinion represents how statistics and evaluation can result in faulty conclusions from a poorly structured model. The paper provides several lessons in what not to do when attempting to evaluate the effectiveness of TBED programs.

After reading the Science Parks paper in early March, we came to the conclusion it was not worth the attention of our readers. It certainly is not up to the high quality of past reports from the Brookings Institute, no matter how controversial the results or conclusions (for instance, see Signs of Life: The Growth of Biotechnology Centers in the U.S.). However, because so many in the TBED community have now seen only Wallsten's erroneous conclusions in Science Parks, printed as if truth, we feel the need to comment.

One of the major lessons learned from this paper is the effectiveness of any tech-based economic development program or policy should be based on variables and factors known to be related to the individual program or policy. Some programs will fail, some will succeed. But you would never be able to determine which is true in your particular situation if you applied the model used in the AEI-Brookings paper.

The Milken Index asked which generic measures could be used effectively to predict state-level growth of a knowledge-economy. In contrast, Wallsten purports to evaluate the effectiveness of a specific local TBED strategy, creating a local science or research park, using only three generic county-level measures: high-tech employment, number of high-tech firms, and venture capital investments. The difference is significant.

Wallsten contends that since venture capital is associated with strong regional technology clusters and communities establish research parks to create technology clusters, then the creation of a park will result in more venture capital. However, that logic escapes us. One would need to determine first if venture capital investments and research parks are associated strongly with each other, and there's no reason to think that they would be.

How many research, technology or science parks have as their primary goal to increase the amount of venture capital? None of which we are aware. Conversely, how many seed and venture capital groups would say the best way to increase the availability of capital in a community is to establish a science park? Lesson: if you want to determine a TBED program's effectiveness, use measures that are consistent with and related to the purpose of the program or policy.

Another concern is the definition and use of the variable for presence of a science park. The author wants to look at employment five years before and five years after establishment of a park. But what defines the establishment of the park? The date the first shovel is turned for infrastructure improvements? The first tenant that occupies space? The community or university commits or receives funds to establish the park?

The establishment or creation date for a research park can vary significantly in meaning, and which date is picked is one of the variables as to when the clock starts ticking for job creation. The author seems to think the success of any science park can be determined within five years of its establishment. Yet, technology parks generally do not yield immediate results. The second lesson, then, is to be familiar with the time requirements of the particular program or policy.

There exist other significant differences among science parks that would affect their ability to contribute significantly to the county-level data the author uses for this paper, with the park's size being perhaps the easiest to consider here. Research parks can be as small as a few buildings on a city block to more than a thousand acres. A park that can only accommodate a few companies with even several hundred jobs in total is not likely to create much of a blip on the county level data if the county itself is quite large. Proximity of the park to one or more research universities, federal lab or large R&D facility may also be an important variable for determining the success of a park. The amount of money necessary to develop the park and the services, if any, provided in conjunction with the real estate also may play roles in the park's success. A third lesson for those attempting to evaluate a TBED program might be to understand the characteristics of the program, the dif ferent approaches that could be encountered, and the other variables affecting program design and success.

The author concludes that research parks are not likely to help generate technology-based economic growth and that subsidies spent on them are likely to be ineffective. The author's model, however, is so fundamentally flawed that it undermines any potential conclusion -- positive or negative -- at which he would arrive.

Since its inception, SSTI has stressed the importance of evaluation and encouraged state, federal and foundation funders to help advance the field of evaluating the effectiveness and impact of TBED programs. The AEI-Brookings paper does more to set evaluation of TBED back than it does to move it forward.  We eagerly await the results of any credible assessment of the overall effectiveness of research parks; this paper is not it.