Profs. Wm. Kerr and Wm. Lincoln of the Harvard Business School just released a study in which they claim that Chinese and Indian H-1Bs, especially the Chinese, are highly innovative, and that "shocking" (their term) our economy with a large inflow of H-1B would be greatly beneficial. The paper is already making the rounds—Kerr will speak at the Center for Immigration Studies on Feb. 13, and Vivek Wadhwa lauds it in his BusinessWeek column—and is sure to be highly cited by the industry lobbyists.

The authors' finding that the Chinese are the most innovative might strike some readers as odd? Aren't East Asians known for lack of innovation? Actually, the governments of China, Japan, South Korea and Taiwan themselves believe this, and have publicly wondered how to reverse it. Nobel physicist C.N. Yang has said, "...[those] trained in the Orient tend to be too [intellectually] timid...This attitude prevents them from jumping over hurdles to make important contributions...This too timid attitude is a handicap later in life when they want to be more creative or more imaginative." I must assure you that there are indeed many East Asians that are fantastic innovators (Yang himself of course being a prime example)—but at the same time, I share the views of the abovementioned governments and Dr. Yang.

As I will explain, the study is indeed fundamentally flawed. It is impressive looking, 50 pages in length, extensive data analysis, lots of references, and math that goes beyond the capabilities of people on the Hill (i.e. logarithms). But, sadly, it makes a number of major errors.

Prof. Kerr kindly e-mailed me a draft of the study, asking for comments, and I responded with a detailed list of suggestions. Some of them involved important aspects of H-1B that he seemed to be unaware of, while others were methodological. (I am a former statistics professor, and continue to do research and consulting in the field.) But as far as I can tell, he did not heed any of my comments in the final version of his paper, though of course he does cite my work, primarily "On the Need for Reform of the H-1B Nonimmigrant Work Visa in Computer-Related Occupations," N.S. Matloff, University of Michigan Journal of Law Reform, Fall 2003, Vol. 36, Issue 4, pages 815-914.[PDF]

Where, then, did the authors of this study go wrong? The most glaring error is their failure to properly account for the relation of H-1B hiring to the boom/bust nature of the tech field. They do note that boom/bust nature, but don't understand the implications.

Clearly, there is more patenting during boom times and less during busts. During booms, there are more jobs, in particular more R&D jobs, since R&D is a luxury for almost all firms, and thus more patents. Moreover, there is much more venture capital available during booms, and since startup firms tend to be more innovative, this makes the boom/bust variable even more important.

This ties directly to the number of H-1Bs. Congress has been willing to expand the H-1B program during times of boom in the tech sector, and has declined to expand it during times of bust. Thus the authors' analysis boils down to showing that more patents are produced during booms, rather than showing a direct H-1B effect on patenting.

This is of course already a central problem in the study's analysis. But in addition, I pointed out to Prof. Kerr that this in turn exacerbates a methodological problem called multicollinearity, in which the variables are so intercorrelated that it can change regression coefficients from positive to negative and vice versa. No need to go into the details here (the Wikipedia entry is actually a pretty good exposition of the issue if you want a basic summary), but the point is that since the author's entire analysis rests on the signs of these coefficients—positive meaning that H-1Bs have a positive effect on U.S. innovation—the authors have real statistical problems in addition to the "common related variable" flaw due to the boom/bust pattern discussed above.

One aspect the authors were especially interested in was whether Americans were "crowded out" of the field—i.e. displaced—or "crowded in," meaning that the presence of the H-1Bs actually enhanced the innovation efforts of the Americans. They find that there appears to be a crowding-in effect, though they mention that it is not statistically significant, i.e. their sample size is not large enough to be very sure. What they don't mention, though, is that the confidence intervals they have for those regression coefficients suggest almost as strongly that the coefficients are negative.

Also on the crowding-in/out issue, the authors cite the work of fellow Harvard professor George Borjas that "natives are crowded-out from graduate school enrollments by foreign students, especially in the most elite institutions, and suffer lower wages after graduation due to the increased labor supply." The authors say that Borjas' findings don't jibe with other work on this topic, but in fact the papers they cite don't address the Borjas issue at all, a serious, disturbing logical error.

Moreover, the National Science Foundation (one of the funders of the Kerr/Lincoln study) itself stated at the time Congress was considering instituting the H-1B program that the program would indeed crowd OUT the Americans. In the paper I've quoted often here, the NSF said that "A growing influx of foreign PhDs into U.S. labor markets will hold down the level of PhD salaries...[If] doctoral studies are failing to appeal to a large (or growing) percentage of the best citizen baccalaureates, then a key issue is pay...A number of [the Americans] will select alternative career paths." That of course is exactly what happened in the subsequent years; enrollment by Americans in tech grad programs has gone way down, and tech PhD salaries have not kept pace with those of comparable professions, exactly as Borjas found.

So the crowding-out is quite clear and, as mentioned, actually forecast by the NSF. That invalidates Kerr's and Lincoln's regression analyses right at the outset, because they are based on data that doesn't account for the crowding-out at the graduate school level, which the authors agree is the source of most of the later patent activity.

Another key issue is that the authors do not address the counterfactual. What if natives were to hold the positions taken by H-1Bs? Would the number of patent applications filed be the same as, greater than or less than the number we now see? Again, failure to address those questions renders the authors' findings invalid.

Aside from all these problems with the study, it showed an insensitivity that I tried (unsuccessfully) to warn Prof. Kerr about. It counted everyone with a Chinese or Indian surname as foreign-born, and by implication, a current or former H-1B. This ignores the huge numbers of Chinese- and Indo-Americans who were born here or immigrated as minors with their families. On the other hand, anyone with a British surname is counted as a U.S. native, and the authors repeatedly describe U.S. natives in this manner, using terms such as "English inventors," "English patenting," "English ethnicity," and so on. While I can understand why the authors might find it convenient to use such proxies, I pointed out that it might be offensive to some (especially given the "English" surnames of the two authors). But the authors made no change.

You can download the paper here.

Norm