A new method that analyzes funding source acknowledgments in scientific papers ranks the National Science Foundation first among agencies worldwide in staking capital to computer and information science researchers.

Penn State University (PSU) researchers developed the method, which analyzes papers indexed on CiteSeer, the world's largest digital library of information technology research, with over 425,000 papers.

"This automated method allows us to see which agencies are funding influential research in computer and information science," said C. Lee Giles, David Reese Professor of Information Sciences and Technology at PSU and a CiteSeer developer. "We speculate that these measures could be used to evaluate the efficacy of funding agencies and programs both at the national and international level."

And I'd Like To Thank ...

At the end of most scientific papers, authors include citations and acknowledgments.

Citations acknowledge previous research that authors build upon or reference in their work -- a form of passive participation by cited sources.

Acknowledgements reflect active participation, documenting financial support and editorial, conceptual or direct scientific contributions.

"One of the measures of efficacy is impact, and a measure of impact is citations and acknowledgements," explained Giles, who created the new impact measurement tool with doctoral student Isaac Councill.

While corporations do not always request acknowledgement, funding agencies do.

Of the 15 most-cited funding agencies, NSF had the highest number of acknowledgements -- 12,287 -- followed by the Defense Advanced Research Projects Agency (DARPA) with 4,712.

"NSF funds established researchers, as well as takes a chance on new researchers," Giles said. "In the context of scientific research, NSF's total number of acknowledgements suggests it is doing extremely well."

The Seer Makes Clear

"Using machine-learning methods that identify and classify data, the researchers developed new algorithms to extract acknowledgements from 335,000 documents" from the CiteSeer library, said PSU spokesperson Margaret Hopkins.

"While we used computer and information science papers, this automated method can be applied to any area of academic documents," Giles said. "And it also allows us to add new metrics such as accounting for funding amounts and getting some idea of the impact for funds spent."

One good metric leads to another, and the PSU researchers created a second measure: a ratio of citations (C) to acknowledgements (A).

"DARPA leads in this measure with a C/A of 17.12, suggesting that DARPA funds only established researchers who have already done high impact work" that is abundantly cited, Giles said.

In the C/A metric, NSF placed third after DARPA and the Office of Naval Research.

Dollar Signs

The researchers also examined corporate funding. IBM topped the list and Siemens Corporation had the highest C/A.

But Giles and Councill dug up some bad news -- funding research does not insure survival.

Frequently acknowledged companies, such as Digital Equipment Corporation, either do not exist anymore or are greatly diminished, Giles explained.

A third metric derived from the PSU research might create "an interesting opportunity," suggested UCLA electrical engineering professor Mikhail Simkin, who with UCLA colleague Vwani Roychowdhury pioneered the new science of extracting and analyzing citation data from research papers.

"If one can learn dollar amounts of grants, the ratio of this amount to the total number of citations in papers funded by those grants would be the 'average price of a citation in dollars,'" Simkin told NewsFactor.

Researchers who study bibliographies have long recognized citations as a form of "symbolic capital," Simkin explained -- "capital because citations are socially recognized legitimization, and symbolic because they are not directly economically productive."

Using the PSU algorithm, Simkin suggests a way to put a price tag on social recognition -- at least in the sciences.

"This PSU work can help to establish a quantitative correspondence between symbolic and economic capital," he explained.