*****  To join INSNA, visit  *****

Normalizing the measurement of citation performance: Principles for
comparing sets of documents
Authors: Loet Leydesdorff, Lutz Bornmann, Rüdiger Mutz, Tobias Opthof
(Submitted on 20 Jan 2011)

Abstract: Using citation analysis, sets of documents can be compared as
independent samples; for example, in terms of average citation counts using
potentially different reference sets. From this perspective, the size of
samples matters only for the identification of significant differences and
estimating margins of error. Using the percentile rank approach, differences
among citation distributions can be studied non-parametrically and in a
single scheme. Comparison among the sets clarifies that the different sizes
of samples affect the weighing of the probabilities and therefore the
rankings. We distinguish among (1) the normalization of papers against
external reference sets, (2) normalization in terms of frequencies relative
to the margin-totals of independent versus dependent samples, and (3) the
potentially normative definition of percentile rank classes for the
evaluation (e.g., top-1% most highly cited, median, etc.). When the sets to
be evaluated are considered as subsamples of a single sample, the consequent
citation indicator can be negatively correlated to citation indicators used

Preprint version available at  

** apologies for cross-postings

Loet Leydesdorff 
Professor, University of Amsterdam
Amsterdam School of Communications Research (ASCoR)
Kloveniersburgwal 48, 1012 CX Amsterdam.
Tel. +31-20-525 6598; fax: +31-842239111
[log in to unmask] ; 
Visiting Professor, ISTIC, Beijing; Honorary Fellow, SPRU, University of

SOCNET is a service of INSNA, the professional association for social
network researchers ( To unsubscribe, send
an email message to [log in to unmask] containing the line
UNSUBSCRIBE SOCNET in the body of the message.