Print

Print


*****  To join INSNA, visit http://www.insna.org  *****

Just to add some brief comments to this thread -

I would certainly agree that the existing evidence points to the fact 
that both self and third-party reports of social interaction (broadly 
defined) are error-prone.  However, I am not aware of any study whose 
data actually supports the extreme cognitivist view that such reports 
are effectively unrelated to behavior.  (There are certainly those who 
have claimed this in print, but reanalysis of the data in question does 
not support this assertion.  This includes the BKS data, although I 
don't know that B, K, or S would currently endorse that position in any 
event!)  On the contrary, there is both direct and indirect evidence 
that informants' accounts do reflect observable social reality, albeit 
with a fair amount of noise.  (Some of this is literature is reviewed in 
the 2003 _Social Networks_ piece cited below.  See Freeman et al. (1987) 
for a nice example of a case with a well-observed criterion.)

As my colleagues Kim Romney and Bill Batchelder demonstrated some years 
ago, it is often possible to pull external reality out of such noisy 
reports using appropriate inferential techniques.  (Gest's comments seem 
to be in this vein as well, although I am not familiar with their 
particular approach.)  There are obviously limits on the efficacy of 
such methods, and they are not "magic bullets" which can solve all data 
problems; as folks like Devon Brewer have emphasized over the years, 
there is no substitute for good data.  However, they have been shown to 
be effective in a range of settings, and Steve Borgatti tells me that he 
can even use the approach to accurately grade his students' exams (not 
that he does, mind you!).  In the network case, simulation studies I 
have performed vis a vis a Bayesian version of this approach (some of 
which are very briefly described in section 3 of the 2003 paper) 
indicate that the models are fairly robust to structurally correlated 
errors, high variation in error rates, etc.  This does _not_ mean that 
they are always right, or that they cannot be broken...however, there is 
some reason to think that they can ferret out a pretty good 
approximation of the truth under the kinds of error rates which have 
been observed in the prior literature.

I think that the debate over informant accuracy has been tremendously 
important to the field.  Unfortunately, I suspect that many researchers 
are still unaware of the nature of the problem, and of the kinds of 
techniques which now exist for minimizing it (or at least estimating its 
consequences).  Obtaining multiple reports on each edge, for instance, 
can greatly improve the accuracy of subsequent analyses, yet few 
researchers include such measures when designing their studies. 
Hopefully, this situation will improve as these newer tools and 
techniques diffuse into the network community.

-Carter


References:

Batchelder, W.H and Romney, A.K.  1988.  Test Theory Without an Answer 
Key.  Psychometrika, 53(1): 71-92.

Butts, C.T.  2003.  Network Inference, Error, and Informant 
(In)accuracy: a Bayesian Approach.  _Social Networks_, 25:103-140.

Freeman, L.C.; Romney, A.K.; and Freeman, S.C.  1987.  Cognitive 
Structure and Informant Accuracy.  _American Anthropologist_, 89(2):310-325.

See also the "bbnam" and "consensus" functions in the sna library for R, 
or Borgatti's ANTHROPAC, for software implementations of these models.

_____________________________________________________________________
SOCNET is a service of INSNA, the professional association for social
network researchers (http://www.insna.org). To unsubscribe, send
an email message to [log in to unmask] containing the line
UNSUBSCRIBE SOCNET in the body of the message.