***** To join INSNA, visit http://www.insna.org ***** On 25 Jan 2006, at 14:39, Andrew Cleary wrote: > It is my suggestion to be pretty skeptical of the ability of these > types of programs, not because the *will* isn't there (it is, > though I sure wish it wasn't), but because these are very difficult > technical problems to pull off. Having access to a lot of data > doesn't wash away the problems of telling the useful data from > noise. Just because someone hangs a scary government three-letter- > acronym outside their door doesn't mean they have magic bullets > that give them solutions to longstanding, very difficult problems, > and the incentive structures in place in government labs are not > very good ones for encouraging innovative solutions to problems > that the rest of the world hasn't solved. For example, they say > below that the government's systems "connect the dots", but > frankly, those "dots" are subject to the same problems of > canonicalization, the need for ontologies to apply *semantics* to > that data, data trustworthiness and cleansing, natural language > processing, inference in small and noisy datasetc, etc., etc., that > very small data mining projects have, but at a *much* larger scale, > and *all* of those issues need to be solved for the whole system to > really start making sense out of anything. > > IOW: it would be my estimate that these kinds of programs are much > more bark than bite. Andy: You are, of course, correct in stating that there are a lot of problems to be solved here. However, in my opinion, there are nevertheless reasons to be concerned about "these types of programs". (1) NSA (and other "scary government three-letter-acronym" agencies :) ) employ a lot of very smart people that have access to large amounts of resources, are aware of these problems (it's probably safe to say that they read this list, for one thing), and have been working with large data sets for some time. I don't necessarily assume that they have secret methods that blow the doors off anything known to the academic or industrial world, but I would not underestimate the amount of effort that they're willing to put into addressing these problems. Also, of course, it's worth mentioning that even if you're right about their attitude towards innovation, they have all the public research in this area to draw on. (2) That said, the potential for misuse--whether politically motivated or not--of such programs is immense. Ironically, one of the most likely scenarios, in my opinion, for misuse is one in which the models don't work very well, but are used anyway...and end up connecting all sorts of dots that should not be connected. I suspect that we've actually been seeing evidence of this already. Joshua O'Madadhain > > Cheers, > Andy, formerly of DOE research labs... > > Dr Andrew J Cleary > Director of Algorithms R&D > Visible Path Corporation > [log in to unmask] > > > -----Original Message----- > From: Social Networks Discussion Forum on behalf of Barry Wellman > Sent: Wed 1/25/2006 2:04 PM > To: [log in to unmask] > Subject: networked spyware > > ***** To join INSNA, visit http://www.insna.org ***** > > http://www.govexec.com/story_page.cfm?articleid=33212&dcn=todaysnews > > > NSA spy program hinges on state-of-the-art technology > > By Shane Harris, National Journal > The furor over the National Security Agency's domestic eavesdropping, > authorized by President Bush, has focused largely on legal > questions -- > whether the NSA has the authority to spy on Americans inside the > United > States and whether the commander-in-chief can order the agency to > do so. > > But that debate has largely smothered examination of how the nation's > largest intelligence agency is collecting -- and analyzing -- > information > intercepted from hundreds, possibly thousands, of Americans. Since the > 9/11 attacks, the NSA has abandoned the mantra that guided it in > earlier > decades -- Do not spy on Americans inside the nation's borders. Things > have changed, and the NSA may be on the cusp of employing state-of- > the-art > technologies to uncover more information about potential > terrorists, and > about Americans here at home. > > .... > > the NSA has pursued cutting-edge data-mining technologies that > don't just > find key words but also uncover hidden relationships among data > points. > These technologies can even detect how a particular analyst thinks, > identify what his or her biases are, and then suggest alternative > hypotheses. > > Data-mining systems, which the NSA has publicly pursued and spent > millions > of dollars researching, don't just "connect the dots" but also alert > analysts about which dots to connect, which to disregard, and how to > connect them in ways they may never have considered. It is unclear > which, > if any, of these data-mining tools the NSA is using to analyze the > domestic information gathered in the current eavesdropping program, > but > the tools themselves offer a telling look into the agency's > potential to > exploit what it collects, regardless of its legal basis for doing so. > > etc > > Barry > _____________________________________________________________________ email@example.com...Obscurium Per Obscurius...www.ics.uci.edu/~jmadden Joshua O'Madadhain: Information Scientist, Musician, and Philosopher- At-Tall It's that moment of dawning comprehension that I live for--Bill Watterson My opinions are too rational and insightful to be those of any organization. _____________________________________________________________________ SOCNET is a service of INSNA, the professional association for social network researchers (http://www.insna.org). To unsubscribe, send an email message to [log in to unmask] containing the line UNSUBSCRIBE SOCNET in the body of the message.