*****  To join INSNA, visit http://www.insna.org  *****

Hi Carl,

if you'd like to crawl e.g. google blogs, you could use the software  
Condor:
http://www.galaxyadvisors.com/documents/condor.pdf

Regards,
Lukas

---
Lukas Zenk, PhD.cand.
Member of the scientific staff
Department of Knowledge and Communication Management
Danube University Krems - Austria / Europe
www.donau-uni.ac.at

On Jul 10, 2009, at 2:56 AM, Carl Nordlund wrote:

> *****  To join INSNA, visit http://www.insna.org  *****
>
> Hi!
> Inspired by the amazing work done by the ppl at Berkman center at  
> Harvard (http://cyber.law.harvard.edu/publications/2008/Mapping_Irans_Online_Public/interactive_blogosphere_map 
> ), I've been thinking about how to gather bloggosphere data, i.e.  
> the creation of a (national) network dataset in which each node is a  
> blog and where the edges/links are the number of directional links  
> (external) from-to each pair of blogs. I have started working on a  
> php script that recursively crawls a website, check for external  
> links, and builds a dataset - this of course has to be combined with  
> a check on the nationality of the blog (comparing with national IP  
> ranges and/or language analysis of a sample text).
>
> But perhaps I'm trying to invent the wheel again. Are there any  
> suitable web crawling software that can do the trick? As I have  
> understood it, the consulting firm Morningside Analytics helped the  
> Berkman group in their mapping - judging by the rather large  
> dataset, I assume that they used some sort of web crawler. Anyone  
> knows anything more about this?
>
> Yours,
> Carl Nordlund
> ---
> Carl Nordlund, BA, PhD student
> carl.nordlund(at)hek.lu.se
> Human Ecology Division, Lund university
> www.hek.lu.se
>
> _____________________________________________________________________
> SOCNET is a service of INSNA, the professional association for social
> network researchers (http://www.insna.org). To unsubscribe, send
> an email message to [log in to unmask] containing the line
> UNSUBSCRIBE SOCNET in the body of the message.

_____________________________________________________________________
SOCNET is a service of INSNA, the professional association for social
network researchers (http://www.insna.org). To unsubscribe, send
an email message to [log in to unmask] containing the line
UNSUBSCRIBE SOCNET in the body of the message.