## SOCNET@LISTS.UFL.EDU

#### View:

 Message: [ First | Previous | Next | Last ] By Topic: [ First | Previous | Next | Last ] By Author: [ First | Previous | Next | Last ] Font: Proportional Font

Subject:

Re: size limits in R?

From:

"Mark S. Handcock" <[log in to unmask]>

Reply-To:

Mark S. Handcock

Date:

Tue, 8 Jan 2008 22:53:55 -0800

Content-Type:

text/plain

Parts/Attachments:

 text/plain (117 lines)
 ```***** To join INSNA, visit http://www.insna.org ***** Hi Uri, R, and the 'network' package in particular, are pretty good with memory. For example the code: library(network) a <- network.initialize(n=14000) b <- matrix(sample(1:network.size(a),size=2.1*70000,replace=T),ncol=2) index <- b[,1]+100000*b[,2] b <- b[!duplicated(index) & b[,1] != b[,2],][1:70000,] sim70k <- add.edges(a,b[,1],b[,2]) summary(sim70k) produces a network with 14k nodes and 70k edges that takes up about 36Mb. However if a full sociomatrix is naively stored it takes about 14000*13999*4 = 694 Mbytes (so 'network' uses only 5% of this storage). An edge list alone could take about 70000*2*4= 0.5Mb. In sum, 'network' does a good job storing networks, giving it stores for computation purposes. However some functions in R may be written to use the full sociomatrix form, which means it does not take to many sociomatrices to use up the RAM on a computer. I think the current plotting functions are like that as they are meant for visualization of smaller networks (you need good eyes to distinguish 70K nodes on letter paper). So R need not be bad on memory. 'network' is pretty good, and some functions have yet to take advantage of the strengths of the 'network' class. Kudos to Carter for 'network' - It is a real accomplishment. Mark ------------------------------------------------- Mark S. Handcock Professor and Chair of Statistics Department of Statistics, B-313 Padelford Hall University of Washington, Box 354322 Phone: (206) 543-6774 Seattle, WA 98195-4322. FAX: (360) 365-6324 Web: www.stat.washington.edu/~handcock internet: [log in to unmask] On 1/8/08 3:58 PM, "Robert Ackland" <[log in to unmask]> wrote: > ***** To join INSNA, visit http://www.insna.org ***** > > Hi Uri, > > I investigated R (the sna/network packages by Carter Butts, in particular) > for large graphs and ran into the problem you describe. I'm pretty sure > that Carter mentions somewhere in the documentation that there are > efficiency issues with large graphs. Perhaps there are other R packages > that handle large graphs, but for the time being I've been using the boost > graph library (http://www.boost.org/libs/graph/) which is definitely able > to handle large graphs but of course then one needs to work in > C++ and hand-code SNA routines, which is probably not what most people > want to do. > > Rob > > ------------------------------------- > Dr Robert Ackland > Fellow, Australian Demographic and Social Research Institute > College of Arts and Social Sciences > The Australian National University > > e-mail: [log in to unmask] > homepage: http://adsri.anu.edu.au/people/robert.php > project site: http://voson.anu.edu.au > ph./fax/mob.: +61 2 6125 0312/+61 2 6125 2992/+61 438 833 525 > mail: Coombs Building, 9 > Canberra, ACT 0200 > AUSTRALIA > ------------------------------------- > > On Tue, 8 Jan 2008, Uri Shwed wrote: > >> ***** To join INSNA, visit http://www.insna.org ***** >> >> Hi All, >> How does one deals with a really large network (14k nodes, 70k edges)? >> I'm trying my first steps in R - I thought it could do anything, but >> this size scares it away "Error: protect(): protection stack overflow" >> for edgelist and "Error: cannot allocate vector of size 1523603 Kb" for >> plot request. I have raised R's max memory to the point it tells me not >> to be silly...Should my relationship with R be terminated at this early >> stage? >> I'd love to compose a succinct summary of replies and repost for the >> public good. >> Thanks, >> >> -- >> Uri Shwed >> ISERP, Columbia University >> Room 805, 420 W 118th St. >> New York 10027 NY >> Tel: 212 854 9374 >> >> _____________________________________________________________________ >> SOCNET is a service of INSNA, the professional association for social >> network researchers (http://www.insna.org). To unsubscribe, send >> an email message to [log in to unmask] containing the line >> UNSUBSCRIBE SOCNET in the body of the message. >> > > _____________________________________________________________________ > SOCNET is a service of INSNA, the professional association for social > network researchers (http://www.insna.org). To unsubscribe, send > an email message to [log in to unmask] containing the line > UNSUBSCRIBE SOCNET in the body of the message. _____________________________________________________________________ SOCNET is a service of INSNA, the professional association for social network researchers (http://www.insna.org). To unsubscribe, send an email message to [log in to unmask] containing the line UNSUBSCRIBE SOCNET in the body of the message.```

Advanced Options

#### Options

 Log In Get Password Search Archives Subscribe or Unsubscribe