***** To join INSNA, visit http://www.insna.org *****
Dear socnet members,
Sorry for the delay on getting back to the list the answer for my query.
Here are the original question and the answers:
Dear socnet members,
I have been following the discussion of this discussion group for more than a year but this is my first message to the community.
I am a Brazilian economist, working in an organization called Instituto Inovação (http://www.institutoinovacao.com.br/index.php?idioma=2 ). We are working in a project with the local government to build a system aiming at fostering innovation in our state. In order to do so, a social network portal, like “My Space”, but focusing only in the innovation crowd, is being created. The project is based on the Web 2.0 concept and the objective is to integrate business, university, venture capital, governmental organizations and research institutions, providing a place where they can communicate to each other, share information and establish partnerships. The site will be at http://www.simi.org.br/ but right now there is only a Portuguese power point presentation about it on this address.
Right now, I am thinking about a methodology for evaluating and doing the impact assessment of the project. This is still in a very preliminary stage and I would like to ask community members to help me brainstorm on possible methods and recommend relevant material.
Some ideas I had so far (please, consider it as a brain storm):
1) Impact Assessment with user survey before and after participation.
We can collect some data through an online survey as soon as the user registers. Although distressing for the users and potentially discouraging, we would have user data before the program. After 6 months or 1 year, we could apply the same survey and see how the evolution was. As some users are more active than others (and we can have reports on individual usage), we have enough variation to see how an extra hour spent in the social network impacts whatever we want to measure (number of events on C&T attended, participation in tax exemption programs for innovation based enterprises, etc.). Also, users that have registered but never came back to the site can be used as a control group (with all the selection bias that comes with it).
This design has the advantage of dealing with a large number of observations and being a cheap way to obtain data. But still, I am not sure about endogenity problems within it.
2) Impact assessment with progressive knowledge testing.
One of the functions of this initiative is to have folks better informed on subjects related to innovation in the local level. Who are the key actors? Which institutions do what? What are the available government contribution and support programs? Where to go if you have an idea but no resources? Those are the kind of information that we want people to know better and we expect that the site will accomplish it.
Having this in mind, one idea is to measure the evolution of people knowledge of it by having a quiz coming up in a pop-up every now and then with random questions and some alternatives. This would enable us to see how much more people know about innovation related issues and check if there is any correlation with usage of the site and improvement at answering those tests.
Again, this is a relatively simple and cheap way of doing it.
3) Social Network Analysis
I know SNA is a powerful tool for impact assessment but I have no idea on how (and where) to use it. Insights and reference material is highly appreciated.
Apart from the evaluation and impact assessment aspect, the fact that we can have reports of the activity inside the site enables us to obtain data for other uses. More than a way of assessing impact, this could inform us on which public needs (or is interested by) what. I would also appreciate ideas on that. So far, I have thought about:
4) Impacting how each member network grows, who connects with who, if there are isolated groups, etc.
5) Measuring who downloads what from where, which profile visits more the communities from a certain subject area, etc.
I hope I was not too vague. Again, this is just a brainstorm. Any help is welcome!
Thank you for the responses to my inquiry. Here is a compilation of the feedback I received:
1) Jonathon Mote ([log in to unmask]) from the Center for Innovation, University of Maryland sent me the article “New directions in the use of network analysis in research and product development evaluation”.
Renato,I've attached a recent article on the use of SNA in R&D evaluation. Itdoesn't provide specific direction on the uses of SNA, but rather asks anumber of pointed questions about why is it being used. I hope this ishelpful. Best of luck!Jonathon MoteAssistant Research ScientistCenter for Innovationhttp://www.bsos.umd.edu/socy/centerforinnovation/University of Maryland, College Park
2) Thomas Langenberg ([log in to unmask]) pointed me up some very interesting studies.
a co-author and I have recently published a document that describes a method to study the sustainability of social ties in online communities (sustainability of online ties). You can find and download it here (http://www.ksg.harvard.edu/netgov/files/png_workingpaper_series/PNG06-002_WorkingPaper_MergelLangenberg.pdf).
In my dissertation I have been working on information sharing in social networks and how firms can benefit from it. Please find papers and further documentation here (http://drthomaslangenberg.wordpress.com/rnd/publications/).
Best Regards from Switzerland,Thomas
3) Rick Davies ([log in to unmask]) pointed me to his site. I definitely recommend the study on MSC methodology.
Hi RenatoI cant contribute right now, but am very interested in the evaluation of networking initiatives, and the use of network models of change to conceptualise causal processes in development projects. I have a website focusing on evaluation methods and one small part of this looks at the evaluation of networks. See www.mande.co.ukPlease keep me in your loop, as your ideas/plans progressregards, rick davies, Cambridge
4) Valdis Krebs ([log in to unmask]) also pointed me some information on the net.
Renato,You might want to look at our blog... we have many instances of SNA applied to improving economic and social networks -- especially around innovation.http://www.networkweaving.com/blogScroll down through the blog to see how we apply network mapping and network weaving in economic development. You may also find this white paper of interest...http://www.orgnet.com/BuildingNetworks.pdfEnjoy!
Many thanks to everyone that helped! I will keep you all informed on what happens.
Conheça o Windows Live Spaces, a rede de relacionamentos conectada ao Messenger!
SOCNET is a service of INSNA, the professional association for social
network researchers (http://www.insna.org). To unsubscribe, send
an email message to [log in to unmask] containing the line
UNSUBSCRIBE SOCNET in the body of the message.