Motivation

In the past few months I have been using DropBox for syncing my work files between my home and work computer. It has saved me from numerous mistakes and from sending the files to myself via e-mail.

Recently I found this service highly useful for sharing files with 4 other people with whom I am working on a data analysis project. Being so happy with it (and also by gaining more storage space by inviting friends to use it), I thought of sharing my experience here with other R users that might benefit from this cool (free) service.

What is Dropbox?

Dropbox is a Software/Web2.0 file hosting service which enable users to synchronize files and folders between computers across the internet.
This is done by installing a software and then picking a “shared folder” on your computer. From that moment on, that folder will be synced with any computer you choose to install the software on (for example, your home/work computer, your laptop – and so on)

DropBox also enables users to share some of their folders with other DropBox users. This seamless integration of the service with your OS file system (Windows, Mac or Linux) is what’s making this service so comfortable, by allowing me to work with co-workers and have the same “project tree” of folders, all of which are always synced.

You could also share a file “online”, by getting a link to it which you could share with others. So for example, you could write an R code, share it online, and call to it later with source(). This is the easiest way I know of how to do this.

Dropbox is a “cloud computing” Web2.0 file hosting service offering both free and paid services. The free version (which I use) offers 2GB of “shared storage” (unless you invite other users, in which case you get some extended storage space. Which is one of my motivations in writing this post).

Dropbox has other non-trivial uses allowing one to:

• Monitor your Computer Remotely,
• Selective Folder Syncing (BETA),
• Remote Launching of Applications with Dropbox and AutoHotkey,
• And some other uses.

The service’s major competitors are Box.net, Sugarsync and Mozy, non of which I have had the chance of trying.

How to start?

Simply go to: DropBox.com
Sign up, install the software, use the new shared folder, and let me know if it helped you

In it Dr. De Mars wrote (I allowed myself to emphasize some parts of the text):

Contrary to what some people seem to think, R is definitely not the next big thing, either. I am always surprised when people ask me why I think that, because to my mind it is obvious. [...]
for me personally and for most users, both individual and organizational, the much greater cost of software is the time it takes to install it, maintain it, learn it and document it. On that, R is an epic fail. It does NOT fit with the way the vast majority of people in the world use computers. The vast majority of people are NOT programmers. They are used to looking at things and clicking on things.

Here are my two cents on the subject:

First, I agree with Dr. De Mars that R (out of the box) is not very (non programmer) user friendly – there is (almost) no point and click capabilities.  And while there are several projects offering a GUI layer interface to R (a good list of them can be found here), still non of them is in the level of refinement of what softwares like SPSS, JMP or SAS offers to users today.

But is traditional “point and click” the next “big thing”?  My suspicion is that the answer is – no.
Neither does Dr. De Mars thinks so, since her predictions for the next big thing are “Data visualization” and “Analyzing enormous quantities of unstructured data”. Both of which R is offering quite powerful solutions to (assuming that you will go through the learning curve).

Dr. De Mars question is a fascinating one – what IS going to be the next big thing?

I think that the next BIG thing is (becoming to be) “Statistics in the Cloud“.  This intuition came from (among other things) my review of the “Future of Open Source” Survey (see “conclusion 3″).

In the near future, I believe, we will see more statisticians and data analysts tapping into the opportunities that cloud computing offers them.  Here are some examples of what I came a cross (or covered) lately in the topic of cloud computing and R:

1. Easy online collecting of data (via google forms)
2. High-performance computing - Running a statistical package software on the cloud for accessing a powerful computer or running stuff in parallel.  The former can be done through services like Amazon cloud, Elastic R, and lately R-Node combines the running of R and Protovis on a server.  The later I don’t have experience in, but understand there are various solutions in R (a known company in the field is, of course, REvolution computing)
3. Online statistical analysis/visualization of data - Having a web interface to a statistical analysis.  One wonderful example of that is Jeroen Oom’s (beautiful) web interface to ggplot2.  Such projects offer “point and click” capabilities through the internet (/cloud)
4. Online interactive visualization of data.  I came a cross three people offering to develop solutions for doing this with R in this year’s Google summer of code, I hope something will come out of it

All of these are well connected to the emerging trend of “web of data”/“linked data web” that some are talking about. For example, here is a good Ted talk by Tim Berners-Lee (the inventor of the World Wide Web). Talking about building a web for open, linked data that could do for numbers what the Web did for words, pictures, video: unlock our data and reframe the way we use it together.

The same plea is given by Hans rosling in his famous Ted talk showing GapMinder.
Although at he same time, some R users are saying – “You don’t have to bother linking the data. I’ll do with just the data, really, just release it…”

In conclusion, I don’t know what capabilities other projects/products offer for doing statistics in the cloud.  But it is clear to me that the R community is (not surprisingly) bringing very diverse and innovative solutions to the world.

Is R the next big thing?  I don’t think so.  But I do think that some of the next big things will be built with R.
* * *
I would love to know your thought about Dr. De Mars post, and also about what the “next big thing” is going to be (and what role will R have in it).

Jeroen Ooms released a new version of his (amazing) online ggplot2 web interface:

yeroon.net/ggplot2 is a web interface for Hadley Wickham’s R package ggplot2. It is used as a tool for rapid prototyping, exploratory graphical analysis and education of statistics and R. The interface is written completely in javascript, therefore there is no need to install anything on the client side: a standard browser will do.

The new version has a lot of cool new features, like advanced data import, integration with Google docs, converting variables from numeric to factor to dates and vice versa, and a lot of new geom’s. Some of which you can watch in his new video demo of the application:

The application is on:
http://www.yeroon.net/ggplot2/

p.s: other posts about this (including videos explaining how some of this was done) can be views on the category page: R and the web

* * * *

The integration of R into online web services is (for me) one of the more exciting prospects in R’s future. That is way I was very excited coming across Jamie Love’s recent creation: R-Node.

What is R-Node

R-Node is a (open source) web front-end to R (the statistical analysis package).

Using this front-end, you can from any web browser connect to an R instance running on a remote (or local) server, and interact with it, sending commands and receiving the responses. In particular, graphing commands such as plot() and hist() will execute in the browser, drawing the graph as an SVG image.

You can see a live demonstration of this interface by visiting:
http://69.164.204.238:2904/
And using the following user/password login info:
User: pvdemouser
Password: svL35NmPwMnt
(This link was originally posted here)

Here are some screenshots:

In the second screenshot you see the results of the R command ‘plot(x, y)’ (with the reimplementation of plot doing the actual plotting), and in the fourth screenshot you see a similar plot command along with a subsequent best fit line (data points calculated with ‘lowess()’) drawn in.

Once in, you can try out R by typing something like:

x <- rnorm(100) 
plot(x, main="Random numbers") 
l <- lowess(x) 
lines (l$y)

The plot and lines commands will bring up a graph – you can escape out of it, download the graph as a SVG file, and change the graph type (e.g. do: plot (x, type=”o”) ).
Many R commands will work, though only the hist(), plot() and lines() work for graphing.
Please don’t type the R command q() – it will quit the server, stopping it working for everyone! Also, as everyone shares the same session for now, using more unique variable name than ‘x’ and ‘l’ will help you.

Currently there is only limited error checking but the code continues to be improved and developed. You can download it from:
http://gitorious.org/r-node

How do you may imagine yourself using something like this? Feel invited to share with me and everyone else in the comments.

Here are some of the more technical details of R-Node:

How does R-Node works

(Credit: The following text is based on this forum thread)

R-node, uses protovis for drawing graphs. Protovis is a visualization toolkit written in JavaScript using the canvas element. Using simple graphical marks, like boxes and dots, one can construct custom views to present or explore data.

Besides Protovis, R-node also uses jquery and ExtJS core on the front-end.

Most R commands are passed back to the server and their results returned to the client. Some, such as the graph commands, are parsed and the arguments used in javascript re-implementations of the R commands (e.g. the R command ‘plot’ has a protovis equivalent).

The server side is R+Rserve, and to connect the browser client to the R server Jamie used a nodejs based application server.

Projects utilised in this include:

• Protovis – http://vis.stanford.edu/protovis/
• Nodejs – http://nodejs.org/
• R – http://www.r-project.org/
• Rserve – http://www.rforge.net/Rserve/doc.html
• Shjs – http://shjs.sourceforge.net/

I would love to read your thoughts about this in the comments.

The answer people gave to it where on various ways of using excel.  But excel files (at least for now),  are not “on the cloud”.  A better answer might be to create a google form that will update a google spreadsheet that will then be read by R.

If my last sentence wasn’t clear to you, then this post is for you.

First view the following video (2:50 min) which explains what google spreadsheets are

Then view the next video (2 min), which explains what using google forms with google spreadsheet is all about

The next step for you is to access the spreadsheet without having to make it public is to go thorough this tutorial on how to use the R package RGoogleDocs.

If you feel comfortable to publish the google spreadsheet, there is a simpler tutorial for you here.  And you could see examples of using this method here and here.

Update. Farrel Buchinsky wrote (in the mailing list) the following:

There is also a package called RGoogleData that can import data from Google
Spreadsheets (even when they are not public). It has one benefit over
RGoogleDocs; RGoogleData can download a spreadsheet as a csv file to your
harddrive. You can then read the csv file into R – read.csv(). Those steps can
easily be written into a script so that you do not have to manually futz with
them each time. The benefit of the csv download is that it is very fast. Both
the RGoogleDocs and the RGoogleData are slow in reading the spreadsheet into R.
I understand, but may be wrong, that Google Stores all the data as HTML and its
API spits it out as xml which these r packages then need to parse out line by
line. The csv treatment by comparison is instantaneous.

Only one big problem. As of a few months ago, RGoogleData has stopped working
for me. I think it is still working for its writer, Adrian Dragulescu. Adrian
has been very generous with his time in trying to help me. He predicts that he
will go back to exploring his package once Google updates their API.

Try it out. I would love to hear what happens.

http://r-forge.r-project.org/projects/rgoogledata/

Do you have any good example for using R with google spreadsheets? please share with us in the comments…

In this ~7 min video tutorial, Jeroen shares with us:

1. How to write ”Hello World” in a website using RApache.
2. How to extract arguments from a form submited by the website visitor (and then inserting it into an “rnorm” function so to control the output). And finally,
3. How to catch an error in case of an invalid argument on an R Web Application.

Thank you Jeroen for a very simple, step by step, tutorial:

p.s: For more videos by Jeroen, have a look at

You can also view the slides for his talk and view (great) examples for: stockplot, lme4, and gpplot2.

Thanks again to Jeroen for sharing his knowledge and experience!

R-Bloggers.com is a central hub of content collected from bloggers who write about R (in English) and if you are an R blogger you can join it by filling in this form.

I built the site with the aspiration to  help R bloggers and users to connect and follow the “R blogosphere”. When I am writing these words, R-bloggers already has 17 blogs in it, and I hope for many (many) more.

How does R-Bloggers operate? This site aggregates feeds (only with permission!) from participating R blogs. The beginnings of each participating blog’s posts will automatically be displayed on the main page with links to the original posts; inside every post there is a link to the original blog and links to other related articles. While all participating blogs have links in the “Contributors” section of our sidebar

What does R-Bloggers offer it’s visitors?

• Discover (for all): Find new R blogs you didn’t know about. And Search in them for content you want.
• Follow (for people who don’t use RSS): Enter your e-mail and subscribe to receive a daily digest with teasers of new posts from participating blogs. You will more easily get a sense of hot topics in the R blogosphere.
• Connect (for facebook users): Click on “Fan this site” to become a “fan” of R Bloggers. You can then “friend” other people and share thoughts on our wall. Or just by leaving comments on the blog.
• Participate (for bloggers): Add your R blog to get increased visibility (for readers and search engines) with permanent links on our Contributors sidebar. Your blog will also gain visibility via our e-mail digest and through your presence on the main page with posts.

Who started R-Bloggers (and way)? R Bloggers was started by Tal Galili (well, me).  After searching for numerous R blogs I decided that there must be more R blogs our there then he knows about, and maybe the best way for finding them is to make them find him.

After writing about it in the R mailing list, I got some good feedbacks but also questions about why use only R blogs and not all the R feeds that exist. Who is the website actually for (when there are services like Google reader for us to read our feeds with), and what am I hoping it will do. So here is what I answered:

For me there are two audiences:
One is that of the web 2.0 power users. That is, people who know what RSS is and use it, maybe evern write their own blogs. These people have only one problem (as I see it) that R-bloggers tries to solve, and that is to know who else lives in their ecosystem. Who else they should follow.
For that, google reader recommendation system is great, but not enough. A much better system is if there was a one place where all R bloggers would go, write down their website, and all of us would know they exist. That is what R-bloggers offers for the power users. I think this is also why over 20 of them subscribed to the site RSS feed.
BTW, The origin of this idea came to me when I was trying to find all the dance bloggers for my wife (who is a dance researcher and blogger herself). After a while we started http://www.dancebloggers.com/ while knowing of only 10 bloggers. They list now has over 80 bloggers, most of which we would have not known about without this hub.
The same thing I am trying to do for the R community, that is way I hope more R bloggers would write about the service – so their network of readers which includes other R bloggers would add themselves and we will all know about them.
If that was my only purpose, a simple directory would have been enough. But I also have a second one and that is to help the second audience.

The second audience I am thinking of are people of our community who are not so much early adopters (and actually quite late adapters) of the new facilities that the new web (a.k.a: web 2.0) provides.
To them the all RSS thing is too much to look at, and they are used to e-mails. And because of that they are (until now) disconected from many of the R bloggers out there, simply because it is in-efficient for them to go through all these blogs each day (or even week). So for them, to see all the content in one place (and even get an e-mail about it) would be (I hope) a service. I believe that’s why 5 of them (so far) has subscribed via e-mail.
I also hope teachers will direct their students to this as a resource for getting a sense of what people who are using R are doing.
Another thing that hints me about the R community is seeing how the “facebook fan box” is still empty. Which tells me that (sadly) very few R users are actively using facebook as a means for connecting with the outer networks of people out there.

All I wrote also explains why R-bloggers will only take feeds of bloggers and only (as much as can be said) their posts that are centered around R (hence the website name ).
It both follows what Gabor talked about – having a site who’s content is only about R. But also what I wish, which is to have “content” in the sense of articles to read (mostly). And not so much things like news feeds of wikipedia or new packages published.

I hope this post will both notify people about this new resource, encourage more R bloggers to join, and will help for future people to better understand what this R-Bloggers thing is all about

Today Jeroen announced a new version (0.11) of his web interface to ggplot2. See it here:
http://www.yeroon.net/ggplot2/

As Jeroen wrote:

New features include 1D geom’s (histogram, density, freqpoly), syntax mode (by clicking the tiny arrow at the bottom), and some additional facet options. And some minor improvements and fixes, most notably for Internet Explorer.
The data upload has not been improved yet, I am working on that. For now, it supports .csv, .sav (spss), and tab delimited data. Please make sure your filename has the appropriate extension and every column has a header in your data. If you export a dataframe from R, use:
write.csv(mydf, ”mydf.csv” , row.names=F). If you upload an spss
datafile, none of this should be a concern.
Supported browsers are IE6-8, FF, Safari, and Chrome, but a recent browser is highly recommended. As always, feedback is more than welcome.

Here is a little demo video that shows how to use the new features:

The datafile from the demo is available at http://www.yeroon.net/ggplot2/myMovies.csv.

I wish the best to Jeroen, and hope to see many more such uses in the future.