Okay I'll lead off by summarising some of things that have been said on Twitter, our email thread, and bobbing around in my own brain:

Get the basics right

From the conversation we've had so far, it sounds like there is consensus about not straying too far from the ggplot2 grammar of graphics interface for traditional 2D statistical plotting. As @hrbrmstr said, a lot of folks already struggle making useful visualisations for their analysis and sometimes interactivity is an unwelcome distraction. From this base we would like to build out well known interactivity mechanisms like zoom, brush, hover, and selection.

Be Reactive?

Looking at this from the development UX perspective, I am really excited by the ideas Mike Bostock put forward in his D3.express reveal, and how we might incorporate reactive bindings to enable users to more rapidly learn the grammar of graphics. This is one way we could making building the appropriate vis easier.

Be Modular, Adaptable, Extensible

It was mentioned that previous efforts have fallen down by being closely coupled to graphics back-end that was not portable. Web technologies as the solution to this have been implicit in the conversation. There are multiple viable back-ends and it will be worth investigating whether a flexible decoupled architecture can give us the best of a few worlds (first suggested by @thomasp85). WebGl has been mentioned as worthy of interest. I firmly agree with @thomasp85 in that an extension mechanism needs to be baked in from the start.

Handle Streams Natively

My personal views, that relate to the source of the Tweet-storm, are that whatever comes next needs to handle visualisations of streaming data. This is useful for raw data vis and extremely important for diagnostics and predictions of sequential models. This requirement means the rendering framework needs to be FAST (Hello WebGl) and have some inherent concept of plots that are update-able in a more fundamental way than layering.

Lead the Way to What's Next

Can we create an interface that allows a reasonably seamless transition between 2D, 3D, AR, or VR data visualisation? There's a lot to bite off here, but extensions into this space would be a huge draw card for R.