Time for Views.js?

Web applications are changing. Whereas most of the processing used to happen on the server, the current generation of browsers is capable of performing non-trivial tasks via the increasingly powerful HTML5 + JavaScript environment. Drupal, however, is still a server-heavy platform where JavaScript is used mostly as user interface candy. This needs to change if Drupal is to retain its status of an all-encompassing Web platform.

My own work with the client-side spreadsheet Sheetnode, started 3 years ago now, made me aware of JavaScript's power, and of the limitations of server-heavy applications. I also started an experiment in rich-client video consumption called Feedeo, which moves the logic of media navigation and playback to the browser.

However, I believe it is time to deepen Drupal's Lego-like approach to the client-side, by creating JavaScript components that are configurable and assemblable just like server-side components - via modules - are today. I want to present here a concrete use case and proposal for such an initiative.

Consider Views. The uber-query generator and renderer has proved to be a cornerstone of Drupal development, and yet it still lacks many features related to user interface flexibility. For example, I was trying to reuse the results of a node view to create a smaller, sidebar view of the node authors that is differently grouped and themed. In traditional Views programming, this would probably require creating a second view in a block that performs the same query as the main view, but with a different theme. On a busy site, that means doubling the amount of SQL queries right there.

My proposed solution involves a different approach to Views programming that moves to the client the theming and interaction tasks, leaving only query generation and execution on the server. Here's how this would happen in a very simplified form:

  • Query is generated and executed on the server as usual
  • Results are JSON-formatted and returned to the client
  • View styling for fields, exposed filters, etc is generated on the client
  • Filtering can happen on the client or on the server depending on whether we are drilling-down on results or changing the initial result set
  • Changing the view style happens easily on the client without re-querying the server (think of your typical file manager view of files: list, icons, details)
  • Several views can share the same result set

For this to happen, many Views components that are currently implemented as server-side code would need to be re-implemented for the browser. For example, render() methods would need to move to JavaScript. I hope that Views 3 could be used, with no or minimal patching, to provide the infrastructure upon which new JavaScript-oriented plugins and handlers would be developed. I am also thinking that using a robust JavaScript framework such as Backbone.js would simplify the client-side code and make development more fun.

I hope to free up some time to make a demo of this approach soon. In the mean time, I'd love to hear what you think!

Comments

There's a module for that

There's a module for that (and basically the whole of Drupal.js) - Backbone - digging into it as we speak.

You mentioned backbone.js.

You mentioned backbone.js. It's a great app framework for javascript. You should also look into Spine.js, its basically the same idea but it uses the power of MVC in a very ruby on rails inspired sort of way. I think that level of sophistication, intelligence, and efficiency would be a welcome addition to drupal frontend development.

Completely agree. Things are

Completely agree. Things are moving fast in the javascript world, and hopefully the WSCCI initiative will enable ideas like yours. Js frameworks like Backbone or Spine are killer tools to build such tools.

We considered using CSI

We considered using CSI (client side includes) but decided to stick with SSI (server side includes) for SEO purposes. What's the SEO outlook of your approach?

No SEO outlook whatsoever at

No SEO outlook whatsoever at the moment :-)

To expand a little bit: I am

To expand a little bit: I am sure that SEO will adapt to CSI and rich-clients, just like Google was forced to adapt by parsing Flash files.

Services:

Services: http://drupal.org/project/services .... It can pull/push JSON data from Drupal.

Can't views already do this

Can't views already do this with the datasource module? http://drupal.org/project/views_datasource I think the biggest problem is that the theming layer in Drupal doesn't support this. If you could make a JS powered theming layer, maybe with http://mustache.github.com/ then you could theoretically create client-side generated websites.

The good news is that, yes,

The good news is that, yes, many components of what I'm envisioning already exist. The JSON Views renderer is definitely one of them. What's needed is to bring all the pieces together and add the missing ones.

Supporting mustache templates

Supporting mustache templates would have another benefit: the same template can work on PHP and JS.

mustache looks cool, if only

mustache looks cool, if only for the rotated curly brace. I'm a little concerned that it is logic-less.

A related Drupal group is

A related Drupal group is node.js. They say: "This working group aims to explore how node.js can be leveraged to provide real-time services that integrate with Drupal sites."

See also:

etc...

Thanks. I am aware of efforts

Thanks. I am aware of efforts to integrate node.js with Drupal, although I'd prefer to keep just one backend technology (Drupal) and focus on richer clients.

You are right, it's a

You are right, it's a different approach. Node.js is also JavaScript but server-side. I've read your interesting client-side proposal with more time now.

Right on! I would support

Right on! I would support such projects out of my own pocket (or at least our company will). Get the initiative going, make some noise and you have the community behind you.

Drupal, however, is still a

Drupal, however, is still a server-heavy platform where JavaScript is used mostly as user interface candy. This needs to change if Drupal is to retain its status of an all-encompassing Web platform.

This overhaul is exactly what Drupal 8 aims for, thanks to WSCCI and other initiatives. In certain situations, it's no longer feasible to rely solely on Drupal's PHP-based theme layer, and the approach you're discussing is exactly what some of us have in mind for the next generation of Drupal. This is an awesome idea and I look forward to seeing it materialize!

Totally Agree! There needs

Totally Agree! There needs to be a way to extend the new AJAX API to handle things like backbone.js and turn more over to the client side and reduce the amount of views queries! We are currently running into the same issues you are while building a soft-realtime interface that heavily leverages views and AJAX CRUD.

We are developing standalone node.js projects in addition to doing Drupal development, and the sensibility behind something like backbone has become extremely apparent. Thus, your point of results being JSON formatted is super important - and I feel like there would be a smart way to translate entities and fields into a JSON schema that would be usable and recognizable by Backbone (hopefully enough to use something like JSV for validation since most of these projects won't be saving data to a mongo-based or other document-based persistence store).

I will share this with our drupal development people and see what they might suggest and to see if they would be interested in helping.

P.S. I think Tim Hemphill (one of our devs) helped you merge some SocialCalc repos for synchronous editing for a project hes working on at NASA. He actually took a different approach on the project entirely, in case you were wondering where he went!

Found this the other day:

Found this the other day: Backbone.js sandbox - http://drupal.org/sandbox/cormac/1404998

Thanks for the feedback

Thanks for the feedback Richard. I would love to work on this with a community of interested developers.

And yes, I was wondering where Tim went! I still believe that enabling synchronous editing on Sheetnode would make it a killer app :-)