229: BigPipe in Drupal 8: Better Perceived Performance for Free

Wim Leers, Senior Software Engineer in the Acquia Office of the CTO (aka “OCTO”), has been busy in the last few years making Drupal 8 amazing! His contributions include working with Fabian Franz on aspects of Drupal’s new caching and rendering systems to make Drupal 8 performant. Today’s podcast is a conversation he and I had about who he is and what he’s been up to following our own collaboration preparing my own post on BigPipe.

Below is a transcript of parts of the conversation you can hear in full in the audio and video versions of this podcast. In the audio and video versions, we also touch on:

  • aspects of contribution and the professionalization of contribution in open source, especially in the light of Wim being paid by Acquia to be a full-time contributor to Drupal.
  • how even small contributions, like a well-written bug report, add up to making a big difference ... and my daughter’s commit credit in Drupal 8 :-)
  • Hierarchical Select module
  • Many hands making light work in open source
  • Plus everything below about caching, BigPipe, performance, and more in the transcript!

Learn more about BigPipe in Drupal 8

Interview video - 41 minutes

BigPipe in a nutshell: “What matters in the end is not the number of requests, but how fast it actually feels for the end-user because that's what you care about and that's where BigPipe makes a huge difference." - Wim Leers

Guest dossier

  • Name: Wim Leers
  • Work affiliation: Senior Software Engineer, Acquia Office of the CTO
  • Drupal.org: wim-leers
  • Twitter: @wimleers
  • LinkedIn: Wim Leers
  • GitHub: wimleers
  • Blog/Website: http://wimleers.com/ - "Hello! My name is Wim and I’m interested in WPO, Drupal and data mining. I’ve worked on Facebook’s Site Speed team. And I love llamas."
  • Drupal/FOSS role: Drupal core contributor
  • 1st version of Drupal: Drupal 5 beta

BigPipe loading principle demo gif

Partial Transcript

How did you discover Drupal?

Wim: I was going to build this website – or I needed to build a website but I was looking for a way that will allow me to set up a website that was maintainable, that didn’t require too much digging around in code, and that looks like it would be a good choice for the long run. I looked at WordPress, at Joomla, at Drupal, and I think a few others maybe, but Drupal stood above the rest like it was the obvious better choice back then, I believe. It was the time of Drupal 5.0 being in active beta. 4.7 was I think the active version. I never used that. I jumped straight to the beta because it looked much better.

jam: I had the joy of installing 4.6 and 4.7. The good old days. Wow. Drupal 5.0 was such a massive leap at that time.

Why did you stick with it now for nine years?

Wim: Yes. I got kind of rolled deeper into the community as I think is the story for many of us. That was 2006 - the end of 2006. It was the Christmas break at my first year of University. I was trying to actually do less work on this Open-source project that I was working on before by building a website so that others could maintain it. So it’s kind of funny that I used Drupal 4 and other Open-source projects. In doing so, I needed a few things to be built myself in order for this website to really function well. So I started working on that and then I noticed - back then it was very easy to get an account that allows you to create a project, a module on Drupal.org. Right now, we have to go through a quite tough review process or back then at least, there was not review process. It was just if they saw you on IRC quite a few times, “Yes, sure, you can get an account.”

jam: The pendulum swings back and forth on that one.

Wim: I know.

jam: We’ve been in a fairly Draconian period recently.

The path to becoming a performance expert

... from doing something Open-sourcey to choosing Drupal because it seemed the best, to then getting annoyed by sites being slow and then looking at how Drupal could be faster for a Bachelor thesis, to them better understanding it through my Master thesis and at Facebook and then eventually, I ended up at Acquia. It’s a path that has definitely been big, mutually influenced by Drupal.

Wim: Yes. I don't know the details. At any case, back then it was very easy. That’s all that matters. That’s why I managed to publish a module very early on and that started to get quite a few users and get more users and I found it interesting that my module that was growing in feature sets and getting more and more users on the hundreds of websites using it. That was so fascinating to me that I kept working on it and improving it more. Then, I got freelance work doing that in the summer so instead of having a crappy student job for the summer, I managed to do freelance work while further developing this module. So that led to more freelance development and that led to my bachelor thesis being about Drupal and CDNs so performance in general, but then a few years later led to my master thesis being about Drupal, not very strictly Drupal but again performance. Performance plus data mining to better understand why a site is slow in certain scenarios. That basically led to Drupal being a significant part or significant presence during my entire period of studying computer science.

jam: What was that first module?

Wim: Hierarchical select.

jam: Picking up at your Master's thesis, talk about your work in performance and where that’s led you.

Wim: Yes. It’s quite interesting to see how Drupal allowed me to do a bachelor’s thesis on Drupal plus CDN to make Drupal faster. Then I wanted to better understand in which scenarios a site could still be slow so for example when accessed from a certain region even while using a CDM or when using a particular browser or maybe a particular piece of JavaScript was slow in a particular browser. Those kind of things, figuring that that that is quite difficult. People complain it’s slow, but they don’t really explain why, or it’s normal, that’s regular, and users if you will – non-technical people will just complain and say it’s broken or slow but will not be able to pinpoint that’s the exact reason.

There can be so many reasons and it can be very difficult to simulate that, to actually see it happen in front of you as a developer. So for my Master’s thesis, I worked on data mining and collecting performance information, performance data. So applying data mining on the performance data allowed me to automatically figure out which situations, which combinations were slow. So then it will allow you to see which exact scenarios are also the things that are most commonly slow. Therefore, our most worth attention from a person fixing them to look into those problems. From that point of view ... And I published that Master thesis and a series of blog posts about that ... Somehow, a person at Facebook discovered that or stumbled upon that and he reached out to me. He was from the – what was called back then the Site Speed Team. I first, I literally didn't believe it that there was a person from Facebook contacting me. I was looking at the email headers to figure out if it was spoofed or something.

jam: Somebody’s pranking you.

Wim: Yes, yes. Exactly. That's what I thought. So it looked like it checked out. I was saying, “Okay. Let’s send a reply, I guess.” Then, two days later, I think I had an informal Skype call with that person and it looked like they were at Facebook office or something, Silicon-Valley-like in his background.

jam: Because you could tell.

Wim: Well, it looked like he at least wasn't in a cellar somewhere pranking me. It was somewhat legit looking. Yes, then I had phone interviews and I think I actually was asked in the beginning even to do a full-time position but I was still studying so I asked if it was possible to do an internship instead and so that’s how I ended up doing an internship there while continuing to work on that same data mining and performance data project piece of software that I started for my Master thesis. They’ve led me to an interesting place so from doing something Open-sourcey to choosing Drupal because it seemed the best, to then getting annoyed by sites being slow and then looking at how Drupal could be faster for a Bachelor thesis, to them better understanding it through my Master thesis and at Facebook and then eventually, I ended up at Acquia. It’s a path that has definitely been big, mutually influenced by Drupal.

On to making Drupal faster

jam: Your biggest contribution to Drupal 8 has also been in the performance area. Would you like to talk about caching and cache tags and BigPipe?

Wim: Sure. I've now been working on Drupal and working at Acquia full-time for about three and a half years, close to four. The first part of that was Spark, so authoring experience. That's a WYSIWYG editing CKEditor, the toolbar to it to some extent – those kinds of things.

jam: You came in during the Spark initiative?

Cache Tags, Render Caching, Cache Invalidation

Wim: Yes. So that was 2012. Then, for the last one and a half to two years probably, I’ve been working pretty much entirely on performance so making Drupal faster. A big portion of what was looking to be a good candidate for making Drupal 8 significantly faster was cache tags. That was a concept that was added a long time ago, I think even in 2011 or so. But it wasn’t really being used in many places because it was only being used in a handful of places across Drupal Core. For example, entities. So nodes, terms, users did not use them at all even though they seemed like a prime candidate. At the same time we had the concept of render caching which is basically we’re rendering something. Render caching allows you to cache the fragment of the page that is being rendered so that you don’t have to do all of the work of getting the data and then turning that into HTML in the theme system, which can take some time. The point was to use render caching in more places, the most expensive places, for example rendering entities such as nodes and users. That actually made for an interesting overlap between render caching and cache tags because when you have rendered something, you want that data to be updated as soon as the data it depends upon is changed. For example, if you change a node title, you want the render cached nodes to be updated. Otherwise, we’re looking at the old thing.

jam: Right. Now, I think that just about every one who is listening to this will know this already. But what you’re actually describing is one of the harder problems in computer science. How do you cache something and how do you find out in a cheap way when that cache should be cleared and you have new data and how do you avoid having stale stuff showing up as much as possible?

Wim: Yes. So the saying in computer science goes, “There are two hard problems in computer science, naming things and caching,” or “cache invalidation” I should say. To be clear, I did not invent cache tags. That was something that very smart people came up a long time before me. I had the ability also because I’m working on Drupal core full time to bring cache tags to many places in Drupal core and so that it’s an inherent part almost of many parts of Drupal core. So I made it sure that for example every single entity, entity type, so whether it’s config entities or nodes or terms, anything that has proper test coverage and whenever those things change, the corresponding cache tag is invalidated which then allows us to have this rendered caches and any other cache to be updated automatically, to be invalidated automatically. Indeed, it’s just a small bit of metadata that is associated with whatever is cached whether it’s rendered or computed or whatever it is. That allows us to very efficiently update those things. Cache tags everywhere makes sure that we can reliably invalidate things and reliably have things update when they should which was an impossible problem to solve in Drupal 7 and before because we didn’t have such a concept.

jam: And “performantly” as well, if that’s a word.

Wim: Yes, that’s a word. Yes.

jam: Without huge cost on my server.

Wim: Yes. There is always some cost because there is something additional that needs to happen. You’d have to retrieve something from the cache then check if the cache tags that are associated with it have been invalidated in the meantime. That’s the additional cost. But it’s a pretty small cost.

jam: That’s a much smaller cost than re-rendering the entire page.

Wim: Yes. Every single time, which is what happened in Drupal 7 before. Actually, to be honest, in most other systems. So most other CMSs on Frameworks or what you quite often see as a solution which is not really a solution is to just assume it’s okay to cache something for five or for ten minutes. But that means that if you as a blogger for example and you fix a typo, your broken title, your wrong title with still that typo in there is going to be there for another 5-10 minutes. So your changes are not showing up right away which is a very annoying, disconcerting experience.

jam: Sure. It’s a kludge. It’s a hack. I mean Cron versus Poor Man’s Cron comes to mind ...

Wim: Yes.

BigPipe, Cache Context, Max Age and the "Dynamic-ness" of things

jam: Yes. So I’m going to imbed two recent webinar videos that you’ve done on this podcast page [See links above!]. If anyone who’s listening to the podcast, in real time we’re speaking in early 2016, we recently did two webinars at Acquia about a thing called BigPipe and BigPipe is essentially the next step in this conversation. I’m going to imbed those videos and when slides are also going to be available and I’m going to link to all of the stuff that we’re talking about. We’ve got this fantastic caching architecture in place and working in Drupal 8. What is BigPipe and tell us about the magic that it does with all of this stuff?

Wim: Yes. First off, actually the two webinars you were mentioning, the first one is actually a subset of the second one so I would recommend to only link to the second one which concludes everything. So then people have one coherent story. That’s probably going to be useful to them. [Check out all the links above in this post!]

jam: All right. Okay. Now, BigPipe.

Wim: Yes. So far we talked about cache tags and rendered caching. But cache tags are not the only bit of cacheability metadata that we have in Drupal 8. We have two more. Those three things together actually allow us to know comprehensively and with complete certainty what things it depends on, what it varies by its own. So cache tags are for declaring dependencies on data, for example on entities so that we know when something has changed. But we also have cache context which allows us to define which context something depends on, what it varies by. For example, if jam has user role A and I have user role B and we have access to different things, then the outputs, the rendered HTML, should also be different. Or maybe it says, “Hi jam” or “Hi Wim,” then the output needs to vary by user. So those kinds of variations are what cache contexts are about. So cache tags and context and then there is a third called “max-age” to describe something that expires after a certain period of time. That’s less commonly necessary, max-age zero means that something is absolutely not cacheable so it needs to be requested or updated every single time. But it’s useful for things like maybe temperature data that is okay to cache for – that remains the same across say one minute or two minutes or 10 minutes.

So those three things together allow us then to know the "dynamicness" of every part of the page. In Drupal usually we have blocks. Most people build Drupal sites using the block system. So when blocks are appearing in different parts of the page, very often, some blocks are personalized. For example, the menu block below will only show menu links that are accessible to the current user. Maybe there is a shopping cart, maybe there is a “Hi, jam. Your friends have just sent you so many messages.” Something like that. So those kinds of things are dynamic. Then, usually there’s also parts of the page--and it’s not limited to blocks by the way but that’s just an easy way to think about it. Usually, you have blocks that are the same across users and usually even across everything. So for example, a menu of the footer or a search form like a search block.

jam: Or the main content.

Wim: The main content, yes. So all of those are actually cacheable across users like if it’s rendered once, we can reuse it for jam, for me, for anybody else. Thanks to those cacheability metadata, so tags, context, and max-age, we know that a given block is going to vary that much. It’s going to be stale at that point when certain entities invalidate it. For example, when jam changes his user name into something like a “Llama” for example.

jam: Just to pick a random word.

BigPipe and Perceived Performance

Wim: Yes. So the fact that we know for any given block what things it depends on, makes us able to know when something is very dynamic and when it’s not. So that allows us to pull out that part of the page, delay rendering it so that we can send the entire page minus the personalized parts first and then send the personalized parts like the “Hi, jam” blog, the shopping carts, those kinds of things, we can send those later. So the difference on this, the perceived performance. How fast a site feels, how fast a site looks and actually just how fast a site shows up. It makes it so that the sites show up instantaneously regardless of user, regardless of the complexity of those dynamic parts of the page because the parts that are the same--which is usually significant parts of a page--those show up immediately. They can be sent right away extremely fast, which means ...

jam: Usually, when I'm browsing that's the stuff that I actually care about. That’s the article I want to read. That’s the photo I want to see because that's the point of the page and that’s what everybody's getting already and that’s usually pre-cached, ready to go.

Wim: Yes, exactly.

jam: Barely or not at all dynamic.

Wim: Yes, exactly. Basically, the crucial parts of the page are usually not personalized and in that case we can make that available so, so much faster because Drupal and just about every other system out there, what they do currently is they render everything and only then once every single detail is rendered, then they send it to the end user. That makes it so that you have to wait even for the stupid, smaller things that are maybe not that important to you. Then you have BigPipe which, because of that metadata, it's just a module you can install; you don't have to configure anything. Thanks to that metadata, it can figure out which parts are too dynamic or are very dynamic or personalized. It can delay the rendering of that. Send the majority of the page first and then send the dynamic parts later. That makes for a much, much faster experience. We're trying to get that into Drupal 8.1 and it looks like many people are happy with that [BigPipe is an experimental core module in Drupal 8.1!]. It will not be enabled by default. It will even be marked as an experimental module at first because we want to make sure that it works in even the most extreme cases. So it's better to first have it experimental so sites can opt into it so we can get more experience and then hopefully in 8.2 we can make it a non-experimental module. That will be a great performance boost with no efforts basically for every site.

jam: So cache tags rendering, cache context, all of that is in Drupal 8 and on always by default and I don’t have to think about it. I’m just benefitting from your work.

Wim: Yes. Yes.

jam: As of the beginning of February 2016, if I want to take advantage of this delivery mechanism which builds on techniques, that’s called the BigPipe module?

Wim: Yes.

jam: And as of Drupal 8.1 or 2 or probably you're moving that into Core as well. [In core as of Drupal 8.1!]

Wim: Yes.

jam: Wow. Exciting.

Wim: Yes. It is very exciting. Actually, this is actually a technique not invented by us. I should say by the way that it was not just me who worked on this. Fabian Franz also from Germany by the way.

Thank you, Fabian! Thank you, Facebook!

jam: Thank you, Fabian.

Wim: Yes, thank you, Fabian. He did a great amount of work. He did the initial pioneering, the initial proof of concept. I worked a lot with him to actually make it happen and get it to a more final state, but he did a lot of the work. Even Fabian didn’t invent this. It’s a technique pioneered by Facebook. They invented or published about this some years ago. I forget the exact ...

jam: No, no, no. When you were an intern there…

Wim: No, no, no.

jam: You used all the documentation and you snuck it on to a photocopier and smuggled it out.

Wim: Then, I probably would be in trouble. No.

jam: Wim Leers, master software spy!

Wim: That’s actually a pretty cool title. I should try to get that to happen. Yes, they pioneered it. The whole point is that currently, or in the classical way of delivering webpages, what happens is first you do a request, then the server does work, you wait, you wait, you wait. You have a blank screen you’re looking at. Then the server sends a bunch of things. Then the client – the browser has to fetch all the CSS, the JavaScript, the images and can only then start rendering. So it's a sequential process and BigPipe allows us to make that a more parallel process where the browser immediately gets a response, not with everything, but with probably the majority if not all of the CSS and JavaScript and images so it can start downloading and rendering that already. Then dynamic parts show up. That’s the reason it’s called BigPipe in the sense that it becomes a bigger pipe along which to send things because things are happening in parallel instead of in sequence.

jam: All right. Wim, this is so interesting. I wrote a small post about this and I did some research into this and every new thing I find out about it, it's just so exciting. It's such a great bit of technology. Thank you Fabian from Tag1 Consulting for all of your work. Fabian Franz. Thank you Wim for your amazing work on this. I am going to link to everything that we’ve been talking about and I’m going to embed the webinar videos where people can learn a lot more about the technical nitty-gritty of all this [see links above!]. Wim, I guess you're working on getting this into Core now, right? That’s pretty much your job right now?

Wim: No, I’m working on other things as well but that is one of the things that I’m going to focus on in the next few weeks. Yes.

jam: Fantastic.

Wim: I’m very happy. I wanted to get this into Drupal ever since I read about it on Facebook’s engineering blog. It’s finally to the point where it already works. You can download it for 8.0 if you’re running Drupal 8 already. It will hopefully be in 8.1. It’s great that Open-source is then able to get this awesome technique which doesn't require any infrastructure which usually is the case for making things faster. You usually need a lot of infrastructure and money and servers. It’s just a more efficient way of delivering HTML and getting the browsers around the stuff. I'm very excited that it’s going to be available in an Open-source project like Drupal. As far as I know, nothing else has something like this so pretty cool.

jam: I'm working through the title for this podcast in my mind . It's got to be something like “Bigger, Better Performance for Free,” right? Actually, the point that you only just touched on now that I hadn’t thought of this morning of course was you don’t need massive parallel server infrastructure and all this stuff to get things really, really cracking. In this case, you get like a ton more bang for your buck out of Drupal now, just with the all of this default stuff that’s…

Wim: Yes, because usually people measure things in terms of requests per second. That is actually going to be identical with BigPipe. The entire duration of a request is going to be the same. It's just that we send useful information much earlier and then continue to send additional things, the dynamic parts later. So if you look at those traditional things to measure which are easy to measure but don't actually give you a good idea of how fast a site is, because what matters in the end is not the number of requests but how fast it actually feels for the end-user because that's what you care about and that's where BigPipe makes a huge difference.

jam: Wim, thank you for taking the time to talk with me this morning. It's been so great. It’s so interesting and thanks for everything that you've been doing. It’s great, and keep up the good work.

Wim: Thank you. Yes. Thanks for having me, and maybe see you next time and have a great day.