179: Deploying better software with confidence with Behat & BDD - meet John Bickar

While speaking with Melissa Anderson about behavior driven development (BDD) at BADCamp 2014, she suggested I get John Bickar from Stanford Web Services in front of my cameras to talk about his experience during last year's "Drupalgeddon" security vulnerability. The result is this podcast and some great insight into how this kind of testing can significantly improve initial, ongoing, and emergency delivery of software. As John puts it, using BDD means: "delivering better software, delivering it faster, and knowing that it is delivering the value that we have promised to our partners." I would have named this episode of the Acquia Podcast more in the spirit of Dr. Strangelove: "Behat tests mean death to Linky-Clicky or how BDD helped Stanford Web Services recover fast during Drupalgeddon," but reason won out.

We also touch on the changing nature of digital needs and access in education in the last few years and how that affects how and what gets delivered through digital channels.

Note: Funny camera angles! – I recorded this interview the very first day I had my new, 2-camera setup. I was doing some guessing, especially on the 2nd one, which hasn't got a monitor screen, so in the end, let's say the angles are "interesting" :-)

What is Stanford Web Services?

"Stanford Web Services is an in-house Drupal development shop. We do Drupal site builds for partners on campus. We're also an infrastructure and hosting organization. We run a software-as-a-service [SaaS] platform that we call Stanford Sites. It runs mostly Drupal 7; we started it in Drupal 6. And we maintain this self-service website building platform for anyone at the campus to use for free. We also make much of our code available for free to the campus community and the wider Drupal community," including a responsive theme, and several features and modules.

What is Behavior Driven Development?

"Behavior Driven Development [BDD] is a process by which you define your user stories and your business values based on what a user of the website wants to get done and why it matters to them. BDD then allows you to write those steps down, in plain English and you can then use that to build towards "complete". So when those use scenarios and those features are complete and testable, then you know that you are finished." The creators of behat ("an open source behavior-driven development framework for PHP 5.3 and 5.4"), describe it thus: "It’s the idea that you start by writing human-readable sentences that describe a feature of your application and how it should work, and only then implement this behavior in software." Thinking one step further, once your site is in production, behat tests help you prevent functionality-regressions in future releases, too.

Build towards agreed-upon business value with BDD

John and Stanford Web Services started using BDD and Behat "as a way to do functional testing, but we've really started to adapt the BDD part because it allows us to get everyone involved in writing the user stories and in creating and agreeing on the business value. And then the developers, we know what we're building towards."

"We care about delivering websites that work to our clients. We care about partnering with the various stakeholders on campus and delivering cutting-edge communications. That means it always need to be up, it needs to deliver the information that they need. BDD is a way that we can meet that in a rapid and accurate manner."

"There are a number of ways that we use BDD to deliver better software. We use tools that are part of the BDD toolkit to ensure stability when we make changes to code. So at heart, the function of what I do is that I am a software developer, but no one cares about the code that I write ... and I can't blame them. They care about whether or not they can do the things that they came to a website to do. When we push out code updates, we use tools like behat and Mink [an open source browser controller/emulator for web applications, written in PHP 5.3.] to ensure that the business value – the things that the users want to do – remain intact. It's incredibly valuable because it allows us, as software developers, to completely refactor the code underneath and ensure that the functionality of the website still remains intact.

Delivering better software, faster with BDD

John got interested in BDD, spent some time getting it up and running and created a few sample tests that he showed to his colleagues at Stanford Web Services. "Everyone immediately saw this could be. Our method of QA when we would launch a website used to be literally, 'click through the top-level navigation, click through the sidebar navigation.' It was a very sophisticated methodology that we referred to internally as 'Linky-Clicky.' Linky-Clicky was not sustainable, so our ability to automate rote tasks and check for navigational elements, check for functional elements has completely transformed the way that we work because it allows us to focus on building things that matter and not spend time doing QA on things that should just still be working." It also eliminates a lot of room for human error.

"When I first showed it to my team, everyone saw that we could automate a lot of the things that we were doing manually and remove the human element and remove the human error because when you're clicking though, you miss things. And if we could define those tests in a programmatic way and run them in a programmatic way, all of a sudden, we can focus on delivering better software, delivering it faster, and knowing that it is delivering the value that we have promised to our partners."

Build what matters: automated testing with BDD

"We have a line of products for the university that we call 'Stanford Sites Jump Start'; a set of Drupal installation profiles that come with default content, default menu structures, default layouts, roles, things that people should do. In these installation profiles, there are certain things that people should be able to do in various roles. As an example, one of the features that we just rolled out was the ability for site owners to go in and change the layout of the homepage."

The business need can be expressed as a "Feature" and the BDD test as a "Scenario" like this:

    As a site owner
    I want to be able to change the layout of my homepage
    So that I can respond to varying needs of communication within my department.

    Given that I am logged in as a user with the site-owner role
    And I am on the customized design page
    When I click the select button for a the 'Lomita' homepage layout
    Then I should get a different set of selected options
    And when I am on the homepage, I should see element X in region X and element Y in region Y

(the last line above tests for the blocks being in the correct configuration)

And ...

    Given that I am logged in as a user with the editor role
    And I am on the customized design page
    Then I should see: "You do not have access to this functionality"

Keep what matters with BDD testing

The Stanford Jumpstart line of products is in its fourth major iteration. "Each time we restructure things, it has been a monumental undertaking to make sure that we haven't broken pre-existing functionality that users have come to expect. Now that we have tests written for this and we've made the business value explicit for these operations, we can refactor the underlying code to fit what the developers need and fit what the project managers need, and what the Web Services teams needs and still provide the same user experience to the end users of the product."

Fast, confident emergency deployment: Drupalgeddon and BDD

"Drupalgeddon was a wide-scale SQL injection vulnerability in the Drupal CMS. That means that all websites created with Drupal 7, as of 9 a.m. Pacific Time, October 15th, 2014, had a severe security vulnerability that could be exploited from anyone in the world." Learn more about Drupalgeddon; Acquia's Moshe Weizman wrote a great blog post – Shields Up! – about it on Acquia.com. Another excellent post by Campbell Vertesi also includes his approach to Drupal security from now on, which includes BDD and behat: Drupalgeddon: Best Practices Aren't Good Enough Anymore.

"What [Drupalgeddon] meant for Stanford University is that all of a sudden, we had easily 1500+ websites that were vulnerable. When the announcement came out that Drupal 7.32 was released that patched the [vulnerability], our team scrambled to assess what the vulnerability was, determine if it affected us, tried to determine the scope of how it could impact the websites under our purview, and determine a course of action. We reviewed the patch. It was very small; one line. We analyzed ways this change could affect our product."

"We have over 1400 websites on Stanford Sites that are in Drupal 7 and we're using around 100 contributed modules. [The patch] was a change to the database layer. We very quickly analyzed and came up with a few situations or things that could be affected. We did a run through the codebase and saw that a lot of contributed modules use [the affected] function call. So we had a good idea of the base of contributed modules that could be affected."

"Fortunately, we had a broad suite of behat tests – functional tests – for all of those modules. We were able to run those tests in about 30 minutes against a testing environment that had been upgraded to Drupal 7.32. All of those tests passed, so we could communicate to the infrastructure team that this was a change that needed to be made on an emergency basis that same day. We had deployed the update to a testing environment. We had tested it and we had full confidence that it could be rolled out without any user-facing impact."

"What it gave us was a huge degree of confidence that we could roll this update out and it would not cause any regression errors. We don't have full test coverage. We have focused on providing coverage on the things that are the most valuable, the things that make the most sense or that mean the most. Despite not having complete test coverage, we have enough that we could roll out with 95% certainty that it isn't going to break and that this is severe enough and important enough that we need to roll this out right away. Without the test coverage that we had, it probably would have been days before we were able to get that patch rolled out."

Guest dossier & resources

  • Name: John Bickar
  • Drupal.org: john-bickar
  • Work affiliation: Drupal Web Developer at Stanford Web Services, "an in-house Drupal development shop. We self-host a Drupal software-as-a-service platform at Stanford University. We currently have about 1500 websites on that."
  • LinkedIn profile
  • GitHub: jbickar
  • In a session proposal for the Stanford University 2014 IT Unconference, John includes good explanations of some behat basics.
  • Doing Behaviour-Driven Development with Behat – DrupalCon Amsterdam session by Inviqa BDD Practice Manager Konstantin Kudryashov.
  • Large Scale Drupal/ Drupal Watchdog Behavior Driven Development Guides
  • Behat Drupal Extension - "The Drupal Extension is an integration layer between Behat, Mink Extension, and Drupal. It provides step definitions for common testing scenarios specific to Drupal sites."

Interview video