agile

What I've learnt about scaling agile to programmes

in ,

This was first published on the Cabinet Office website. Well, we did it! We delivered GOV.UK. It was big and hairy and we did it by being agile.  The benefits are clear:

  • We were more productive: By focussing on delivering small chunks of working product in short time-boxes (typically 1 week development sprints) we always had visible deadlines and a view of actual progress. This is powerful stuff to motivate a team.
  • We created a better quality product: We used test driven development; browser and accessibility testing were baked into each sprint and we had dozens of ways of testing with real people as we went along to inform the design and functionality of the product.
  • We were faster: By continually delivering we were able to show real users and our stakeholders working code very early on and get their feedback. What you see now on GOV.UK was there in February, albeit in a less fully featured and polished way. Lifting the lid on it did not seem like a big reveal; it felt like an orderly transition.

These characteristics are true of all our projects, but I wanted to talk about agile at scale with 140 people and 14 teams and what I've learnt. In so far as I know this is a relatively new area and there is not a consensus view of how to do it.

Be agile with agile

The foundation of this success was people that were agile with agile. Props to people like Richard Pope who helped set the tone of our culture at GDS, which is one where people are open to learning, improving and workspace hacks without the dogma of big 'A' agile. It’s all about that. If you don’t have this embedded in your teams then what I write below doesn’t matter.

Lesson:  a working culture that values its people and embraces experimentation is essential to success.

Growing fast is hard

We grew from a cross-functional team of 12 for the Alpha version of GOV.UK, to a programme of work involving 140 people over 14 teams. There was a period from April to June where we grew 300% in three months, which was crazy but felt necessary due to the scope of work.

Our expansion was organic and less controlled than our original plans had suggested. Had we focussed on fewer things from the outset, the overhead of getting people up to speed and the additional communication needed to manage this would have been far less and our momentum would have increased.

Lesson: We should have committed to doing less like the books, blog posts and experts say.

Don’t mess with agile team structures

Clearly defined roles within teams as Meri has said are vitally important. Our teams emerged rather than beginning the project formed with the core roles in place. The consequence of this compromise was a blurring of roles which meant that certain people took on too much. In these teams people faced huge obstacles around communication, skills gaps, confused prioritisation and decision making and ultimately productivity suffered.

Lesson: This experience reinforced the importance of agile team structures. Don’t mess. The team is the unit of delivery!

Stand-ups work for programmes

When you have a team of 10 people conversations can happen across a desk or during stand ups. As we expanded rapidly communication became increasingly strained. There were fewer opportunities for ad-hoc conversation and talking between teams was harder. We solved this with a programme level ‘stand-up of stand-ups’ attended by delivery managers from each team.

We had up to 14 people and we nearly always managed to run it within 15 mins. We held ours twice a week which gave us good visibility and opportunities to help each other. People willingly came along, which I take as a good sign the meeting was valuable.

Lesson: In the future we’ll probably split this into another level of stand-ups so we have one for teams, one for working groups of teams (or sub-programmes) and one for cross-GDS programmes.

Monitor with verifiable data

We organised the programme into working groups which had one or more teams. Most teams used a scrum methodology and split their work into releases (or milestones), epics and user stories.

By tracking these we generated verifiable data about progress, scope completeness and forecasts of delivery dates. By aggregating this data we created a programme dashboard for teams and senior management.

The image above shows the GOV.UK programme dashboard. This view of the programme was created using data generated by the teams doing their day-to-day work using tools like Pivotal Tracker and was not based on subjective reporting by a project or programme manager. The callout shows two milestones for two different teams. One was completed, the other is shown to be 90% complete with an estimated delivery date a month later than our target date. This was verifiable data based on the number of stories and points left to deliver the milestone with historical data on a team’s speed to build, test and deploy the remaining scope.

This was the bit that made me most happy and made tracking and managing the programme much, much easier than gantts.

Lesson: Use independently verifiable data from your agile teams to track your programme

Use Kanban to manage your portfolio

We used a system of coloured index cards to map out the components of the programme. This captured key milestones, major release points and feature epics and because it was visible, it encouraged shared ownership of the plan and adaptive planning throughout the delivery.

We gathered Product Managers, Delivery Managers, the Head of Design and the Head of User Testing around this wall every two weeks to manage our portfolio of projects and products. The process forced us to flag dependencies, show blockers and compromises.

Lesson: This approach was successful up to a point but in hindsight I wish we had adopted a Kanban system across the programme from the outset. It would have provided additional mechanisms for tracking dependencies, limited work in progress and increase focus on throughput. I would also formalise a Portfolio Management team to help manage this.

Everything you’ve learnt as a project or programme manager is still useful

When I started using agile, someone said me, “when things get tough and you want to go back to old ways, go more agile, not less”. This has stuck in my mind.

When I was designing the shape of the programme and working out how we would run things I wanted to embed agile culture and techniques at its heart. For example, we used a plain English week note format to share what happened, what was blocking us sprint to sprint rather than a traditional Word or Excel status report.

In tech we always say we’ll use the right tool for the job and in programme management the same is true. In the same vein I opted to use a gantt chart as a way of translating milestones and timings to stakeholders but internally we never referred to it and were not slaves to it.

Risk and issue management is an important aspect of any programme. The usual agile approach of managing risks on walls scaled less well into the programme. Typically in smaller teams you might write risks, issues and blockers on a wall and have collective responsibility for managing them. We scaled this approach to our stand up of stand ups and this worked well for the participants: it was visible, part of our day to day process, we could point at them and plan around mitigation.

But there was a moment when the management team asked for a list of risks and issues and pointing them at a wall was not the best answer so we set up a weekly risks and issues meeting (aka the RAIDs shelter) and recorded them digitally. This forum discussed which risks, issues, assumptions and dependencies were escalated. By the book it fits least well with the agile meeting rhythm but it gave us a focal point to discuss concerns, plan mitigation and fostered a blitz spirit.

We have some way to go to make it perfect but we have learned that within GDS agile can work at scale. We’ve embraced it culturally and organisationally and we’ve learnt an awful lot on the journey. Some of the lessons we've learnt have already been incorporated into how we’re working now and I look forward to sharing more about this in the future.

Teaching kids Kanban and coding

in ,

Last week I volunteered to take part in Money Day at Furzedown Primary - a school for kids aged 5-18 with special educational needs.  I've never taught before and I don't have kids myself so I really did not know what to expect but it was an incredibly rewarding and fun day. Money Day introduces primary school children to the world of work by getting them to write a CV, go to a mock job fair, take part in interviews and experience a day of work.  They even get paid a very small amount at the end of the day.  My job was to represent a pretend web-design company and teach the kids to build a website.

The day started like this...

All the kids filed in to the assembly hall, clearly very excited and sat cross-legged on the floor. The teachers set the scene about the world of work (with a cautionary word about unemployment too) and gave a run down of the agenda for the day.  There were five companies with jobs on offer: garden centre, department store, office, web design and a cafe.

There were presentations from a couple of older pupils that had set up a business selling hand-made wooden ornaments at local craft fairs, another from an ex-pupil called DJ Jack, a couple from the teachers and then my presentation. The best bit was telling the kids that when I asked them 'who da boss?' they had say that I was; on first practice I had forty kids point at me and shout 'You Da Boss!' Crowd control badge unlocked!

They then had to go off and apply for a job at the 'recruitment fair' that had been previously set up. They queued at one of the employer's desks to run through their CV's which described their best qualities, such as 'I like to help people',  'I like playing games', 'I walk the dog with my mum' and I asked each of them a few simple questions: 'why do you want to become a web-designer?'  Some of them were very clued up and were mostly only 9! I was very impressed, although one boy responded confidently 'PLANTS! I like plants!' - bless him. I sent this fellow over to the garden centre queue.

I had 5 jobs going: 1 Project Manager, 2 Designers and 2 Developers.

Screen Shot 2012-06-24 at 17.03.15
Screen Shot 2012-06-24 at 17.03.15
Screen Shot 2012-06-24 at 17.03.00
Screen Shot 2012-06-24 at 17.03.00
Screen Shot 2012-06-24 at 17.03.29
Screen Shot 2012-06-24 at 17.03.29

After picking my team and signing our pretend contracts we went back to the classroom.  I asked who wanted to be the project manager and a confident chap called Bailey put his hand up, I appointed him and he obediently replied 'Ok Boss!' There was less certainty about who'd be the designer or developer.  Playing with computers and taking photos held equal appeal.

Objectives, design rules, users and our backlog

I set  our objectives for the day, which were:

  1. Tell the story of Money Day
  2. Publish it to the the Internet and tell people

We discussed what makes a good website and a bad website. They all love CBeebies and YouTube; they'd all heard of eBay and Amazon but they reckoned the key to a good website was 1) brilliant backgrounds, 2) clear layout and 3) good information.   These became our design rules.

Here's Bailey posing in front of our first card wall.

Bailey, the Project Manager
Bailey, the Project Manager

They very quickly identified that the number one user for this website was going to be their parents, followed by teachers (theirs and from other schools) and lastly their friends.  They got to this point unprompted and with real confidence which I thought was remarkable and exactly the answer I would have given.

We then created a backlog of cards, imagining ourselves as parents, teachers and friends and discussed prioritisation.  The two cards we prioritised were 'What kids are doing today' and 'comments about Money Day'.  The designers quickly went off with the cards, their cameras and notebooks around the school and the developers helped set up the FTP software and create a basic HTML template.

Backlog and users
Backlog and users

Multi-disciplinary team delivers

When the designers came back they selected some images, we did some pairing to reference the images in the HTML file, added some text (their words) and we were good to deliver our first website. Bailey sounded the bike horn, we high-fived and he awarded the team with stickers - just like real work.

The Delivery Team
The Delivery Team

Instant feedback from 'users'

I tweeted that we'd delivered the site and asked followers for feedback.  We got some lovely replies back and the kids could not believe that grown-ups with proper jobs had seen their work.  It helped that I was able to say 'this person works for Channel 4, this person worked at the BBC...'.  They loved it.

twitter-replies
twitter-replies

What Money Day taught me

Teaching is bloody exhausting.  I'll be careful not to take the piss out of teachers getting home at 4pm and taking long holidays.  I think I passed out on the sofa even before my nightly fix of Newsnight.

The class room was very similar to my workplace.  There was bunting everywhere- just the GDS office - lots of bright colours and cards on walls, stickers, toys, progress charts and circle-time.  Actually, the smaller seats suited me quite well given my little leggy-wegs.

And then there's the stuff that people desperately try to fight back.  In my world it's the gantt chart, pointlessly long documents and corporate buzz words; in teaching it's words like 'step vocabulary, plenary, WALT' and the rigidity of tests and the National Curriculum over creative learning and singular focus on the children rather than The System.

The kids totally got the Kanban wall by the way and the idea of delivering things incrementally. It made perfect sense to them and I did not need to explain anything twice.  In fact one of the teachers said she might start using it to organise their days.

It was great to see that they were excited by the Internet and making things and I would recommend the experience to anyone. I had great fun and all my team would would fit in perfectly well in a workplace like mine. I'm sure their parents would be proud.

Agile does work in government

in ,

Some feel that agile is poorly suited to government and is just a fad. One of the principle objectives for this project was to demonstrate that an agile approach can work. For us that meant delivering a prototype that:

  • demonstrated a user-centred central government, single domain website
  • used a small, multi-disciplinary team of developers, designers, editors, sat in the same room
  • came in for a budget of £261k and was delivered by 10th May

I drew upon Scrum for a structure to manage the project but I knew not to over do it.  In my experience projects can quickly become about the process and not the product, with everyone getting hung up on big ‘A’ Agile.  I claim no particular expertise – I am a practitioner and you have to allow room for pragmatism.

Working iteratively, within an agreed upfront budget and time box allowed us to get a prototype out the door quickly.  In that time we were able to deliver two releases of the software before finally releasing to the public 3 days ago. We have been pretty open throughout the development and that has been useful for us as a team and hopefully for those people outside of the project room with an interest too.

This is my view of how things developed and some of the challenges we faced.

Obsession with user need

We spent the first two weeks in February recruiting a team from inside and outside of government, talking through the scope, agreeing some design rules and agreeing a vision for the Alphagov product based around the recommendations of Martha’s report.

We ended those two weeks with a list of priortised user needs (based around search analytics from Directgov, Hitwise and departments), roughly grouped into functional areas and stuck to the wall. Each card (or user story) represented a user need, prioritised roughly from left to right and top to bottom.

Crucially also there was a fair amount of@tomskitomski and @memespring‘s product experience. All this was more than good enough to get on with twelve weekly development sprints.

Small ‘a’ agile

We kept the rhythm of the project with daily stand up meetings, weekly show and tells, and certainly in the early stage of the project we’d get our heads together to work out as a team what worked and what hadn’t.

We talked a lot – regular interruptions across the room were accepted, sometimes there were creative tensions but crucially everyone worked alongside each other in the same room with a focus on delivering a product.

The product evolved as we moved through the weeks and our view of the product changed.  A language developed around ‘glue’, ‘gov’, ‘tools’ ‘guides and answers’.  User stories cards became a mix of user needs and  placeholders for a bunch of tasks.

We used a backlog and estimated the size of each story card  to negotiate the weekly work loads. But the team did not get hung up on the concept of velocity (apart from @jamesweiner who thrived on story points and would try to game the system to deliver a ‘winning’ week).

Mostly though we organised ourselves around what felt right.  In fact, for the last two weeks we relied totally on a list on the wall of ‘things to do’ – because it was the quickest and simplest way of polishing up what we’d already done.

The now infamous Sprint #5!

We (or rather I) didn’t get everything right. By the end of sprint 5 I was just about ready to hang up my Excel and opt for a life at the RSPCA.

One of the challenges was balancing momentum with getting the team together.  I started on the project in the last week of Jan, by Feb the 5th we had Richard on board along with Jimmy, Dave, Helen, and Peter.  It wasn’t until mid-March that we had afull team in one place all of the time.  This was disruptive and made syncing design and development through iterations of development and working together that much harder.  In Sprint 5 we only delivered a couple of stories and Sprint 6 was equally tough but thanks to a monumental effort and some pragmatic concessions, we delivered.  From then on we properly started to fly.

A talented, self-organising team.

Most of the Alphagov team are individually capable of designing and building digital products and despite everyone having a role with a title, the boundaries are difficult to establish in practice.  We knew who the drummer was, that be Russ, but everyone was capable of being the lead guitarist – so to speak.

Like every other project team it took us time to settle into each other but by working so closely in one room, with space to draw on the walls, with the minimum of interruptions and a single focus around product we were able to produce a lot in a short space of time.  We launched a day later than we announced, partly to do with the rhythm of comms, but happy to suck up an extra day to tweak and take the password off in good order.

The quality and amount of feedback and active debate on Twitter, Facebook andGetSatisfaction since then has been phenomenal.  We are responding to feedback and able to iterate the product day by day. That’s got to be a more effective way of consulting with people to design products that help them deal with the Government online.

This project was not just about the team in the room. A tip of the hat to Chris Chant and Tony Singleton for taking a chance and giving us the space to do this, and to  Jimmy Leach, Neil Williams, Darren Leafe, Simon Everest, David Pullinger, Neil Franklin and many others in the Civil Service that helped make it possible.