Pilots vs. Experiments

I have recently been working on some internal proposition development, expertly facilitated by Matt Cushen, and he brought something to the table today which really resonated with me: the key differences between pilots and experiments.

Pilots are typically fully formed ideas which can be expensive, involve a lot of resource, are long term, and where the risk of failure is high.
Experiments can be just part of an idea, frequent, quick to deploy, with a focus on learning, and where the risk of failure is low.

Albeit subconsciously, I’ve certainly observed these differences and the associated difference in behaviours during my career, and today this led me to reflect on a recent piece of work we delivered for a client in a way that I hadn’t previously considered.

Right at the start of lockdown I was approached by one of our clients with a simple question; “could we do deliveries?” Sure, delivering stuff to stores, or managing the logistics of a large brand experience event is a critical part of our day job, but we would in no way consider ourselves to be delivery company!

My reflection today was around how this might have transpired under “normal” circumstances; if the coronavirus impact had never necessitated the immediate pivot that so many businesses have had to employ to survive.

Firstly, it is almost a certainty that the brief would have never crossed my desk. Why would a client with a commercially successful business model feel the need to flip the status quo and divert so drastically from their current operation? The logic just doesn’t make sense on the face of it.

Secondly, if for whatever reason this had been an idea from the client, I can easily see how that plays out in reality; something we would have all seen at some point in our careers. A lengthy process to scope it out with no fixed timeline or end point. Endless project meetings. A huge group of stakeholders. Iteration after iteration of business case. Every possible scenario meticulously thought through and documented in a process flow. Multiple levels of sign-off, maybe finally agreeing to run a pilot with a subset of customers or in a selected geographical region… and so on and so on.

If after all of that it was decided that there might be a commercial case to answer and the brief finally landed with us, we’d no doubt have a similar process to run through. How will this work? Who needs to be involved? How do we recruit? What systems do we need? How do we price it? When could we start it?

Don’t get me wrong, getting this particular job off the ground was a herculean effort by the iD team, particularly under lockdown conditions, trying to bring this to life in a matter of days via video calls! But it was a small team, very much focused on turning an idea into an operation as quickly as possible, with limited resource and budget. Consequently, we’ve all been able to learn as we go, focusing our efforts only on the elements which require it, and optimising the parts which need optimising. We were constrained in many ways, but equally free from the usual constraints in others, it was invigorating!

When we first launched, we were using various Excel spreadsheets to manage the order and delivery status. We begged and borrowed iPads from other projects which we could turn into mobile EPOS systems straight away. We were briefing our frontline team on the fly and flexing the process daily based on their feedback. None of this was perfect, but with a single-minded focus on getting things up and running immediately, it worked and served its purpose phenomenally well. Three months in and we have since invested in using app-based delivery management tools. We’ve automated our reporting. We have glossy briefing packs for new delivery recruits. We’ve established how and where we can be more efficient, whether through direct cost or time. Crucially, we’re focusing only on the 20% of project that needs optimising, rather than the 100% of the project that we’d still be sat around discussing had it not been for the incredibly time-sensitive nature of the Coronavirus impact.