Quick starting BSX: Part II – the Delivery Process
May 7, 2015
We have a contract, which means it’s time to execute. Normally, before signing a contract, I would have already arranged for a discovery process. We use this process to essentially identify the user personas, perform user research and product research, agree on what we all want to build (and we means both the customer and MM), declare what is the MVP, create a release backlog, and then begin the delivery process. Normally, members of the delivery team are part of the discovery process so that we can nicely segue into delivery. Since the customer wanted a quick-start, all of that was off the table.
First thing, we had to identify a delivery team. Since the pursuit of BSX had started, I had already reached out to some of the functional managers at MM such as the iOS, Android, and Web departments in order to let them know that we had a hot pursuit. As I continued to learn more about BSX, I realized that we needed to start fast.
One of the things I proposed was to make a hybrid app using web technology. Doing so would allow one small team to develop the app vs an Android and an iOS team who would have to coordinate with each other. This results in one code-base (which is arguably gray area since plugins are involved), a common set of tests, a common set of bugs (hopefully), and a single user experience. All of these points are efficiencies that a delivery team could gain, and since the MM web team (who is an amazing team of engineering wizards) recently figured out how to build and use their own Cordova plugin for BLE applications, this was a no brainer.
It also made sense for the customer, since their server team was comprised of web engineers. This meant that if BSX wanted to take over development of the mobile app in the future, their knowledge of the web stack would directly apply to the web stack that would be used to construct the mobile app. Our engineering approach was set.
Any software engineer who has had the pleasure of working with a top-notch QA engineer knows that they can be a real life saver. The MM QA team is the best QA team I have ever worked with in my career, PERIOD. While we were pursuing work from BSX, I had asked the MM QA team to please help me design a testing strategy for the Insight product. We quickly learned that a sports science product was unlike anything we had worked with before. It was arguably a new class of product. Could MM test it?
Well, in time we could have figured out how to do so, but we determined that this product needed to be tested by elite/professional athletes. Why? Well, for one, our QA team couldn’t regularly offer blood samples to compare the results of Insight to a lab test. Doing so wasn’t practical. Second, not everyone on the MM QA team is an athlete, and to my knowledge, no one is an elite/professional athlete. So, with our solid relationship with BSX, I called Dustin and explained the situation. Being the practical person he is, he agreed to assist us.
At the beginning of the development cycle, the software developers performed basic testing of the user interface. At the same time, Dustin looked for a QA engineer to join the BSX team who was very familiar with the science of testing and who was also an elite/pro athlete. BSX delivered, and thus became the owner of testing the mobile product as we built it. Some delivery professionals might think this was a crazy idea, as most engineering teams (myself included) prefer to own the discipline of testing. Nonetheless, the decision made sense and it ended up being the right thing to do, as the BSX test cycles were fast, thorough, professional, and riddled with fun.
Another efficiency that we applied were the test devices. We decided to test on iOS 7 and 8 using an iPhone 5, 6 and 6+. For Android, we selected OS version 4.4 and solely tested on a Nexus 5. We essentially decided to limit the scope of devices to test on and to bear the brunt of this decision by reacting to problems that would surely be reported in production by users of other devices. Some of you may be shocked by this, as many of our customers choose to test on many more iOS devices and up to 20 Android devices, but we (“we, meaning both MM and BSX–we’re all in this together) knew that the rollout of this product would be slow enough to mitigate the risk.
For the record, the BSX QA engineer chose to test with a few other devices near the release based on user feedback, which I think was a healthy decision as it was a data-driven decision vs a prediction-based decision. But at the early stages of the project, it was more important to build the right product vs ensure that the product works on every popular device being used in production. Plus, this single decision would greatly reduce the scope of the QA effort which at the time was not well known because the product we were building was revolutionary.
Server, analytics, algorithms
I quickly learned that the head of the BSX software development team was sharp and skilled. The server team was ready to create APIs for us using technology that I felt was appropriate and that would allow for fast delivery of rest-based services. Huzzah! In addition to this, BSX would own the development of all of the analytics and algorithms that would be needed to achieve success. This was huge as it meant MM could focus on building the user experience.
We’ve come to what I believe is the most important part of a great product, the UX design. Don’t get me wrong, the engineering needs to be awesome and the quality of the product needs to be awesome too. However, if your UX stinks, no one will care that the engineering is awesome and you won’t get to affect the quality of the product, because no one will use it.
I reached out to the MM design team early in the pursuit and asked the managers to please identify a team member who is ideally an athlete and who would be thrilled to design an experience of a digital product for an athlete.
By now we have an engineering team (Web), a testing plan (BSX owned), a product owner (Dustin), design (MM), a server (BSX owned it), a PM (MM), and a sensor (developed by Sparx Engineering). It was time to build.
Less than a week after signing the contract, BSX, Sparx and MM all convened in Houston, “Home of BSX world headquarters,” for a one-day, 12-hour, Friday summit. During that time, we filled up a huge whiteboard with all of our concerns. It was an awesome experience. All of the previously mentioned disciplines were in attendance. While BSX already had a spec for their product, they needed to go through one more iteration in order to have something that MM could use to develop a mobile app.
The summit was a UX driven effort, which was perfect. We debated many scenarios about how the product should work. By the end of the day, we all knew what we needed to build. We decided to go with a very lean project management style of approach; Kanban. We documented the high-level epics in a simple spreadsheet, prioritized them, and got to work the following week.
What made this approach especially wonderful is that the customer participated in the design of the mobile product every day and continues to do so while the team prepares to launch another release of the mobile product.
Make a prototype first
There’s a cycling conference in the US named Interbike. The conference was to take place in 2014 around October. BSX had plans to show their product at this conference. This was a very good idea because, as MM prefers, we believe in building products (quickly) that work. What we initially build will certainly not be the end product, but there is no substitute for a working prototype that you can analyze, glean feedback from and improve. This is the heart of iterative development. The sooner we can build something that works, the sooner we can improve it. We at MM certainly prefer to start with a small team to build something and then let need be our guide. Fortunately, BSX did too.
The prototype of the app almost worked at Interbike. The app was shown to users, but BSX ended up using their backup plan (a PC using the ANT+ protocol) to control the Insight device. Nonetheless, the foundation that was built for the mobile app was solid and it continued to be iterated upon until the product was launched to production. The feedback that we obtained from the prototype was invaluable.
Each day, we would participate in a one-hour standup. “WHAT?” says the agile and scrum evangelist. This is a technique that we have used at MM for quick-start projects and for projects that involve a small, hyper-efficient team. We had a one-hour meeting each day, during which time we would set daily goals. The team would then accomplish the goals. Or, sometimes not. Or, sometimes achieve more goals. In the end, it balanced out. The best part was, we only had one meeting per day; we spent the rest of the time building the product.
Much of the first 7 weeks of the project was spent iterating on the design with the product owner. Nearly every day, the MM design team would submit designs (both VD and UX) to the customer, which we would all review during our daily meeting. Engineers would give feedback, the PO would give feedback, the server team would give feedback–everything centered around the design.
At the same time, the MM engineering team was setting up the technology that would be needed to solve the obvious problems such as authentication, BLE operations, and hooking up to the API, which the BSX server team would build on the fly. As soon as a design was approved (such as the login screen or the profile screen), the MM engineering team would start building it.
This was an amazing experience. We all knew what the goal was, and we all worked through the problems and frustrations in order to experience the irreplaceable joy of a job well done, as a team.
We knew that there was no time to validate the user experience in advance, so we all chose to build what we all thought was the right product–based on very thorough, frequent, and kind guidance from the product owner–to start testing with real athletes as soon as possible. We knew this would be a problem since BSX had to procure the necessary components to build the production version of the Insight sensor and doing so would take time.
Yikes! This is something that non M2M projects don’t often have to manage. With traditional mobile software projects, the app is the product. With M2M projects, the app is a portion of the product, with the hardware device being the other piece of the puzzle.
In this case, the hardware device was comprised of many pieces. It also had to be sealed so that perspiration from athletes wouldn’t affect the electronics. It needed product packaging, a charger, a marketing campaign, and the list goes on. BSX had to manage this process in addition to helping us manage the construction of the mobile app. They did a great job. So, during development, we used early versions of hardware units to test with. It wasn’t until much later that we would gain access to the product versions of the sensor.
When the production versions of the sensor arrived, about 1.5 months from the launch to production, BSX started user validation testing. In the end, the original design stood up to the tests pretty well. The MM design team is truly amazing. Sure, there were changes that had to be made to the design, but our combined efforts and strong collaboration skills ended up guiding our decisions very well. We all predicted and expected that the ecosystem wouldn’t behave the same for all athletes, since not all people are built the same, which meant that the algorithms had to be tuned until a sweet spot was found. This was a very hard and time consuming problem for BSX and their analytics partner, but they got the job done.
The Insight product launched in January of 2015, just 5 months after we were awarded the work by BSX (WOW). As for the Insight product, it is currently receiving rave reviews from the first wave of athletes, and will undoubtedly continue to do so as more marathon runners and cyclists get their calves on it.
What we learned
Work with customers that you like, focus on the product, let the user be your guide, build things as soon as possible, iterate on what you’ve built, and above all else – collaborate! If you’re all on the same team, you will succeed.
Did you miss Part I, and want to see how the Mutual Mobile/BSX partnership came to be? Read the prequel here.