I will get back to my K.I.S.S. plan soon I promise. But first I wanted to put this out there.
When you order a book from Amazon, do you really care what happens between the time you click the Submit button and the moment the book arrives at your door?
Besides the confirmation and shipping notification emails you receive, your interaction with Amazon (for this transaction) ends once you hit that button.
What would you say if I told you that when you click that Submit button, a piece of paper prints out and a buzzer goes off that signals a monkey to grab the paper and run it across the room and put it through a slot in the wall? Would you care?
What if I told you that on the other side of the wall were scores of high school track team members decked out in full running gear that took the piece of paper and sprinted out of the building, down the road and into the warehouse where they put the piece of paper through another slot in a wall? Would it bother you?
What if I told you that on the other side of that wall was a series or ordinary but organized white boards that neatly listed the exact quantity and location of each and every book in the warehouse and that someone quickly located the book you ordered and wrote it on the paper and slid it through another slot on the opposite side of the room? Would you lose any sleep?
What if I finally told you that someone on the other side of that wall picked up that piece of paper, jumped on a forklift, found your book, wrapped it up and dropped it in the mail? Would you ask for a refund?
Barring the mistreatment of animals (I can assure you the monkeys are well taken care of) or the existence of child labor (all the high schoolers are of age), your answer has to be a resounding "No! I don't care"
The bigger question is: Why don't you care?
You don't care because HOW it happens is not important. What's important is the fact that it happens and furthermore that it happens in the expected amount of time.
This is of course the easiest possible example to give, and while this is certainly NOT Amazon's order fulfilling process (at least I hope not), it is of no consequence because Amazon is operating as a black box. What does that mean? That means that we only care what goes in, our order, and what comes out, our book. What goes on behind closed doors is irrelevant to us.
This is neither a new concept nor a unique one. You probably encounter such a situation about 100 times a day. Where does your food come from? What about that coffee? Do you even know how your mail gets to you other than a person drives it (or walks it) to you? No you don't and ultimately, other than natural curiosity, you don't care.
So what then is the big deal? Well the big deal is not the big black box. The big deal is what is in the black box.
Many processes are written as a single black box. Input comes in, gears turn, lights blink, data is processed and output returns. While this is still of no consequence to the consumer, the owners of this process, the ones that do know what's in the box, are saddled with one giant machine. When that machine is working, life is good, but when that machine stops working or needs new parts or is due for an upgrade, the whole operation is affected. The only solution to avoid down time is to have two giant machines and take them down one at a time. Not exactly the end all, be all solution.
What then can be done? Well, for starters, let's explore this black box concept a bit more. What makes the black box so appealing is the independence that is achieved between the consumer of the process and the process itself. As long as the rules of interaction don't change, the consumer is unconcerned with the inner workings of the process. This independence provides enormous freedom to the consumer as well as to the provider since the process can be changed freely as long as the interaction contract is upheld. Even if the contract requires alteration as part of other process changes, the consumer need only be concerned with the changes that directly affect them, and not what's going on inside the box.
Well this sounds like a nice situation to be in. Freedom to make a change without notifying consumers (we're talking technical changes here only). Even our "two giant machines" approach works here since one can be brought down, changed and then brought back up followed by the second one. The consumers have not been impacted (with the obvious exception of performance), and the process has been upgraded.
But what if the upgrade only applies to a very small portion of the process and is a very minor change? Does the whole process really need to come down for such a minor change? Will the business owners understand why the entire system is affected by something that by all accounts is minor? And what of this minor change? Is it really as minor as it seems or are there other dependencies that have not been identified? The process is one giant machine. The designs for the machine are still available, but have they been updated with every minor change that has happened since it was first built? How many minor changes happened before it was even completed?
Perhaps this change, the one that seems so minor, is not so minor after all. Perhaps it is major. Perhaps it is beyond major. Perhaps a thorough audit of the process is necessary to evaluate the impact of the change before the change is even planned. How long will that take? What's the ROI of such an allocation of resources? And what about the new features that were planned for the process? Will they now have to wait until someone figures out how to make this "minor" change? Suddenly that giant machine, excuse me, I mean both of those giant machines, have lost some luster. Suddenly things have come to a grinding halt. And why? Because you wanted to make a minor change. Try explaining that to your boss.
Ok, so what then? Well the answer lies within, so for once, I will tell you to think inside the box. The same freedom gained by the first black box's isolation can be expanded. Think of a Russian Stacking Doll. From the outside it looks like a doll. But within each doll is another doll. While this is a fairly rudimentary example, the same can be done with black boxes.
Let's look at the fictional Amazon process closer. We have already established that when you enter your order, you don't care where it goes or how it gets there. Your order has thus entered a black box. When the order prints out and the buzzer goes off, the monkey has no idea where the order came from nor does he care (assuming he would even understand but these are Amazon monkeys remember). You may have submitted it from home on your laptop or perhaps on a train with your wireless. Heck, for all the monkey knows, you could be on the other side of the wall tapping out your order in Morse code. So while you don't care how your order gets fulfilled, nor does the process care where your order came from or where it is going (of course it must know where it is going so it can get it to you and who it is going to so it can bill you, but these are simply points of data not permanent relationships).
The interesting thing here is what happens next. The monkey takes the order and slips it through a slot in the wall. At this point, the monkey is done with his job and can go get a treat. So not only did he not care where the order came from, he didn't care where it was going other than through the hole in the wall. So in essence, the tasks of printing the order and delivering it is its own process and has itself been created in a black box. This is important but let’s keep going.
Now the order comes to the high school track team members. Once again, the students have no idea that a monkey has just hand delivered this order and once again they don't care. Their job is to take the order and run to the warehouse and slip it through the wall. That's it. Job done. Get some water and walk it off. Again, black box.
The order is now in a room with hundreds of white boards all delicately spaced to avoid the accidental butt erasure. Those in the room have no idea and no time to worry about how the order got there. They need to pick it up, find the book, write down its location and get it to the other side of the room as fast as possible. Black box.
Once there, the warehouse worker has no idea that this order has passed through the hands of a monkey, a high schooler, and a slightly loopy (from all the dry erase markers) clerk. But once again, he doesn't care. His job is clear. Get the book from the location on the order, package it and mail it to the address on the order. You guessed it, black box.
We could keep going with the mailman picking up the package or the mail processor routing it correctly all the way to your friendly mailman who delivers it to your door, each one not knowing or caring exactly how the package got into their hands. Black boxes, each and every one.
So what? What have we shown? Instead of one black box, we showed several. Big deal right? Right! It is a big deal because the very freedom that once existed only between the consumer and the process now exists within the process itself! Each black box has very specific (albeit simple) interactions, namely a slot in the wall. The input? A piece of paper. The output? For all but the final step, a piece of paper. What if the monkeys become belligerent? What if they start passing their poo through the slot instead of the order (I've seen code that actually does that)? Do you shut down the whole process until you can find more monkeys? No. Simply find some well trained dogs, put them in another room, and start routing the orders to them instead. The high schoolers still get the orders and have no idea that anything has changed (although the saliva might give them a clue, but they're high schoolers so their use to that right?).
What about when all the high schoolers are taking the SATs? Do you need to shut things down? No. Simply go grab some kindergarteners (with parent approval) and buy a bunch of candy. Trust me, they'll go for hours and hours and hours and...
Don't look now, but if you haven't already noticed, no where in their little process here has Amazon actually billed you for your order. Oops! The solution? Yep, you guessed it, buy a bunch of parrots and put little colored visors on them. Have the orders from the monkeys (or the dogs if the monkeys have already started throwing poo) go to the parrots' room. They can process the billing and then pass the order through their own slot to the high schoolers and off they'll go.
The monkeys and dogs have no idea any thing has changed and neither do the high schoolers (although they are starting to ask questions about the feathers now stuck to the saliva). So just like that, the process has been upgraded and no other portion of the process was impacted.
One more. Amazon has decided that it can save time and money on white boards if it uses them to track only inventory and a sophisticated rolodex system to track the locations of the books in the warehouse. So a new room has been created to house the rolodexes. The clerks in the white board room now need to verify the availability of the book and then pass the order to the rolodex room to get the location. The rolodex operators write down the location of the book and returns it to the white board room where the inventories are updated, and the orders are passed on to the warehouse floor.
High schoolers: unaffected. They still deliver the orders the same way. Warehouse workers: unaffected, orders are still delivered the same way with the same information. The only ones affected are the white board clerks who must now perform three tasks: Check the inventory, pass the order to the locator room and then update the inventory.
The point of all of this is simple. Why create one giant process when you can create several independent processes that work together to form that giant process. In fact, many of these processes are so simple that you can hardly call them processes. Instead, let’s call them services.
I know, I could have just said that from the beginning right? But that is exactly what we've created here. Services. Each one focused on a specific task and each one using a mix of processing and other services to accomplish their work.
I know what you are thinking. "Amazon, monkeys, dogs, parrots, high schoolers; this is all cute but the real world is not that simple."
Ah, but isn't it?
I'll grant you that your black boxes will not be so easy (or fun) to design and build. It takes time and perspective to see all the pieces and to decide which of those pieces belong together and which belong apart. But don't let time and effort be the deterrents that prevent you from accomplishing the end goal.
But what exactly is the end goal? That can best be answered by returning to the single black box we described earlier. Remember that minor change that turned out to be not so minor? How much money would ultimately be spent trying to make that small change? How many person hours would be wasted? How much revenue lost because you had to delay new features that new sales were depending on? It all adds up and it all adds up quickly. It won’t be long before your competitors either catch you or surpass you in features and sales. Soon you will be the ones playing catch up and all because it takes an enormous amount of time, effort and money to make simple changes to your process.
But, perhaps your process is not like this at all. Perhaps your process is nice and clean and can be changed easily and was designed to be flexible. If this is the case then I commend you. But what about other processes in your organization? Does your process rely on any of them? Are they designed as flexibly as yours? If not, your process is at the mercy of those not so flexible processes. Maybe your process is self contained and does not have dependencies. Great, but do any other processes do similar tasks? How much time and money could be saved if those similar tasks were combined and shared?
I am not going to tell you that this is an easy thing to do. There is no magic bullet and no product that will do it for you. It takes time and it will take money to create processes and systems that are capable realizing this vision. But what I will tell you is that time and money spent now will have huge returns down the road, and if you truly commit to the plan and see it through, then those returns will not be as far down the road as you think.
