Manufacturing Capacity and the Organ Grinder’s Monkey

This is a story about a young entrepreneur named Bill. Nearly two years ago, Bill started a high-tech manufacturing company and stopped sleeping.

Bill spent the first 18 months of his journey working on a business plan, raising capital, and starting his operation. Bill is exhausted and nervous and excited and scared and enthusiastic and overwhelmed. But, his idea seems to be working and there are some hopeful signs that his enterprise may actually be a success. This makes his board, his investors, his wife, and his dog happy.

Now, Bill is faced with a decision. Based on the production forecast in his business plan, his company (SiriusCyb, Inc.) will exceed the manufacturing capacity of its current facility in the next 3 quarters. Manufacturing space that can handle the type of equipment and processes that SiriusCyb requires is not very common and is expensive when it can be found. Also, the process of outfitting a new facility and moving his machinery will take at least three months. His realtor has let him know that a facility has just come on the market, but he would need to act quickly in order to secure it.

Signing a lease now would be slightly early but workable IF his production volume ramps according to his projections. If it doesn’t, then Bill will be spending a LOT of his investor’s money on non-revenue generating manufacturing space. This is capital that he had to travel around like an organ grinder’s monkey to raise in the first place. So, he doesn’t relish the thought of trying to do it again – this time because of a mistake in timing.

On the other hand, if he does not grab the space, he could be in a situation where he is turning down or delaying business until he can find another facility. This would make his board, his investors, his wife, and his dog irritated.

So, what to do…. pull the trigger now or wait a little longer to see how SiriusCyb develops? If only he only he knew what his actual production volume would be by the end of year one…..

Luckily, Bill is not without assets. First, one of his board members helps direct several other young manufacturing companies and has become somewhat of a mentor to him. She is also very well connected within the venture community.

Second, Bill has a brother-in-law who has a job in the decision sciences field and an occasional drinking problem. More than once, Bill has bailed him out of a sticky situation without telling his wife. So, he is owed a favor.

Bill knew that, while his mentor would be happy to help in any way she could, she would be expecting him to be the ultimate decision maker. So, before he discussed the lease any further with her, he decided to call his brother-in-law at work (where he knew he would be sober). The conversation went something like:

Bill: “….., so I need to determine my actual future production volume per month vs. my forecast.”

Brother-in-law: “I’d love to help but my expertise is in working with really large sets of data. So, unless you can find a database containing recent historic monthly volumes from companies like yours when they were in the same development stage that you are in……. ”

Bill: Long pause. “Sorry, I was just reminiscing about the time that I drove all the way over to that bar to …”

Brother-in-law: “Okay, okay, okay … Let me think about this more today, and I will call tonight if I have any ideas.”

Later that evening, Bill received an invitation to a Zoom meeting with his brother-in-law who started off with, “Alright, maybe I have an idea. Like I told you this morning, if your question is: ‘How do we produce a better forecast when we have no data?’ My answer is still ‘We can’t.’ Hold on. Stop. Hear me out.”

“The problem is that the future is unknowable. So, no one’s forecast will ever be completely accurate. Because you are in a nascent industry, you are the leading expert on your company and your market segment. But, you are smart and spent a lot of time and effort producing this forecast. Right? The point is we would have no reason to believe that, even if someone were to use historic trends from their industry, their forecast would be any more accurate than your current one. Your forecast is the best you are going to have.”

“So if we travel down that line of thought, then maybe the question is not ‘How can I make my forecast more accurate?’, but rather ‘How much confidence do I have in it now?’ Or, to make things even easier, ‘At this time, do I have enough confidence in it to risk contracting for the currently available manufacturing space?’ Right? I mean, you don’t really have to quantify exactly how much you believe in it if there is just some threshold below which you would decide to wait. Does that make sense?”

Bill took a few seconds to try to decide if his brother-in-law was on to something or just trying to weasel out of helping him. He couldn’t come up with anything obviously wrong with the theory so he said, “I could probably go with that. But it sounds like figuring out my level of confidence would be just as difficult as finding a more accurate forecast since we have no data.”

“Maybe not. Can you show me your actual forecast?”

Bill pulled it up on the screen¹:

His brother-in-law looked at it for a few minutes and said, “So, you’re saying that you need to be manufacturing 1,050 units by month 12 to make a new facility work?”

Bill replied, “Well, that’s my forecast. But, if I had orders for 850, I could still justify the expense of the larger facility.”

“Okay. So, how confident in your forecast do you need to be before you would sign the lease?”

Bill thought for a second. “Well, very! It is a big risk. So I think I need to be at least 90 percent confident that I would be turning out a minimum of 850 units by month 12.”

“Got it. What would be the lowest possible volume you think you would be at in month 12?”

Bill turned slightly pale and, after a taking a deep breath, lied. “I really haven’t put a lot thought into that.” “But, well, let’s put it this way….if I am still manufacturing 600 units by the end of year one, I would probably have to re-think this whole enterprise.”

Bill’s brother-in-law said, “Okay. Hold on for a minute”, and worked on his laptop while Bill tried to clear the image of failure from his mind. Finally, his brother-in-law raised his head. “Okay. So, given what you told me, the probability distribution of units produced in month 12 would likely look something like this. Agreed?”

Bill is an engineer by training so he understood the distribution but not where his brother-in-law was going with it. “I guess. What is your point?”

“The point is that if this distribution is reasonably representative of your situation, your probability of producing at least 850 units is around 92%.”

Bill’s first instinct was to be thrilled with the result. He would have loved to believe that his problem was solved this easily. But, his brother-in-law did not look nearly as self-satisfied as he would have if he had actually come up with a useable solution. So, Bill stayed quiet and waited to hear the rest.

“The trouble is that this doesn’t really tell us much more about our confidence level than what we already knew. It is really just a more detailed description of your existing forecast. So, if your forecast is not valid, then neither is the distribution.”

Bill was still formulating a sarcastic reply when his brother-in-law hurried on. “What it does do though, is allow us to start working with a range of possible outcomes rather than the one value that you forecasted. It is a way of describing your estimation that you will produce around 1,050 units in month 12 and maybe a little more. But, a number as low as 600 is not outside of the realm of possibilities – it is just very unlikely. Now, we need to test this distribution against some data.”

It had been a long day and Bill couldn’t help himself. “So, thanks, we are right back to where we started.”

“Not really. Because, remember, we are not trying to produce a forecast. We are just trying to figure out how confident we should be in your forecast. Or, in other words, ‘How good you are at forecasting?’ ”

“When I was thinking about this today, I remembered a book excerpt from Daniel Kahneman where he recommended comparing an Inside View with an Outside View. The inside view, in this case, would be your forecast. You produced it to be as accurate as it could be at the time. But, you know, everyone who has been in your situation has had to produce a forecast of their own. So, maybe what we should be comparing is not production volumes but the ability to produce accurate forecasts. If we do that then the Outside (or baseline) view would be ‘How accurate were others in your situation?’ More specifically, ‘What percentage of their original forecasts did others actually achieve?’ Let’s face it, if their companies in any way resemble yours, then they probably used the same techniques and methods to produce their forecast as you did. So, do you think there is any way that you could get a few companies similar to yours to give you that information? ”

Of course, Bill thought about his board member and mentor. “Possibly. So, you are not asking for their production numbers. You just want what they actually produced during each time period divided by what they had originally forecasted they would produce. Right? I doubt that they would consider that proprietary – especially if the request came from the right person. Let me get back to you.”

The next morning, Bill called his mentor and tried to explain what he was asking for. He had to go through it several times (because he was still trying to explain it to himself), but she finally understood and applauded him for the effort. As she said, “For the kind of money you are talking about committing, we need to be really sure that you need the space.” She could think of three companies that might be appropriate and, since she was on the board of all three, did not foresee any difficulties. In fact, she still had the companies’ original presentations so all she would really need was their actual production volumes. Within three days, she sent him an email with the information and told him she was looking forward to seeing his analysis.

Bill looked at the email before breakfast and immediately wished he hadn’t. All of the companies had missed their forecasts on the low end. So, not only was his confidence shaken but now his mentor was looking for an “analysis”. He no longer felt much like eating.

Bill’s first idea was to put the information into context by multiplying his forecasted units by the percentage of their forecasts that the other companies attained.

This created a nice visualization, but it still left him guessing about his confidence level. He knew that it would be lower and maybe much lower. But how much? Since his brother-in-law was the expert, and seemed to actually be willing to help, he forwarded his chart over to him with the subject heading, “What does this do to our probability distribution?”.

Around noon, he got a message saying, “Call me. I am free until 2:00.”

When they finally connected, Bill’s brother-in-law told him to check his email for an updated chart.

“To come up with an updated forecast distribution based on the information that you sent over, I used a technique called Bayesian Inference². You do not have to know much about it other than it is a really well established method for combining an educated guess with actual data (or vice versa). I wanted to talk to you while we were looking at it together because there is a lot of room for error in the analysis. For one thing, even though we have some data, it is still a very small amount which means the projection will be very sensitive to small changes – up or down. So, I cannot tell you how accurate the 36% probability is. Probably not very. BUT, even if it is off by a magnitude of 2, it would still be well below the 90% threshold. In other words, given what we have, it would be really hard to claim that you are confident enough to sign that lease. Right?”

Bill replied “Right. I get it, and it is nice to have a clear decision. I do really appreciate the help. But, still, 36% confidence! Even if that is not completely accurate, it is pretty low. So, I have to figure out what to tell my board member about my forecast.”

Bill’s brother-in-law sighed. “Tell her that your forecast is still your forecast and that it is the best that anyone will be able to produce. I will say it one more time. All that we have done is to answer the binary question ‘At the current time, and given the available information, am I confident enough in my forecast to pull the trigger on this lease?’ Yes or no? That’s it.”

Thinking about the exercise in those terms was a huge relief to Bill; and to his mentor who agreed that waiting was the right course of action. She had been skeptical of the whole idea from the beginning but, like Bill, did not want to pass up an opportunity to nail down the space if he was actually going to need it. So, having some definable method and data behind the decision was immensely helpful. Between the two of them, they decided that, if he really had to, Bill could probably squeeze enough additional capacity out of his current facility to make it several more months. Hopefully, that would give him the time to find another facility or come up with more alternatives.

There are two points to Bill’s story. The first is that figuring out what question needs to be answered is the most critical and often the most difficult part of a decision process. The second is that there is always some data that will help answer the question. And, some data, no matter what the quality, is always better than no data.

The end.