Why are CEOs failing software engineers?

https://iism.org/article/why-are-ceos-failing-software-engineers-56

The one thing that’s drilled into your head as a business major and leader is, “here is how you succeed.” Accordingly, you are taught best practices – things you do to optimize success. As you would expect, you are taught about past successful companies, “here’s how they succeeded.” They teach you about great business leaders, people who have succeeded. You are taught how to do business planning in order to optimize your chances of success, and they teach you how to finance your attempt at success. In effect, signing up to be a business or finance major is an act of self-determined success, and the entire system is a culture that naturally values, and in effect worships, success.

Here is the rub: new value is a function of failure, not success; and, much of software engineering is about discovering new value. So, in effect, nearly everything you are taught as a business major or leader is seemingly incompatible with software engineering. Allow me the time to explain how this is so, why it matters, and what we should do about this situation. I promise you, in its entirety, the following is something you’ve never heard before, and the ramifications of ignoring the advice herein most assuredly leads to unhappy software engineers. As an industry, we've been perpetrating the same management mistakes on developers for several decades now. It's either time to fix this problem or be disrupted by those that already have.

The shortest of management histories as to how we got here, in 3 parts

Traditional management often fails software engineers for honest reasons that have to do with the essence and root purpose for the existence of management, and certain gaps which are yet to be filled by management theory, educational systems, and practices. To understand these gaps which manifest as systemic failure that negatively impact software engineers and other creative types, we must first understand the problems that our current management systems were created to solve.

There are three dominant forms of management, two are well known, but the third is barely known or understood at all, and all three are critical to the realization of value. In historical order by which they were discovered is Financial Management, Business Management, and Creative Management. The next three sections are a short summary of the three dominant management systems, the problems they overcome, and the type of value for which they were invented to govern. To best understand why CEOs are failing software engineers, the essence and properties of these three forms of management must be clearly understood.

Part 1, Financial Management (aka. Liquidation Management to govern Liquid Value)

The problem being solved in 1602 was the vast need for cash to build more ships. Namely, how was The Dutch East India mercantile company going to fund the effort to meet the high growth and high demand for Asian silks, spices, coffee, tea and wine. The answer came in the form of modern capitalism, granting shares in the enterprise to the public in exchange for cash to build more ships and hire more crew. It worked beautifully, the funds began to flow in from the newly formed public market, and the Dutch East India Company quickly became a powerhouse in global trade.

This was the nexus moment of the birthing of the financial industry and financial management. But our take-away from this historic moment is as follows: think of public markets as a management system for transmuting value; ergo, financial management was formed in order to optimize the transformation of one form of value (coin/gold/cash) into another form of value (tangible and intangible assets: ships, companies, recipes and patents) that have a higher return or fulfill a deeper purpose, and when desired, to liquidate said investments back in to cash to invest in the next highly valued venture. The purpose of Financial Management is to manage liquidity, and its practices are to conserve, develop and exploit cash, usually in the form of investments or loans. Therefore, this type of management is better thought of as Liquidation Management as it primarily manages the liquid form of value that we typically refer to as cash.

Liquidation management is naturally return-on-investment (ROI) oriented. So, the most natural way to motivate effectiveness in this type of work is to give financial team members a small stake in any higher than expected returns achieved, and a small piece of cash efficiencies achieved. This is usually done by establishing reasonable financial targets with corresponding bonuses.

Part 2, Business Management (aka. Replication Management to govern Replicated Value)

In the 1890s, as a result of the electrification of cities and an abundance of cheaply manufactured steel from pig iron, the world was in the midst of the second industrial revolution. Rapid growth was being fueled by sprawling railways, a swiftly expanding electrical grid, and an abundance of factories powered by an array of newly created, electrically powered equipment.

The over-arching problem being solved by Fredric Taylor and Henry Gantt was the absence of an effective and reliable system to manage industrial work, particularly manufacturing. The problem was three-fold:

Incentives were wrong – workers got paid a small flat fee for the day without respect to productivity, even when the equipment wasn’t working which was occasionally viewed as a feature by the crew
Work practices were wrong – each man did his job as he felt best without respect to the sustainability, effectiveness and needfulness of motion which often resulted in “slack,” down time when either the equipment or labors could not work
Management was clueless – management practices tended to be governed by friends of the elite who never learned to do the actual work, pushed what work they had downward, and managed badly from abstraction

The solution came in the form of Fredric Taylor’s “Scientific Management.” A system of management which leverages time and motion studies to establish sustainable, documented best practices for all workstations in the factory. Given established best practices that were timed by Taylor using a stopwatch, reasonable production quotas could then be enacted, which if achieved, as Gantt proposed, should result in bonuses for both management and workers. As a side benefit, factory output and time-to-delivery of an order could be calculated with a high degree of precision. Thus, given established times for known work practices, uniform materials and known recipes/designs, reliable master planning was now possible.

The combination of establishing sustainable best practices that are easily trained (Taylor), and aligning incentives based on known, reasonable, production goals (Gantt) worked astoundingly well. Before and after results documented by Gantt in his book, “Work, Wages and Profits,” showed as much as a 300% improvement in factory productivity by using Taylor’s system of management.

As it turns out, adoption of a scalable management system which has actual improvements in results is a big deal.

The system worked so well, that Gantt was solicited by the military to help with ship and munitions production. The corresponding increase in ship production and factory output proved to be beneficial to the United States as it entered World War I, and arguably, subsequent improvements in production over the following 21 years were, in large part, responsible for winning World War II. As it turns out, adoption of a scalable management system which has actual improvements in results is a big deal.

This was the nexus moment for modern business administration and the birthing of management science. But our takeaway from this historic moment is as follows: scientific management is a system that takes known recipes and practices and transmutes them into valuable (desired/needed) replicated goods; ergo, business management was formed to effectively replicate known value, and known recipes, using best practices, elimination of waste, and high-quality predictive planning.

As replication work is naturally date driven according to expectations set with the customer during the sales process, the best way to motivate staff to do this repetitive, labor intensive work, is to provide them a bonus for achieving or surpassing reasonable production goals that were set to meet the demand, and to align management incentives to the production goals such that management is incentivized to help the staff achieve their time-boxed goals.

“Engineering management is largely taught as a specialized form of business management, whereby much of the same thinking used to optimize manufacturing, unfortunately, gets applied to creative pursuits such as software engineering and product design.”

Part 3, Creative Management (aka. Creative Management to govern discovery of New Value)

As liquidation management (finance) emerged out of the need to fund new ventures, and replication management (business management) was formed out of the need to scale production of replicated value, similarly, creative management has naturally emerged out of the need to scale the discovery of new value.

What is Creative Management? Creative management is one of three dominant forms of value management. Its purpose is to establish a system of management that enables and motivates creative staff, such as software engineers, writers, designers and artists, to discover and realize new works of value. Creative management is also the lead system in new ventures and productions, having been established to optimize the discovery and realization of previously undiscovered value in the form of new works, new productions and new solutions.

However, we have not yet had a nexus moment for creative management, though elements of successful creative management systems are beginning to emerge as modeled and practiced in several recent tech companies. Traditionally, engineering management has been the concrete expression, or proxy, for creative management applied to technical endeavors. However, engineering management is largely taught and practiced as a specialized form of business management, whereby much of the same thinking used to optimize manufacturing, unfortunately, gets applied to creative pursuits such as software engineering and product design. This approach to engineering management is the natural extension to Taylorism; but applying replication management techniques to creative work is a systemic mistake that actually disadvantages companies that are developing software – this is because the skills, workflows and founding principles for discovering new value are almost entirely different from those needed to efficiently replicate value. Whereas replication management is focused on transmuting recipes and ingredients (discovered value) into replicated goods (replicated value) by a committed date, creative management is passionately attempting to discover new recipes, to meet unmet needs, through a series of value attempts, most of which fail. Therefore, the purpose of a creative management system is to find previously unknown recipes (unknown value) and transmute it into discovered value – new recipes and new ingredients.

Since at present, we have no formalized understanding and teaching regarding the role of creative management, we must look to empowered creative leaders of dominant companies to understand the practices and policies that best fill the creative management void. It is these types of leaders that have the power to setup supporting management systems that best enable creative staff, such as software engineers, to do the work of discovering new value. In alignment with management systems theory which postulates that management most know how to do the work, virtually all of the leading technical companies that are creating new value and transforming the world are led by people who have professionally done the work of creating new things for several years. It’s also interesting to note that many of these leaders are software engineers. Though this is only correlation, not causation, it does strongly imply that the skill set needed to create new value (new recipes, new designs, and new solutions) is significantly different than the skill set needed to replicate or liquidate existing value.

The problem being solved by empowered creative leaders, the true proxy for a creative management system, is how do you best motivate and enable your staff to rapidly discover new value? Their systemic solutions are too extensive to document here; but in brief, know that the systems these creative leaders have put in place proceed out of who they are as creative forces:

“Move fast and break things” – Zuckerberg
“Every day is day one” – Bezos
“If you're changing the world, you're working on important things. You're excited to get up in the morning.” – Page
“We do lots of Stuff. The only way you are going to have success is to have lots of failures first” – Brin
“When something is important enough, you do it even if the odds are not in your favor.” – Musk
“Success is a lousy teacher. It seduces smart people into thinking they can't lose.” – Gates

In summary, though system embodiments vary, the above quoted creative leaders share the following creative system principles:

You learn by failing, and new value comes from a critical mass of new learning
So, fail often, fail quickly, but fail smartly (failure without learning is waste)
Build platforms, practices and cultures that help people discover value
Relentlessly discover new value, even when the odds are against it
Empower your creative staff to transform the world by solving meaningful problems

Practically speaking, these creative principles take the form of equipping and empowering teams to make a series of rapidly evolving value attempts. These value attempts are passionately revised and improved upon until a candidate solution achieves market traction. Thereafter, additional features may be added to the solution to facilitate deeper market penetration. The work of the creative team is deemed to be successful when the end user needs are reliably satisfied by the new solution.

In order to maintain and foster creative motivation throughout the creative process, many of these leaders employ some form of shared conception, a desire to pursue a common compelling need, which forms a strong shared attraction. There are four elements to these strong attractors:

Need – a thing that needs to be created, something that’s missing from life
Belief – belief it can be created, that the idea is possible, even viable
Opportunity – a career opportunity to create that thing, a chance to do it
Anticipation – shared anticipation of the better, envisioned future

These elements form a strong attractor and shared conception is the practice of sharing and iterating the same strong attractors with investors, customers, creative staff and management, thereby aligning customers, staff and management on a set of highly anticipated emerging solutions. Not surprisingly, this form of creative motivation is often present in Silicon Valley startups, including many of the dominant companies established by the leaders quoted above.

How do you best motivate software engineers? You attract them to significant, life changing work by elevating the need, the thing that's missing from life or society, a problem that needs to be solved, and extending a personal, creative opportunity to solve that problem. Software Engineers are motivated to solve epic problems with solutions that people love. Continuously communicating to them the "why?" is critically important. Knowing why enables software engineers to evolve technical solutions that best fit customer needs.

The best way to motivate creative staff is to attract them to meaningful work through the use of strong attractors. Sure, you must also pay creative staff well enough to meet their standard of living; but, as Maslow points out in his theory of human motivation, beyond physical and safety needs, people have a need for esteem, recognition, and actualization. Just having a real possibility of meaningfully fulfilling these higher order needs is enough incentive to strongly motivate the heart and minds of most creative workers, even in the presence of pressure and adversity which often accompanies the solving of difficult problems through intense periods of failure. All greater transformational works of value are, necessarily, sustained by this type of motivation.

It is important to understand, when motivating software engineers and creative staff via the use of strong attractors, that all four elements must be present. In other words, the staff must believe in your vision, and the envisioned solution or outcome must actually be viable. There necessarily must be a real opportunity to actualize the envisioned product or service, and the staff must catch the envisioned outcome such that they share your anticipation of that better future. When any of the four elements of a strong attractor are missing, the staff’s motivation will be crippled, resulting in under tones of creative drifting, frustration, apathy and possibly even despair. Therefore, it is super critical to have viable strong attractors that are in pursuit of a real opportunity and genuine needs.

Three Types of Management for Three Types of Value

Our takeaway from the above history is that there have been three dominant forms of management. Each naturally proceeding out of the type of value they manage, and each having their own optimal practices and associated means of motivation.

In order to avoid making the same systemic management mistakes, I have become convinced that it’s super important to understand our history (how we got here), namely that value is an organizational force that resulted in the creation of our existing management systems, and yet we must learn from our emerging future which is driven by the discovery of new value (where we are going), lest we are doomed to amble around in our ignorance. Before proceeding to advice regarding what CEOs can do differently to better support their software teams, let’s be certain we are on the same page regarding our understanding of value and where we are going. For the sake of brevity, allow me to just blurt out the obvious observations:

Firstly, for our purposes here, let’s agree that: value is the reliable fulfillment of unmet needs without mitigating-levels of chaos
Let’s also agree that dominant management systems are a derivative of the nature of value
That value comes in four forms: undiscovered, discovered, replicated and liquidated
That value (the fulfillment of unmet needs) is achieved by transmuting value through each of the four states using the three dominant forms of management systems
Each value transformation system has an input value type and an output value type
That these value transformation systems have their own unique forms of work and motivation
That the 3 value systems form 1 value fulfillment cycle (solution fulfillment, order fulfillment, cash fulfillment)

It is this big picture, which as management, we are working to refine, filling in the gaps, and perfecting its practices. Namely: value is the net result of a value fulfillment cycle comprised of:

3 types of management systems
working together to transmute 4 types of value
by discovering products and business solutions
that reliably meets customer’s needs
without chaos

The tip of creative management is learning to think of the value fulfillment cycle functionally as a high velocity change engine that takes value attempts often in the form of minimal viable products and design studies, having some associated customer development (see Steve Blank, customer development) effort running in parallel with the engineering/solution effort.

What is the value fulfillment cycle? The value fulfillment cycle is the process by which unknown value is discovered, replicated, and monetized. This value-oriented process is at the core of all businesses and consists of connected workflows that serve three separate functions: solution fulfillment, order fulfillment, and cash fulfillment. The overarching cycle is overseen and governed by three dominant management systems: creative management, replication management, and liquidation management – traditionally considered to be creative/engineering, business, and financial management. Each of the three value management systems have differing practices and a differing means of motivation that directly tie back to the type of value they are transmuting; yet all three must work together in order to effectively realize value.

This sort of value-oriented thinking is driving creative management to actively automate and commoditize the value fulfillment cycle. In so doing, they are breaking down classically siloed constraints that slow down the discovery of value. Why? Because these silos, often the result of department-oriented thinking, have the effect of slowing down realization of value by serializing the discovery of value, discovery of chaos and discovery of customers. So, to break down these silos, in part, many of the mentioned creative leaders are empowering small teams of creative staff to swing for the fences (doing the full value cycle) on big, important problems, and they are having a great deal of success. Because, as it turns out, having an over-arching management system that reduces the time it takes to discover and actualize value is a big deal.

How are CEOs failing software engineers?

First, many companies are lacking an effective creative management system, and from the above history lesson, we can correctly infer that healthy companies are always a combination of the three dominant types of management. For example: every exceptional restaurant has an executive chef to do the creative work; hosts, waiters, porters and line chefs to do the replication work; and, an accountant to effectively cashflow the enterprise. To stay healthy, all three types of management must be empowered to hold each other accountable to the pursuit of satisfying the needs of the customer.

We also know that companies that do not have an effective creative system either die over time, or they are tied to another entity that is doing the creative work. Sure, there are some exceptions; but in general, if you are lacking a thriving creative management system, you are definitely impeding the discovery of new value, and you become a prime target for what Clayton Christensen termed as “disruptive innovation.” This is essentially the process whereby market leading companies are gutted by smaller entities that are discovering new value at much higher rates than traditional companies. Stop and consider for a moment, “what happens when larger companies with more funding figure out how to stay disruptive by using creative management systems that empower lots of small teams?” Of course, there are several tech companies that are doing this now, and they are disrupting traditional businesses at an exponentially, increasing rate. This is due, in no small part, to their successful creative management systems.

Considering the above history again, it should be clear that not having an effective creative management system is a huge gap in the value fulfillment cycle. If this gap remains unaddressed, the market will solve the problem by investors, over time, moving their money to entities that are discovering new value at higher rates. So, if you have this kind of systemic void, it’s really in your best interest to address the gap as soon as possible, and I would encourage you to give some serious consideration to empowering an effective creative management system.

Also, as it turns out, the three things that software engineers overwhelmingly want from management, when asked, is for management to:

Provide a clear sense of purpose, vision and mission
Invest in my growth by having opportunities for advancement
Grant autonomy and delegate authority

In other words, software engineers want to function in small empowered teams that are hotly pursuing meaningful value which could potentially transform their community, the company, and their lives into some epic, next level thing. When you think about their response, it kind of sounds like the basis of an effective creative management system that pursues new value. That’s likely because many software engineers are, at their very core, creative staff that yearn to pursue new value and, given their answers, they are in effect begging management to provide the big-picture charters under which they can actualize their creative force. My advice to you is to let them do it. But it is critical that these teams have a clear mission charter and budget, and that they also have the supporting roles needed to succeed embedded in the team itself.

Second, new value (fulfilling previously unmet needs) is the by-product of applied new learning that is gleaned from an often-painful series of individual- and group level- failed value attempts. To put some meat on better understanding this point, new products that truly address new points of need (new value) aren’t a function of time and money; instead, they are the result of learning from hundreds of small failures that must necessarily occur in the pursuit of discovering a new solution (new value). When Mark Zuckerberg created Facebook’s mantra, “move fast and break things,” he was simply giving his team permission to learn quickly by unleashing tens of thousands of value attempts. In point of fact, most smaller products are the result of hundreds of failed individual value attempts and dozens of failed group level value attempts, and large products and services are the result of tens of thousands of failed individual value attempts, and hundreds of failed group level value attempts.

Given this understanding, I have five points of advice for CEOs of companies that do software engineering:

Since learning from failed value attempts is the only path forward to discovering new value, our goal must be to get the cost of failure down (not the number of failures), and the rate of learning up. One of the easiest way to achieve this goal is to get the size of the teams down by investing in individual and small team level value attempts.
The next thing to do is make sure your teams have the research, promotion, and analysis skills that are needed to rapidly discover actual need, and to rapidly test and learn from their solution attempts. Lacking these skills, your teams may never discover industry changing value as their rate of learning will simply be too slow to compete.
Next, invest in platforms and tools that speed your team’s ability to make value attempts. Why? Because, often a platform can help with customer discovery and doing solution promotion, as well as gathering the metrics needed for learning and analysis, all of which can amazingly increase the speed at which your teams discover value. Since you are going to be making a lot of low-cost value attempts, you really should invest in automation that speeds value discovery.
When discovering new value, drive to value; when scaling discovered value, drive to date. Meaning: don’t gate value discovery by setting arbitrary release dates. In effect, a release date is nothing more than saying to the team, “we are going to give up on discovery of the actual needed value by <insert arbitrary date here>.” Of course, that’s nonsense. Also, dates don’t increase the rate of learning from failure; they in fact have the opposite effect, the team will be much less likely to discover the value needed as they will actually increasingly fear failure as the date approaches, and they will therefore slow down the rate at which they are breaking things, which effectively puts an end to new learning – no more bugs found, no more value found. If the customers need hasn’t been met yet, it’s just a huge waste and confusing to all involved to release something that doesn’t actually meet the need. It’s even worse to release something by an arbitrary date that has the appearance of meeting the need but is riddled with business risking- or life risking- levels of chaos.
When motivating the team, the correct motivation pattern to apply to the discovery of new value is shared conception using strong attractors. Go back and reread the creative management section regarding motivation, it’s worth a second look, if not a third.

Here’s the most important bit of advice to CEOs and leaders of software teams

Some people think, in order to best foster a creative learning culture, you should celebrate failure; personally, that’s a bridge to far for me. But, at a minimum, you must at least endure failure, and if it’s possible for you, full-on embrace the failure that’s always associated with discovering life changing value.

I promise (take me up on this), that if you set out to fail every day by taking truly new and different value attempts from which you learn, at the end of 90 days, you will have a small pile of new value. It’s just impossible to fail that much, unless you’re not learning, and not discover some truly new bits of value. Even better, if you and your favorite team of 50 engineers do the same exercise of taking value attempts for 90 days, and you aggregate those bits of value together along the way, you quite often have the basis for an industry leading product or service that you might even spend the rest of your career improving and delivering, and the world would be a better place for you having done that.

Personally, I am still shocked by the number of times I consider my failed value attempts to be waste. I am also appalled by the number of times that I deliberately avoid starting from a blank page; instead, I almost always reach for the safety-blanket of Google in hopes of finding someone else’s work to leverage – an existing known, good, starting place. It’s as if, deep down inside, I am ardently, even violently, trying to avoid the failure that comes hand-in-hand with starting from a blank page. Perhaps you can relate to the thought, “why waste time starting from a blank page when you can be way more successful starting from some known good place?” The truth is, “new” necessarily means you better not be able to google your idea, because if you can, it isn’t new, right? Yet, in place of embracing failure by taking new value attempts, I find myself starting out each new day doing something that I already know, a thing that I'm already good at, because that makes me feel valuable.

This is best referred to as failure avoidance syndrome, and I promise that, as CEOs or leaders, we have a bad and almost creatively lethal dose of it.

What is failure avoidance syndrome? Failure avoidance syndrome is a common and even normal sociophysiological condition that causes us to associate productivity and value to the completion of known tasks, while avoiding attempts that will likely lead to failure. Failure avoidance is, straight-up, a combination of success conditioning and brain chemistry, and it leads to us feeling that starting from a known good place is more valuable. This phenomenon is due, in no small part, to how our brains, in response to the thrill of successfully completing a rewarded task, releases euphoric chemicals that strongly reinforce the behavior that led to success. Conversely, the disappointment and menace often associated with failure results in our brains releasing a different set of chemicals that act to dissuade us from making additional mistakes (even, and especially, new value attempts) that might threaten our safety, social acceptance, and well-being.

The net effect of our success-based conditioning and our natural brain chemistry is failure avoidance. In other words, discovery of new value is actually safe guarded by a sociophysiological curse that most commonly manifests as a natural, strong aversion to failure. However, when doing creative work, we must find positive ways of pushing past the chemically reinforced sense of waste and failure avoidance.

This necessarily means that the journey of discovering life changing value requires us to endure large amounts of failure. For me personally, I’m learning to embrace the failure of taking daily value attempts versus comfortably filling my days with planned success; it’s a big step, but one that I’m committed to doing.

In summary, for better value discovery results:

Empower individuals to take daily value attempts
Use light weight value attempts and amplified learning
Drive to value, not to date
Seed your creative teams with research, promotion, and analysis skills
Avoid forcing individuals into the same ways of thinking about a problem
Encourage your creative staff to move fast and break things
Embrace failure, celebrate learning
Personally, fail more often. If you’re not failing, I promise you, you aren’t learning, and you’re therefore likely not leading your industry either, seriously.

Regarding the institutions who are setting up management to fail software engineers

Listen up Ivy League educators and macro economists. Take note of the three (not two) types of dominant management systems. Whereas you can align the motivation of workers and management within the same type of work, aligning motivations across types of work (such as replication to creative) that are dissimilar in nature using the same incentives isn't a thing. It doesn't work. It can't work. It's confusing, frustrating, and demotivating when a system of management is ignorantly applied to a type of work that it cannot possibly, effectively govern. Yet, as a ‘modern’ society, we've created a prevailing global management system that perpetrates this offense repeatedly. Just stop it. Seriously, it’s ineffective, it’s upsetting to your developers and creative staff, and in the long run, it’s in no one’s best interest.

While you’re at it, go back to your management textbooks and curriculum and write in Creative Management, and go back to your economic models and factor discovery of new value into your theories of economic growth. Also, many of you need to go back to your companies and apologize to the developers and creative staff to whom you aren’t listening.

In closing, if you are genuinely interested in speeding the realization of value, spend some more time studying value fulfillment theory: the cycle by which unknown value gets discovered, replicated, and transmuted into cash. And then, consider reorganizing your company to unleash the creative force of your software engineers.

Do you want to publish our articles on your site? Our nonprofit mission is to develop the careers of software leaders by sharing as much software management theory, research and knowledge as humanly possible. To that end, I'd like to take a moment to introduce iiSM.ORG's very liberal media policy. In short, all of our material and articles can be replicated in part or in whole on blogs and media outlets with a simple attribution back to the source. We especially appreciate those of you that translate iiSM.ORG material and articles to other languages!

Are you a technical leader? Consider investing in your career growth by becoming a member at iism.org/register.