Feature Article: Towards an Integral Leadership Vision in Software Development

Feature Articles / March 2009

keesThis article, primarily using the stages of development component of Ken Wilber’s Integral framework, follows the emerging group dynamics models that are coming apparent in current post-postmodern environments and how one such model was used at Integral Life in the IT Development efforts.

Introduction

Software development is one of the most challenging and evolving areas of business today. The development of solutions stepping from vision to requirements, analysis, design, development and finally rollout coupled with technology advances and people issues makes it one of the most complex endeavors to undertake and succeed at. As we move into the Internet age we find both the speed of social and technological change is accelerating, as is the demand for new software and systems. In this environment certain organizations are adapting to the environment and thriving, while others are not and are failing.

So often in analyzing the critical success factors of business and organizations we focus almost exclusively on the individual leader. Leadership qualities, managing teams and the mastering of leadership skills consume miles of library shelf space. These are hierarchical models where all empowerment and responsibility lies with the leader. Frameworks are looked at as leadership frameworks where the individual leader directs, motivates, cajoles and otherwise gets his team to perform. We have now seen in recent times other frameworks appear with which organizations are achieving superior successes in today’s business environment. This paper looks at the trend or organizations moving towards the perspective of the team framework resulting in empowering of leadership, natural hierarchies and more organic processes.

As the center of gravity of human consciousness of the individuals within the groups being managed moves towards a more world centric post postmodern perspective, there is the ever-growing shift from traditional hierarchical leadership approaches to what Don Beck and Christopher Cowan called in Spiral Dynamics: Mastering the Values, Leadership and Change the holistic leadership approaches.

Does this imply that the groups are leaderless? No, but what it does start to demonstrate is that the success of the group can be channeled from the group in a positive and productive way. This is then more a look at the frameworks or meta models that empower the group and how these empower leadership, rather then the other way around. This is not to imply that one quadrant of development of the AQAL Integral framework (such as LR) is more important then the others or that leadership emanates from one quadrant over the others. What it asks is that all members of the group work from all four quadrants to be successful in the new leadership paradigm. It is also important to keep the concept of the pre/post fallacy in mind while discussing the team dynamics. While group dynamics of earlier times might have outward seeming similarities to today’s groups and organizations these groups are substantially different. Small clans working together organized based on power hierarchies that differ dramatically from natural hierarchies

The Maps of the Interiors

Our ability to understand the territories being explored is in relying on maps that can give us an orientation of the complex interrelationships we are discussing. Two components of the map that we will rely on are the stages of development and the quadrants components of the AQAL Integral framework (also known as Integral Methodological Pluralism (IMP)). The AQAL framework is a map of the essential territories that members of groups and organizations face. Looking beyond simple strategies of motivation or culture, these maps look from the perspectives of self, other and organization.

In “stages of development” each stage represents a level of organization or a level of complexity. The stages are not meant in a rigid or exclusionary fashion, but simply to indicate that there are important emergent qualities that tend to come into being in a discrete or quantum-like fashion, and these developmental jumps or levels are important aspects of many natural phenomena.

pic1

The quadrants provide leaders with a map to navigating the realms of psychology, behavior, culture and society necessary to be able to enable such dramatic shifts in organizational structure.

pic2

We refer to any event as a Holon – or a whole that is part of other wholes. A holon is something that is simultaneously a whole and a part. A social Holon possesses a definable “we-ness”, as it is a collective made up of individual holons. This we-ness emanates based on the dominant mode of discourse of the holons or members of the group.

100 years of the Pluralistic Stage of Development

As late as the ‘70’s and ‘80’s, as described in The Leader by Michael Maccoby, leaders were considered responsible for “forming winning teams at all costs”. Maccoby estimates that this pluralistic period peaked in US history during the Roosevelt era. In this pluralistic AQAL stage of development Maccoby saw a social character more oriented to self than to craft, enterprise or career. Maccoby identified the three negative traits in the workforce: an other-directed marketing orientation, alienation, detachment and disloyalty, where people tend to trade integrity for status; undisciplined self-indulgence and an escapist consumer attitude, fantasy and compulsive entertainment which one rationalizes as self-fulfillment; cynical rebelliousness, an attitude of getting as much as one can by giving as little as possible, rationalized in terms of rights and entitlements. To counter this Maccoby recommended a leadership style expressing an ethic of self-development and appealing to the positive traits of pluralism. He looked to adopting a management style that appeals to the positive traits of what Maccoby calls “The Self-Oriented Character”: flexible, experimental, tolerant, self-developing, playful, fair and participative. While it may be different from autocratic, bureaucratic or gamesman-leader styles, pluralistic leadership is still a very hierarchical model of the leader effecting actions on a group.

A New Center of Gravity

In the last 30 years we are starting to see a shift in the center of gravity of the stage of development of members in groups in the workplace to a more integral level. The integral stage is typically characterized by certain traits such as systemic thinking, integrative processes and interactive structures3. Systemic Thinking can be seen as the combination of analytical thinking and synthetic thinking. The key to synthetic thinking—and consequently to systemic thinking—is the Fractal Phenomenon: systems are made up of repeating patterns. The basic idea of systemic thinking is to list as many different elements as you can think of, then look for similarities between them. In conventional analytical thinking one would list a handful of elements, compare them, rank them and select the best one, discarding the rest. Analytical thinking results in losing benefits of the discarded options that the selected option does not have. Analysis is about identifying differences; synthesis is about finding similarities. Looking at the world from a synthetic point of view then results in integrative processes of large structure sets and their interactivity.

Peter Senge in The Fifth Discipline wrote of a framework based primarily on systems thinking and to be used by leaders to develop a learning organization. Senge articulates how systems dynamics underlie the organizational behavior and how an integrated approach to leadership is critical. It is critical in focusing of the group’s responsibility on achieving leadership and seeing how the group achieves its goals. In The Fifth Discipline’s shared vision and common direction are important, but the book gives no structure or direction on how to attain those. There is little actionable structure that a leader or group can employ to effect the recommendations. In fact while the book focuses on the shared vision, it does so from the perspective of the leader’s view rather then from a group perspective.
Integral Stage Organizational Tools

In his book Complexity Mitchell Waldrop states that the study of complex adaptive systems is clearly critical to avoid the potential catastrophes that face us in the future. The goal is to get there, to a place worth living in or in our case a place worth working at. Complex adaptive systems (CAS) or complexity science is not a single theory but rather encompasses multiple theoretical frameworks and is highly interdisciplinary, seeking the answers to some fundamental questions about living, adaptable, changeable systems.

Examples of complex adaptive systems include the stock market, social insect and ant colonies and the Internet, especially in the social group-based endeavors of on-line communities and social networks. CAS ideas and models are essentially evolutionary, grounded in biological views on adaptation and evolution. CAS bridges development of systems theory with the ideas of generalized Darwinism or, as we stated before, the combination of analytical and synthetic thinking.

CAS organizational models have common attributes that they exhibit. These attributes include self-organization, participatory organization and natural hierarchies.

Self-organization is basically a process of evolution where the effect of the environment is minimal, i.e., where the development of new, complex structures takes place primarily in and through the system itself. Self-organization is usually associated with more complex, non-linear phenomena, rather than with the relatively simple processes of structure maintenance of diffusion. All the intricacies (limit cycles, chaos, sensitivity to initial conditions, dissipative structuration,…) associated with non-linearity can simply be understood through the interplay of positive and negative feedback cycles: some variations tend to reinforce themselves others tend to reduce themselves. Both types of feedback fuel natural selection: positive feedback because it increases the number of configurations (up to the point where resources become insufficient), negative feedback because it stabilizes configurations. Either of them provides the configuration with a selective advantage over competing configurations. The interaction between them (variations can be reinforced in some directions while being reduced in others) may create intricate and unpredictable patterns (chaos), which can develop very quickly until they reach a stable configuration (attractor).

Participatory organization is an organization that is built based on people participation rather than the individuals contract or pre-determined obligations.

Most organizations previous to this [to what?] were contract-based. Contracts define a functional structure that holds such an organization together by imposing mutual obligations on people. For example, an employee of a typical organization is obliged to perform certain functions in exchange for some previously agreed compensation. Once established, such a contract relationship is quite rigid and inflexible. A breach of contract implies severe penalties in most cases. Contracts facilitate organizational planning and often shift risks from one party to another. Contracts are necessary for existence of fixed and rigid organizational structures primarily because these structures cannot easily accommodate changes: a failure of one element can easily become a cause of the failure of the whole organization. On the other hand, the rigidity of contracts creates a major stress for the people involved, primarily, employees.

Participatory organization is an alternative to the contract model. In the absence of obligations, any participant is free to contribute or not to contribute. This requires flexibility and robustness from the organizational structure. This way it gives much flexibility to all people involved, while the organization still performs its function reliably.

A property of evolutionary participatory model is its ability to scale well with the number of participants. Unlike many other organizations, it becomes more efficient as more participants get involved in it. This is sometimes called the network effect, though this term can be sometimes misleading since networks is not the only way of structuring interactions in a participatory organization.

Natural Hierarchy. Also know as self-managing groups, they [What they?] are best seen as social holons. A social holon does not possess a dominant monad; it possesses only a definable “we-ness”, as it is a collective made up of individual holons [Is the collective made up of individual holons or are individual holons simply members of the collective and they are fundamentally two different typoes of holons?]. In addition, rather than possessing discrete agency, a social holon possesses what is defined as nexus agency. An illustration of nexus agency, as best described by Ken Wilber, as a flying flock of geese. Each goose is an individual holon and the flock makes up a social holon. Although the flock moves as one unit when flying, and it is “directed” by the choices of the lead goose, the flock itself is not mandated to follow that lead goose. Another way to consider this would be collective activity that has the potential for independent internal activity at any given moment.

Agile Software Development Management

Software development efforts can be seen as a tradeoff of three variables: time, money and scope. In managing through these the question always arises “where are we”. That question needs to be answered in terms of the three variables comparing actual to plan or expectation. For any given project some of these variables are fixed and others are not. Traditional project management is done using “waterfall” methods. Waterfall methods can best be pictured as Gantt charts where there are dependencies from task to task with a fixed beginning and end. This project management style works best where predictive methods can be used. Predictive or waterfall approaches have a very difficult time changing direction as they are optimized for a certain destination. At a high enough rate of change all forward motion stops.

Software development was always fraught with problems in controlling scope and strategies to keep projects predictable. With the onset of Internet systems development this became impossible as the “rate of change” spiked up. This impossibility was brought on by a combination of the large number of variables, the speed of the evolution on the Internets capabilities and recursive nature of learning what was endemic in the process. By the time someone wrote down all the requirements everything had changed and they were no longer relevant. The choice in systems development was to either start “cowboy coding” with no controls or develop new project leadership paradigms. In fact it could be paraphrased that the changes that occurred n project management was leaving a manager model to a leadership model.

On February 11-13, 2001, at The Lodge at Snowbird ski lodge, seventeen people met to talk, ski, relax, try to find common ground and, of course, to eat. What emerged was the Agile Software Development Manifesto. Representatives from Extreme Programming, SCRUM, DSDM, Adaptive Software Development, Crystal, Feature-Driven Development, Pragmatic Programming and others sympathetic to the need for an alternative to documentation driven, heavyweight software development processes established such a new paradigm1. This was a paradigm for things that are often not linear, not simple and not predictable.

Agile is significantly different than anything seen before because there is no project management schedule, only a committed delivery time for a release. Small teams are given objectives to meet in fixed short time frames. Project teams meet daily and all agree on steps completed and next steps. For large projects, small teams of this size build components and then individuals of groups of teams meet less frequently to work out interfaces between components. The code develops like a biological system via punctuated equilibrium2.

Management must empower and release control of the team and let it function as a self-organizing entity that grows a system like a plant. Estimating package (high-level requirements) completion time is done by determining the velocity of the team, similar to predicting the impact point of a rocket. The impact point can be adjusted by lowering the arc (less functionality) or adding rocket fuel (more resources).

For agile to work, it requires a shift in the organization and the way people work and relate to one another. Product owners become intimately involved in the requirements on a constant basis; management is give product demos instead of Gantt charts to show progress; and developers have a commitment to their team or social holon instead of to a schedule. Agile thus meets the goals of the integral stage system having self-organization, participatory organization and natural hierarchies.

Integral Life Portal

When Robb Smith, the CEO of Integral Life, brought me onboard it was with a simple charter, to build a social networking platform that would enable our development utilizing Web 2.0 technologies. What that might look like, sound like, function like and evolve to be at that point was an unknown. In the early stages there were stakes in the ground of dates, but those shifted as dialogues on the voicing, look and functionality of the portal evolved. This was very stressful from a project management perspective as the schedules never supported the evolutionary nature of our work. Finally after some months I did something that is rare in systems development, I changed the project management paradigm in the middle of the project. We adopted an agile methodology so that we could set and control expectations, measure our progress velocity and start to be able to predict how soon certain feature sets would be completed by and address the still constant evolution that was occurring in the project.

This adaptation of agile recognized the portal for being an organically evolving solution that had no definitive end and no fixed end solution. Instead it was more of a dialogue of what is possible and what is the best solution to attain the goals, which were themselves evolving. We were now able to have constructive dialogues on the play-off of scope changes or funding changes to address what might be available by any given date. A sense of order was placed on the project whereby management could prioritize the features with the highest return on investment and the development team could set expectations on realizing the goals.

Conclusion

The increase in complexity and chaos in software development can be seen as an example of what is occurring in all facets of modern culture and society. Software development has evolved to new paradigms of leadership and control to address the challenges.

For the past 100 years the solution to social and cultural ills has been to turn to leaders who would singularly rescue us. In the new order there are required new social holon paradigms that replace the singular leader. Natural hierarchies of individuals, the social holons, will need to develop the correct nexus agency in order to address these challenges in ways and with capabilities that old leadership models lacked. Only these highly effective and diverse groups will be able to keep pace going forward.

As everyone faces the daunting task of understanding and influencing their lives they have a choice as to what tools and techniques they use to understand them. The AQAL model provides a powerful map from which to analyze the terrain and society is organically creating structures and paradigms from which to view and influence this chaotic and fast paced world.

Reference List

  • Bartlett, Gary (2001). Systems Thinking: A simple thinking technique for gaining systemic focus. The International Conference on Thinking “Breakthroughs 2001”.
  • Beck, Don & Cowan, Christopher (1996). Spiral Dynamics: Mastering values, leadership and change. Malden, MA: Blackwell Publications.
  • Benkler, Yochai (2006), The Wealth of Networks: How social production transforms markets and freedom. New Haven, CO: Yale University Press.
  • Cohn, Mike (2006), Agile Estimating and Planning, Upper Saddle River, NJ, Prentice Hall
  • Heylighen, Francis, (1997). The science of self-organization and adaptivity. Brussels, Belgium: center “Leo Apostel, Free University of Brussels.
  • Knibert, Henrik (2007), Scrum and XP from the Trenches; How to do Scrum, USA, InfoQ
  • Maccoby, Michael (1981), The Leader: A new face for American management, New York, Simon and Schuster.
  • Poppendieck, Mary & Poppendieck, Tom (2003), Lean Software Development, An agile toolkit, Boston, Addison Wesley.
  • Senge, Peter (1990). The fifth discipline: the art and practice of the learning organization. New York: Currency Doubleday.
  • Sutherland, Jeff (2007). The First Scrum: is it Scrum or Lean? <http://jeffsutherland.com/scrum/2007/11/is-it-scrum-or-lean.html>
  • Waldrop, M. Mitchell. (1992). Complexity: The Emerging Science at the Edge of Order and Chaos. New York: Touchstone, Simon and Schuster.
  • Wilber, Ken (2005). “Introduction to Integral theory and practice: IOS basic and the AQAL map.” AQAL: Journal of Integral Theory and Practice 1(1): 1-36.
  • Wilber, Ken (2006). Integral Spirituality: A startling new role for religion in the modern and postmodern world. Boston: Shambhala.
  • Wilber, Ken (2007). The Integral Vision: A very short introduction to the revolutionary integral approach to life, god, the universe, and everything. Boston: Shambhala.
  1. The Agile Manifesto, accessed, October 2008 < http://agilemanifesto.org/history.html>
  2. The First Scrum, accessed October 2008 < http://jeffsutherland.com/scrum/2007/11/is-it-scrum-or-lean.html>
  3. Book Notes Toward Integral Leadership, accessed October 2008 http://www.peace.ca/spiraldynamics.pdf

Kees Steeneken is the founder of the Santa Monica Center for Integral Living, bringing Integral training to Southern California. In 2007 and 2008 he was the Chief Technology Officer for the Integral Institute and Integral Life. With over 20 years of technology management experience, Kees has previously been a technology executive with companies of various sizes from start-up to Fortune 40. He is a certified Project Management Professional and Certified Information Systems Auditor.

For more information, visit www.smintegral.org or mail him at kees@smintegral.org.