Agile Organizing: Theory and Practice of Self-Organizing in Bureaucracy

One of the three most important performance obstacles described by U.S. federal employees are “too many levels of management.”

If you were assigned the task of organizing, say 500 or 1000 persons, what principles of organization would you use?

Complex organization emerges when there are recurrent types of decisions requiring multiple scales of human action. These relationships require coordination and accountability embedded in role performances imbued with a larger sense of purpose or mission. These organizing processes depend upon ethical leadership in developing trust, mutual understanding, and efficient cooperation between a variety of vertical and horizontally contingent roles in order to produce desired tangible outputs.

Organizing is an active managerial function which employs function and form to reflect preferred values and priorities. Peter Drucker and Elliott Jaques’ leading principle for structuring organization is to look at the work priorities that need to be coordinated and completed, and then optimize trust, cooperation, and efficiency in structuring functional relationships to accomplish the work. Organizing is a participative designing process for the purpose of discovering what is more “bedrock” based on recurring needs for initiation and coordination of work at different levels of scale and what is “contingent” based on shifting needs of dynamic situations.

In a large organization, organizing involves contemplating the relevance of the mission, and then gaining consensus on (1) areas which require excellent performance or political visibility, (2) the grouping of activities together around functions of those areas of excellence, and (3) how the functions would be financed and deployed relative to geography and/or functional disciplines or units, and (4) how critical work flow would move horizontally through functions to the customer from input to output with feedback loops in the system.

How many vertical levels–roles reflecting distinct scales of action—are needed? Each vertical level of a work role in the chain of command should add value (with additional costs) depending upon the scope and scale of performance with requisite resource allocations desired by a governing board given their objectives and legislative requirements. Size and scale are important contextual problems of organization to be discovered. There are no a priori number of vertical levels which fit every organization.

Differentiating Levels of Task Awareness Corresponding to Different Levels of Scale

A basic aim of organizational design is to clearly differentiate and integrate roles that are performing work at different levels of scale.

There are criteria based on recurrent differences in individual task awareness which form the basis for differentiating vertical levels of work. These criteria reflect the human expression of different scales of action. Thomas Jordan (p.69) in his article “Skillful Engagement with Wicked Issues”, Integral Review, October 2011 gives examples of recurrent differences in individual task complexity awareness. When responding to a similar problem:

• An individual operating at an operator level looks for a concrete example or action to solve the problem.
• A first-line manager or professional would investigate or diagnose the underlying causes creating the problem or series of errors before deciding upon a discrete response to a particular situation.
• An operations manager would view the same problem from the standpoint of overall systems design and its forward extrapolation, focusing on efficiency and fairness.
• A general manager would view the problem as the “face” of the organization and how relevant external and internal stakeholders understand the problem and its saliency.
• As a managing director of a national company or a public sector department head of a city of one hundred thousand persons, one would approach the problem as a primary enterprise task of adapting the priorities, competencies, and culture (i.e. “the way we do business”) of the organization over time to an appreciation of changes in the environment.

And so on. Each of these modalities exemplify systematic qualitative differences in the way individuals scale, define, select out, and seek information (defining what dots to connect) in order to accomplish their goals. These task awareness levels are thus criteria for organizational choices of what levels of scaled vertical roles and information-seeking requirements are needed to engage tasks based on the interpretations and resource priorities of the governing board.

Thus,the critical task of organizing then is to disaggregate and rescale levels of vertical positions to correspond to aforementioned systematic differences in individual task awareness. Once having identified basic levels of vertical role accountability, one can bring in a panoply of horizontal coordinating, monitoring, and auditing functions as expert staff roles to assist with workload with varying authorities and mutual accountability relationships.

In addition, these recurring threshold levels of individual task awareness, in turn, provide a basic criterion of staffing. While this natural role hierarchy can be viewed as “bedrock”, each individual’s task awareness is progressively maturing, enhancing a person’s ability to navigate varying thresholds of uncertainty necessary to perform tasks within corresponding levels of roles. However, each person experiences at critical points of developmental maturity a qualitative “quantum leap” in their cognitive “affordance” to orient and navigate longer tasks in space and time. Developmental curves systematically vary among persons. When persons mature over these thresholds of task awareness, they characteristically begin to think and act different. They exercise a newly emerging interest to selectively perceive, value, and connect relevant data through a qualitatively new, extended scope of awareness and networking. They engage uncertainty qualitatively different in the scope and extension of the goals they set.

Maintaining a Requisite Structure of Organization

It is the manager’s manager or manager once removed who must actualize requisite staffing and organizing. Their first responsibility is to discover and secure the requisite number of “bedrock” vertical roles in relation to the recurrent scale of action needed. It is important to prevent political and career dynamics from adding additional, unnatural vertical work levels which could degrade trust and two way communication.

Secondly the manager once removed should monitor employee maturational development and promotions in order to both mentor and train as well as instantiate a more impartial performance orientation in the organization. This process can begin by appraising the individual task awareness of a candidate as the first basis for sifting promotional candidates. One can do an appreciation of whether a candidate is “big enough” or “can carry the can” for the job? Identifying the requisite level of task awareness (beyond seniority and experience) and corresponding ability to navigate uncertainty, selecting out and connecting key points of data, for usable information in decision spaces is a necessary but not sufficient condition for insuring probable success in the role.  Other attributes of selection are then also important.

As Peter Friesen indicates, if the manager once removed abdicates or ignores these responsibilities, the field becomes ripe for collusive loyalty relationships between managers and direct staff to predominate over performance relationships.

To recapitulate, it is the ongoing, natural maturational development in individual task awareness which create and sustain recurrent distinct levels of vertical hierarchies of roles who are ultimately accountable to the intentions of the governing body.  The clarity of articulating these levels of work enhance natural accountability and removes the vagueness of “flat” or “decentralized” descriptions of organization. For more information, there is a large body of design principles which have been applied and tested for over forty years in many organizations by the late Professor Elliott Jaques and colleagues.

Avoiding Reifying Natural Hierarchy in Agile, Functional Organization

Differentiating natural hierarchies accounts for recurring scale effects of decisions. However, these dynamics must be integrated into a larger context of creatively relating the parts of an organization and its environment with common enterprise goals.

Philosophically, man is always more creative than any prescribed roles and responses and by a static human algebra and geometry. Human beings must always assume a critical distance from institutions, while using them for their purposes.  This is the source of self-governance.

Mary Parker Follett writes that you cannot schematize men as space objects. Rather, man’s nature is to seek forever new relations through which his individuality is expressed. Thus, our mental models of organizing must change. While hierarchies reflect recurrent needs for scale of organization, from a perspective in real-time, there is no up or down, but simply multi-scalar effects of situations to be engaged. As Margaret Wheatley and others have pointed out, when these situations possess uncertainty requiring horizontal joint coordination, those persons demonstrating “big picture thinking” or grounded front-line knowledge can be brought into a conversation around self-organizing processes.  Accountability has to be linked to eliciting shared purpose even if work tasks are individual.  Thus management becomes a field of cooperative, dynamic relations around defining work interests and quality outputs.

Undue deference does not have to be applied to hierarchy. Plato points out that one individual may have an insight or interpretation which may be unique to all others. This creative insight may not be dependent upon their level of task awareness in space or their managerial maturation. The expression of wisdom and morality are not necessarily dependent upon the scope of individual affordances of cognitive extensions in space.

The Philosophy of Self-Governance: Recognizing Dynamic Interdependence in Bureaucratic Organizations

Mary Parker Follett indicated our philosophical conception of self is critical. Her ideal conception is that the self is by its inherent nature morally and functionally interdependent and interpenetrating with other selves; thus there is a continual impulse to express differences, and peacefully integrate conflict by cooperating toward realizing a dynamic approximation of the common good: identifying with and strengthening each and all wherever possible.

Practically, the cooperative challenge then is to harmonize creative thoughts for the common good of the enterprise. Follett writes, “This involves not the over-individual mind, but the inter-individual mind, an entirely different conception…The individual “can learn how to join his thought with that of others so that the issue shall be productive”. “Community as Process” American Philosophical Review 1919. As Follett writes in her article “Constructive Conflict”, citizens and managers alike need to learn how to fit themselves into an ongoing group process such that their deeply held values are integrated into ongoing deliberations. For example, Mary Parker Follett writes that the challenge is for the individual to not only think out conscious, principled ethics but test, integrate, and refine their application into real-time situational dynamics.

This is a challenge in self-governance which exists in a field of moral interdependence. All work involves uncertainty and judgment to realize a goal or intention aided by endorsement and coordination from others. From an individual perspective, self-governance involves traversing the shoals of uncertainty for task completion on a horizontal axis while on a vertical axis, ethically ordering one’s conceptualization of how one should anticipate and engage situations which can creatively and inclusively draw out the best thoughts of others in order to create new integrative ones. This thinking can include acknowledging apparent limits which can be viewed as a resource for rethinking the possibilities of the situation. In the worst case where integration and even compromise fails, a default position is that each person can give reasons if their position is not accepted, because, as Follett writes, by moving beyond “the myth of the final decision” (since decisions have unanticipated consequences downstream), organizational members with time can come back to “reasons” in fresh situations; thus escaping the psychological trap of defeatism due to the “tyranny of the immediate.”

From an organizational standpoint, individually and collectively, members must generate insight into what is required to pursue the common good of the enterprise in particular situations, and then create and alter role agreements and mutual expectations. Governance emphasizes what to do while self-governance expresses what people agree to endorse as the limits of what not to do. Self-governance enables a collectively endorsed autonomous space for free individual initiative mediated by the need to coordinate complimentary work. This results in dynamic role appreciations and negotiations. The results of ongoing self-governance may require redoing the formal or informal “constitutional orientation”, the meta-rules of role relationships in micro-teams and interest group power relationships in macro systems. Glacier Metal Company exemplified a continual discovering of semi-autonomous, self-governing, “Constitutional” relationships through over forty years of successful performance in England. Thus, constrained degrees of self-governance with mutual accountability can emerge even in bureaucracy with highly prescribed norms and operational requirements for efficiently producing goods and services.

Practical Implementation: Linking Scales of Action to Designing Domains of Multi-Dimensional Participation

Luc Hoebeke in his volume Making Work Systems Better captures the “spirit” of Mary Parker Follett’s work on participative and progressive integrations of conflict around scales of action. Hoebeke includes employees as contributors into a systematically designed planning meetings at threshold levels of shared orientations. These planning meetings, reflecting overlapping “linking pins” of vertical roles, occur where employees and managers jointly contribute to actual tasks and the goals of the enterprise in:

• Operational, value-output scale meetings
• Innovation and strategic implementation scale meetings
• Political value-domain setting scale meetings

Ethical, principled leaders must create a culture which values “contribution” as well as “compliance”. Productive meetings require that the manager create an atmosphere of trust that enable, wherever possible, the “knowledge workers” and even “customers”, the real owners of the system, to make contributions in the spirit of performance through fact-finding, feedback, and joint problem-solving.

Peter Drucker states the managerial jobs are autonomous. He states the essence of the role: “Managers are not helpers and their jobs are not delegated. Their jobs are autonomous and grounded in the needs of the enterprise.” P.238 Drucker Management 4^th edition. Luc Hoebeke explicitly describes these semi-autonomous managerial roles based on scales of human action. They become central points based upon unifying around particular, scaled situations. The possible scale of ramifications can determine the level and spheres of participative inclusion and self-organizing as to “who is involved”, “who controls”, and “who is accountable” based on joint fact seeking and disciplined feedback. This can be accomplished through “action problem-solving meetings” as well as monitoring and understanding “significant variances in the system”. Through these orientational meetings, rules can be continually adapted or other rules invoked so as to meet the needs of both precedent and dynamic future situations. Thus, one evolves a culture of “responsible autonomy” based on flexible, self-organizing task awareness and capabilities.  An agile organization creates a culture encouraging “mindful” leadership in self-organizing rather than habitual compliance.

For example, truly participative, open systems strategic planning processes gather intelligence and give voice to goals and methods beyond the structured hierarchy of roles. In one example, it was a janitor who proposed more room for cleaning materials to the supermarket chain so it could compete as the “cleanest stores.” In the spirit of continuous improvement, information technology and metrics provide feedback for self-organizing systems in public services. See Mark H. Moore Recognizing Public Value chapters 2 and 7 for examples in police and social services.

Thus, while it is important to differentiate a clear vertical accountability chain of individual work levels from the governing group at one level, it is equally important to embrace horizontal, agile, group integrative problem-solving/endorsing sessions with a member who can embody the level of uncertainty and/or grounded expertise for governing and adjusting different scales of action for diverse stakeholders and beneficiaries.

Summary and Broader Questions for Understanding Organization

Leaders of complex organizations coordinate human capabilities to produce useful outputs and societal contributions. The simple rule of thumb: “If you set up and staff basic vertical roles reflecting different scales of action and expertise you desire, you can organize any way you want.” Organizational design choices can add trust-inducing contexts which engender individual initiative and multilevel group problem-solving through flexible scaling.

This principle of organizing for different scales of action and expertise extends into challenging public sector environments where multi-sided arenas of negotiated accountabilities emerge in settings where the public manager must earn portions of their authority as they go. Organizations are always open systems, are “in-the-making”, relational, and partially integrative. Circular relations always outgrow the husk of organization charts while formal accountability relations are always a contingent variable in a larger circumference of creative action.

However, Peter Drucker asks the ultimate design question: If we were going to start over again, would the public good be better served with the services you are currently providing? If so, how would you organize from scratch to provide them better? Donald Schon wrote that organizations are monuments to old problems. Thus, Drucker encourages us to examine the principle of organizational abandonment. Robert Biller wrote that in changing environments, organizations come and go but individuals persist. All rules are good for the average, but most people and situations are not average. The more government does which is good, the more people do not like it Thus, public policy design and corresponding organizational behavior must pursue broader questions of what mix of coordination rationalities—market, hierarchy, and/or community networking– would one pursue with different levels of scale to reflect less coercion and strengthen more self-governing individuals and communities? Public administration is becoming a scaled, relational administration of publics.

REFERENCES AND FURTHER RESEARCH

See Peter Drucker Management 4^th edition for key insights in managing work in the knowledge economy. See Elliott Jaques A General Theory of Bureaucracy (Heinemann 1976) and his many later volumes for basic reference around the authority through a manager exercises discretion, dynamics of maturational managerial capability, and the natural scaling/structuring of a panoply of roles. He makes the case these processes and definitions should be made explicit, clear, and mostly transparent. For a dynamic, open systems, cybernetic approach, incorporating Jaques’ research findings into circular relationships with more revolutionary definitions for society, see Luc Hoebeke Making Work Systems Better (John Wiley).

With a productive line of funded action research, differences in individual task awareness and corresponding scales of human action first developed by Elliott Jaques and associates can be connected, for example, to situated, group problem-solving in the public sector, such as discussed in Christopher Ansell’s Pragmatist Democracy. Peter Friesen and colleagues, for example, did a preliminary, unpublished, exploratory study of judicial administration which revealed promising hypotheses to be tested regarding requisite scale of service delivery and obligation.

Similarly, identifying scales of action can be integrated into a broader societal formation of coordinating large scale cooperation, constitutions, and micro contractual choice(s). See Professor Anna Grandori’s volume Epistemic Economics and Organization: Forms of Rationality and Governance for a Wiser Economy. Her technical perspective can be complimented by including differential individual task awareness in order to embody uncertainty in scaling coordination.

While there are many variables around inter-scalar activities, few of these research studies possess a methodology that can identify generalizable boundary thresholds of collective action as Kurt Lewin originally envisioned. There is a need for more experiments, flexibly adapting the “old fashioned”, unrecognized, “black sheep” of the action research family: social analysis. See Ralph Rowbottom Social Analysis.