Agile and Lean Convergence in software and creative development practices


Looking under the covers of the popular principles of Lean and Agile it’s very obvious why these principles are so very important to progressive organizations, but these two principles also seem to be on a path of convergence into a new and more widely accepted set of principles and methodologies.

In larger scale projects the more significant advances in performance outcomes in recent years seem to be coming from the principles of Lean manufacturing which aims to eliminate waste and by using more reactive demand based communications between the start and finish of an operation “pulls” through the supply of inputs (i.e. information, software components, functionality, content) as required by the next stage of a given operational function.

On the other hand no one can dispute that software development has benefited greatly by it’s departure from Waterfall principles to the widespread the use of Agile. The problem with waterfall is that whilst there is considerable care and effort placed in gathering and documenting requirements these requirements are ultimately; incomplete, insufficient or ill-conceived by the time they reach the development stages. Agile has the ability to make corrections on the fly, encourages mid stream corrections, customer intervention and tests outcomes at the end of short intervals. With agile you build the smallest possible components, and progressively assemble an MVP from the inside out.

Agile is of particular benefit in creative operations such as software, digital media and design, but it still has its drawbacks especially its inability to facilitate working with exact budgets or time frames. As a result experienced agile practitioners are adopting principles that seem more Lean than Agile in origin. Some Agile projects must also allow deviation from core principles where smaller items of development are impossible to test until combined with other components to form a sub assembly or MVP which could be interpreted as the Lean principle of converging streams.

In Agile there is the principle of rapid feedback between the users providing the “information” which in Lean manufacturing is the raw materials, to the transformation process (expert that transforms this into a product in some form), and instead of there being requirements (inventory in manufacturing terminology) there are story boards (orders for a finished product or desired outcome).

Agile is essential in creative environments as it gives the skilled expert the opportunity to use their knowledge and intuition to balance “what is technically possible” with “what the customer really wants”.

Hall et al “Harvard Business Review” (2009, Vol 87) reminds us ‘creative activities are often described as “judgment-based” work,” “craft work,” or “professional work.” The common thread in such work is variability in the process, its inputs, and its outputs.’ and in this “Highly variable environment: where attempting to use structured process management to blindly reduce variability. Not only does that reduce accountability, but it often causes workers to switch to autopilot instead of trying to understand the specifics of each job.” we must remember that the “variations of output is what the customer sees as value”
and
“creative processes can and should reliably produce innovative products and services that many structured [Waterfall style] processes cannot mimic.”

In large teams Agile has to give way to Lean principles ability to coordinate the work of different teams (streams) working either sequentially or in parallel. The Kanban (a lean principle) is widely used for managing Agile iterations.
Since some iterations are completed faster than others, in larger projects there still needs to be multiple points of consolidation or synchronization.

Many larger technology based cloud service providers are adopting Devops practices (a convergence of agile system administration with agile operations)  to reduce time to production increments for new capabilities, where agile has been adapted to deliver finished goods in smaller increments directly to the production systems accessed by customers. This in itself is giving rise to new operational capabilities best described as Lean principles. The Lean continuous improvement practice of Kaizen in conjunction with Agile and Devops teaches the teams to seek out improvements to both the product and the process used to produce the product.