The Unaccountability Machine (Hardback). Why Big Systems Make Terrible Decisions - and How The World Lost its Mind. by Dan Davies. site 
This page is a Forage.
The part that has stuck with me most was in the introduction where Dan Davies paraphrased his earlier book, Lying for Money, as a way to frame the central inquiry: why do businesses consistently made decisions that no one seems to agree with? He described the 2008 financial crises as the result of self-organizing control fraud—systems that commit crimes without criminals.
The core mechanic is to set unrealistic profitability goals and underinvest in legal and compliance teams. Break these feedback loops, and you’ll get systems that run out of control. That paraphrase feels deeply explanatory about what I see in the software business.
I appreciate his writing and the rest of the book was helpful to me to understand Stafford Beer’s Viable System Model (VSM). I’ve been hearing about VSM for about a decade from different people and I think Davies telling of it helped me understand a bunch of earlier conversations.
I am not convinced by his proposal that VSM would fix the problems in the world.
In the context of Resilience in Software, I can imagine resilience engineering being a project to restore, maintain, and balance the feedback loops that keep our systems guided towards healthy outcomes instead of destructive outcome.
A colleague observed "the other part that was illuminating for me was about the rise of private equity and the extent to which the developments of the past 50 years were a conscious effort on the part of capital to break the managerial class."
(from chapter 5: Cybernetics Without Diagrams) > Diagrams present you with the information ‘all at once’ and leave you to work out the flow of cause and effect for yourself, while a verbal explanation usually presents you with the story of cause and effect and leaves you to remember the connections. In the context of systems, where feedback is ubiquitous, the relationships are vital and the flow of cause and effect has no obvious start or end point, it’s not hard to see why people draw diagrams. But this emphasis on connections means that diagrams are often ineffective ways of explaining something for the first time; they give you a network of relationships, but in a context that doesn’t tell you much about what the things are which the relationships are holding between.
For a wiki mashup of diagram and prose that is also relevant to discussion of complex systems and feedback, consider How Thermostats Work.
(from chapter 5: Cybernetics Without Diagrams) > Viable System Model has five parts: > 1. the part of the system that does things, > 2. the part that stops things getting in each other’s way, > 3. the part that decides what to do today, > 4. the part that’s responsible for looking at how the environment is changing, and then > 5. ... to be honest, it’s difficult to explain the last part without going into more detail.
Marc has lots of thoughts about the VSM including some notes about The Unaccountability Machine. See From Whales to Ecosystems.
(from chapter 6: Economics and how it got that way) > The damage caused by assumptions about time and uncertainty is more subtle. … when the information set is reduced to price and quantity, and when decision-making is just optimisation of a single value expressed in money terms, there’s no way to talk about ambiguity, perception or incompatible values. This is how cost–benefit analysis goes wrong; a statement like ‘this passenger aircraft has a faulty stall sensor which might cause it to crash’ fundamentally isn’t a piece of information about expected costs – even if you can get people happy with the idea of assigning a financial value to human life.
(from chapter 7: If You’re So Rich, Why Aren’t You Smart?) > Some of the biggest problems of management are problems that economics not only fails to address, but seems to lack the tools to recognise.
.
Fred Hebert's book review. article
Designing Freedom: The 1973 CBC Massey Lectures by Stafford Beer. archive