The Protocol Paradox

… but is it scientific?” includes an interesting 2x2 illustrating the inherent risk associated with scaling different approaches to scientific inquiry.

The X-axis represents the degree to which a method can be applied reliably and consistently across different contexts and scales. Low-risk scalability implies a method is generalizable, while high-risk scalability suggests a method’s effectiveness may be context-dependent and prone to failure.

The Y-axis draws a distinction between explanatory and predictive claims. Explanatory claims focus on understanding a system, while predictive claims aim to forecast future outcomes or events. There is an implicit tension here between explanation and prediction, as different approaches prioritize one over the other.

I’ve applied this 2x2 to further explore the distinction between strong protocols, weak protocols, and strange protocols:

Sense-Making: Weak Protocols
As mentioned in the original post, the Cynefin framework is “derived from theory applied and modified in practice, but constrained by theory.” It provides flexible guidelines that emerge from the community’s needs and the specific context. These guidelines (or “guide rails”) align with the characteristics I would attribute to weak protocols.

Classic Science: Strong Protocols
Classic scientific methods rely on testable hypotheses and context-specific claims. There are well-established, rigorous protocols for experimentation, validation, and replication. These “guardrails” fit well with the concept of strong protocols.

Case Based Interpretations and Processes: Weak to Strong Protocols
This domain likely spans a range from weak to strong protocols, depending on the specificity of the cases and the generalizability of the derived processes. Some may provide more general guiding principles (weak protocols), while others may prescribe more rigid, context-specific procedures (strong protocols).

Pseudo-Science: Strange Protocols
Pseudo-scientific claims, like simplistic recipes, hype, and overly universal assertions, often promise simple solutions to complex problems. They can exhibit unconventional “reasoning” that defies established scientific understanding, yet can still compel belief and following. In this sense, some pseudo-scientific claims share characteristics with the speculative idea of strange protocols - opaque yet potentially influential frameworks that emerge in complex systems. However, not all pseudo-scientific claims may fit this mold. Some might be better characterized as flawed applications of weak or strong protocols, lacking empirical validation or rigorous testing.

The protocol paradox then arises from the inherent tension between the desire for predictive power, often associated with strong protocols and rigorous methodologies, and the need for explanatory depth, often facilitated by weak protocols that allow for flexibility and exploration. This tension is perhaps further complicated by the emergence of strange protocols, which challenge traditional notions of scientific understanding and may offer unforeseen paths to discovery, blurring the lines between prediction and explanation. Navigating this paradox requires I think not only an appreciation of each protocol type, but a willingness to embrace the uncertainties inherent in scientific exploration.

Source: … but is it scientific? - The Cynefin Co