Prev
Next
Site Map
Home
Please reload page for style change
| . | . | . | . | David McCracken |
Design Principlesupdated:2016.07.13 |
Why waste time learning when ignorance is instantaneous? -- Thomas Hobbes
When I was young it was cool to be dumb, and boy was I cool -- Anonymous
The most fundamental principles of engineering are not engineering. They are logic, reason, and truth. Without these as the bedrock, details, however correct, are nothing but obfuscation. An explanation that depends entirely on details and cannot be pared down to logic accessible to any intelligent person is probably incorrect. As in science, opinions have no purpose in engineering. Theories, testable and based on fact, are the only acceptable means of advancing an idea. The purpose of testing a theory is not to prove it but to determine whether it is correct. Testing should try to break the theory.
Once upon a time waterfall model was seen as the only way to avoid development chaos. Its only shortcoming was that it didn’t work. As Bjarne Stroustrup, the creator of C++ and director of thousands of engineers at AT&T, explains in The C++ Programming Language:
The waterfall model suffers from the fundamental problem that information tends to flow only one way. When problems are found “downstream,” there is often strong methodological and organizational pressure to provide a local fix; that is, there is pressure to solve the problem without affecting the previous stages of the process. This lack of feedback leads to deficient designs, and the local fixes lead to contorted implementations. In the inevitable cases in which information does flow back toward the source and cause changes to the design, the result is a slow and cumbersome ripple effect through a system that is geared to prevent the need for such change and therefore unwilling and slow to respond.
Agile claims to be the antidote and many of its suggestions are useful. However, its focus on short-term goals affords no guidance in the development of large, multi-domain, and leading-edge projects, which necessarily include goals that cannot be realized in a sprint. It is naive to expect a stream of sprints, without an overall plan, to evolve toward some ideal but unspecified form.
Waterfall and agile are both crippled by a lack of guidance in capturing the rationales for decisions. Of course the waterfall-driven system is “unwilling and slow to respond” when it doesn’t know why the decisions were made in the first place. Agile temporarily avoids this problem by having one group of people work on one problem for a short time, during which most of them remember recent rationales. But these are soon forgotten when the participants move on to other things. Long term, agile is no more responsive than waterfall. Any development model that doesn’t demand a record of every relevant issue discussed in the evolution of every decision at all levels will not support revisiting decisions when conditions change or a chosen path proves unwieldy or impossible. Waterfall and agile both fail to provide a framework for responding to long-term changes.
Most agile procedures can be used in a waterfall framework. The only significant conflict is that agile allows immediate customer involvement in setting and evaluating low level requirements where a strict waterfall requires these to be derived from a higher level. In reality, successful projects don’t follow a strict waterfall or agile model. Design teams that dogmatically claim waterfall either overlook the fact that orders from above are often ignored or believe that when they finally get the “few bad apples” in line, everything will be perfect. Those that claim agile purity overlook the fact that someone has defined an overall strategy and provided them with goals within that framework.
I used team programming, sprints, and time-boxing before Agile was invented, but I have always used the waterfall concept of decomposing problems so that work begins on a particular problem only after defining its purpose within a larger framework. The heterogeneous model works because of my overarching Rationale model. Regardless of specific procedures, no decision at any level is considered unassailable. All rationales are public. A developer at any level can examine and question the chain of decisions leading to their current assignment. Even the highest level decisions can be revisited if their rationale is inconsistent with current conditions or new information. If a decision is shown to be wrong then it is changed as soon as possible, even if this causes a painful ripple effect. Continuing to implement a bad idea to avoid the pain of correcting it simply wastes time and effort. This is similar to how Japanese manufacturers significantly improved quality by letting even an assembly line worker stop the process upon discovering a defect.
The Rationale model doesn’t prescribe any procedures that conflict with waterfall or agile. It prescribes procedures for conducting and documenting common design situations that occur under any kind of development model. Ordinary documents, created with a word processor, capture complete design history, using extensive hyperlinking of topic threads composed of relevant information wherever it appears. If a design analysis focused on one topic crosses the path of another, the related sections in both topics are cross-linked. No content is duplicated but no relevant information is lost. A large design may produce hundreds of individual documents and many thousands of pages but all are linked together by a multi-dimensional web of topic threads. Most documents, for example CDX System and Hardware Design, contain links within the document, to other documents in the project web, and to local and remote documents outside the project web.
Cross-topic relevancy and change within each topic are inherent in any successful design effort. But only the Rationale development model avoids the tendency of organizations to produce silos of design documentation for specific topics that either ignore other topics that have influenced the design or contain copies of that information with no automatic means of detecting relevant changes in those topics. In a regulated environment, the Rationale document system is, without any additional effort, a complete and current Design History File and traceability matrix. Other required documents are essentially secondary, derived from the primary document system as a QA function.
The word “method” has practically disappeared from the technical lexicon, having been replaced by the improperly used word “methodology”. Methodology is not the application of a method but the study of methods, particularly of design and problem solving. The systematic study of design methods was introduced as a discipline in the 1960s, largely by Fritz Zwicky, an astronomer at Cal Tech, and Horst Rittel, Professor of Design Methodology at the Ulm School of Design in Germany and Professor of the Science of Design in the UC Berkeley Architecture Department. The “Zwicky Box” is a method for systematically investigating all of the relationships that might exist in a multi-dimensional problem space; essentially a complex mixed-domain Karnaugh map. Rittel, among other things, suggested that the most effective design processes are based on deliberation and argumentation. His IBIS (Issue-Based Information System), an attempt to use computers to assist in guiding the process, was not originally successful because there was no standard means of tracking independent but intersecting threads. With the invention of hypertext we now have the means to realize his vision. This is what I have done in my Rationale Development Model.
Design methodology might be easily dismissed as “ivory tower” musings if it were not so obvious that industry has failed to solve the twin problems of development model and documentation. The one recruitment question always asked of medical product design engineers is “Do you know FDA documentation standards”. This question always refers to specific documents rather than a general ability to document the essential and existential design of a regulated product. In less regulated industries, essential design documentation is rarely even considered. Many companies struggle to produce minimal existential documentation. On the issue of development models, the basic attitude is “we agree to disagree”. No one attempts to formally reconcile the waterfall and agile models, even though without an informal (and undocumented) reconciliation it would be very difficult to produce a useful product.
Traditionally, group engineering centered on detailed reports, analyses, and discussions. Pictures were introduced only if they served to illustrate an analytical point. For anyone not fully immersed in the subject, this was boring. Then someone decided that a slide show would be less boring. PowerPoint presentations became de rigueur and it became common to require project leaders to “sell” their plan with a superficial presentation. This mentality trickles down so that all engineers feel obligated to sell their opinions instead of discuss issues.
PowerPoint is the wrong medium for presenting engineering issues, which require more information density. The factors that lead to choosing one approach over another, the unknowns, the possible downsides, predictions, etc. are essential parts of a legitimate discussion. In 2003, the Columbia Accident Investigation Board at NASA specifically identified the use of PowerPoint as a significant contributor to the shuttle disaster. They said NASA had “become too reliant on presenting complex information via PowerPoint,” citing The Cognitive Style of PowerPoint by Yale professor Edward Tufte, a specialist in the visual display of information. In his analysis, Professor Tufte argues that PowerPoint “forces people to mutilate data beyond comprehension... the low resolution of a PowerPoint slide means that it usually contains only about 40 words, or barely eight seconds of reading. PowerPoint also encourages users to rely on bulleted lists, a faux analytical technique... that dodges the speaker’s responsibility to tie his information together. Ultimately... PowerPoint is infused with an attitude of commercialism that turns everything into a sales pitch.”
PowerPoint doesn’t even really relieve boredom. It just gives the audience something to stare at so that they become hypnotized instead of falling asleep as a single presenter drones on with little discussion or involvement of the audience. A traditional detailed discussion by active participants is less boring for them. PowerPoint homogenizes the audience. They are all hypnotized even if some would be more engaged if they had something to do. In this regard, PowerPoint is an anachronism generally; a throwback to the days when passively sitting in an audience seemed like participation. Today, people want to be actively involved. They want to decide for themselves what details are worth further investigation. They don’t want to be spoon-fed information that is deemed of sufficient general interest. This attitude is consistent with traditional engineering, which encourages participants with deep knowledge and interest to pursue specific points to help inform the general discussion. The correct tool for this is a hyperlinked web.
In general, the simplest method that works is the best solution to a problem in any domain, but in software design the simplest solution is not always obvious. Unlike physical objects, software incurs little or no production cost but high development cost. Anticipating and accommodating future requirements may be simpler in the long run than just barely meeting the current requirement. Frequently, the simple solution that addresses only the initial requirement is replicated with minor changes to address additional requirements, resulting in the worst kind of complexity. For a silly but illustrative example, given the requirement to add 1+1, the simplest solution would just be the answer. But what would we do with a new requirement to add 1+2? Typically, programmers follow an existing pattern because they don’t know the implications and side effects. So the program would become “if 1 and 1 then 2 else if 1 and 2 then 3”. This is the control-flow method that most programmers use to address most problems. With a few more additions this begins to look really dumb. But instead of going back to the root and devising a solution for the class of problem, many programmers convert the predicate logic control flow into an object-oriented control flow using virtual functions. Having hidden the redundancy behind an OO veneer, the underlying problem class and the possibility of its efficient solution disappear forever.
We all know the right solution to the problem of adding. Deconstructing this common solution is instructive. A lookup table (in our minds) provides the answer for any pair of digits. An algorithm based on the ancient Indian invention of positional notation and 0 uses this table repeatedly to generate the answer for infinite numbers. It is no coincidence that this looks a lot like the Chomsky hierarchy, which describes natural and artificial languages. The lowest level is a set of results organized for quick lookup using the inputs to the problem. This provides a very fast solution and easily handles arbitrary information for a relatively small set. It consumes too much memory for larger sets (infinite ones in the case of languages). A more complex process, i.e. algorithm, repeatedly appeals to the lower level to solve more complex problems.
The pattern of intelligent computation is well established. Predicate logic is the most complicated solution to all but the smallest problems but it can handle any arbitrary relationships. Lookup logic is faster and less complicated and can handle relatively simple arbitrary relationships. Algorithm is the least complicated solution but usually has to appeal to lower levels of computation. It is the only computational method that handles infinite sets.
It is easy to design a program using predicate logic, but what seems like the minimal solution initially often becomes the most complicated one. We are certainly willing to wait for millennia for the algorithmic solution to the problem of adding. But what about mundane programming problems? An algorithmic solution is best in many cases but requires the most design effort (in some cases a creative leap). However, there are a few design patterns that can reduce the effort. Most of us don’t have a Gaussian ability to instantly see a problem in terms of an algorithm but we can methodically examine the problem for opportunities. We usually devise our own or inherit predicate logic solutions. It is very simple (in fact, it could be automated) to find redundancies in these solutions. Frequently, this alone reveals a few simple formulas acting on inputs and fixed arbitrary values. Moving the fixed arbitrary values into tables accessed by more generic formulas is often a trivial exercise.
Separating predicate logic into procedure and table is a simple means of reducing complexity. Even more radical reductions are feasible but require some creativity. Control theory provides a very powerful pattern in the concept of multi-term control, for example PID (Proportional-Integral-Derivative). This is a more general concept than most people realize. Reducing a complex ambiguous situation to a single unambiguous answer is extraordinarily powerful and is exactly what we want. One of the most common things people say is “Don’t tell me all the problems; just tell me what to do”, and this is the only way to talk to a machine. Equally powerful is the concept of combining measures of things weighted by their relative importance to derive an answer. Of great practical value in many situations is that the control formula is a deterministic problem, with a constant solution time. Ideally, nearly every problem would be addressed by a control formula. In reality, it is difficult to devise terms other than the ones that control theory suggests, which are error (between current situation and the goal, i.e. P) and various time-frame views (e.g. I and D) of the error and whatever mechanism the answer is applied to.
There is no simple method for discovering new control formulas. We just have to think about the problem. For example, users were sometimes frustrated trying to move the display cursor onto a precise target with my differential capacitive touch device. It seemed that acceleration, which increases or decreases the virtual distance response to physical finger movement according to speed, was the primary source of frustration. Like most small touch devices, acceleration was a discontinuous function and impossible to control near the discontinuities; and it either lagged or overreacted. These are classic characteristics of inadequate single-term control systems, but this does not exactly match the control system pattern. The error is in the mind of the user. The first creative leap was to see this as a control system problem even without a directly measurable error. A measurable error had to be inferred from finger movement. To provide this, I invented sub-perceptive gesture analysis, essentially a means of measuring user intent and frustration. A multi-term formula combining different time-frame views of finger speed drastically improved acceleration, but did not address all sources of frustration. For example, it did not prevent rubber banding around the target. However, there is no absolute means of detecting this situation. If acceleration suddenly changes because we erroneously think we see rubber banding, the user will be even more frustrated. A far better solution is to make the likelihood and severity of rubber-banding another term in the control formula. This has proven extremely effective. Unfortunately, this new control formula is not a generally applicable pattern like PID. It only serves to illustrate how difficult it can be devise new formulas. Nevertheless, they may be the best or only solution to some problems.
Minimalism not only provides good strategic guidance but tactical as well. Most obviously we want to reduce redundancy. Humans naturally recognize patterns but not when they are cluttered with noise or too large to see in a single view. Redundancy clutters the essence of a program and consumes space (display or mental) that would be better utilized presenting a compact view of that essence. Further, knowing that small but important non-redundancies lurk in replicated code we are afraid to make changes without careful inspection. But the redundancy itself bores us into overlooking these details.
Related to redundancy is the concept of strength reduction. Circumscribing the extent of what a piece of code can do reduces the effort needed to understand and verify its behavior. This is the underlying rationale for using specific loop patterns, like for, do, and while, instead of goto. The goto is simply more powerful than necessary. However, programmers are so afraid of goto without understanding why that they will do anything to avoid using it, including creating excessively complex loop controls, while loops that actually never loop, and local exception mechanisms. These may have reduced strength compared to goto but they are much more complex and the purpose of strength reduction is to reduce complexity.
Most programmers understand the value of functions in reducing replication of blocks of code. Many don’t understand mechanisms of more limited scope. Most languages have a conditional operator and most programmers don’t realize that it is not intended to replace if-then-else statements. It is an expression, which can be used to reduce redundancy by embedding alternation within a larger expression instead of duplicating the entire large expression. This can be a significant strength reducer. Many programmers also use if-then-else for a predicate that can be expressed by a conjunctive expression of lower strength. A crude programming style has consequences. The code is bigger, more complex, and more difficult to understand and verify.
Modeling is a useful software design tool when appropriately applied. The domain must either be simple and relatively obvious or thoroughly studied. For example, UML (Unified Modeling Language) effectively models business applications, which are simple and obvious. It is accepted without argument, for example, that a deposit to one bank account should not show up in another. The best known example of a complex but thoroughly studied application of modeling is the use of regular expressions and BNF to model the tokens and grammar of a computing language.
It is a mistake to think that modeling is useful for everything. Some of its proponents have suggested that UML could serve as a general-purpose language, completely modeling complex systems. That it cannot model any system more complex than one that can be described by a simple state machine should convince anyone educated in computer science that it cannot be effectively used in this manner. The DFA (Deterministic Finite Automaton) upper limit of UML’s language recognition capacity is the lowest model of computation in the Chomsky hierarchy. Even if UML were able to recognize higher level languages, it would still be ineffective for modeling complete systems. As Hopcroft and Ullman argue in Introduction to Automata Theory, Languages, and Computation:
The computer itself can be viewed as a finite state system, although doing so turns out not to be as useful as one would like. Theoretically the state of the central processor, main memory, and auxiliary storage at any time is one of a very large but finite number of states...Viewing a computer as a finite state system, however, is not satisfying mathematically or realistically. It places an artificial limit on the memory capacity, thereby failing to capture the real essence of computation.
Of significant value in modeling is that some or all of the application is specified by declaration instead of procedure. That declarative programs are less prone to error and easier to maintain is well established. However, formal modeling is not the only means of declarative programming, as I point out in my Dr. Dobb’s Journal article Software Partitioning For Multitasking Communication. For additional examples see BM-Hitachi 747-Communication and Coupling, Abbott CD3200-Algorithm Transformation, and Hybrid Tool For Universal Microprocessor Development. Any good non-trivial program should comprise a mix of declarative and procedural code. A model-based design builds a declarative framework within which procedural code is invoked as rules are matched. Alternatively, within a procedural framework, many decisions can be made by data tables instead of control flow. Both patterns are valid means of replacing procedure with declaration.
It is a mistake to design anything to requirements. Requirements state the minimum acceptable performance. They don’t anticipate changes or alternate uses. Only the design can do this. The requirements should be considered an instance of a larger class. One of the most difficult design tasks is to determine what the boundaries of this class should be. If they are excessively wide, there is a risk of waste due to unused potential. If they are excessively narrow (the worst case being the requirements) the design may have to be completely redone to accommodate a reasonable change or correction. Most engineers cannot do this task correctly and insist on designing exactly to requirements, producing inflexible systems that are broken by modest changes.
Boundary testing and unit testing are essentially identical. Although it is possible to test a complex system to the limits of specific parameters, it is not feasible to perform full boundary testing of a complete system because of the combinatorial explosion of the test condition space for all of the constituent parts. Most engineers design to a midpoint of minimum requirements and only reluctantly test at the requirement boundaries. If they were to do boundary-based design, boundary testing would not only be natural and easy but essential, as a complete system cannot create the boundary conditions for a component that is deliberately designed for a wider range of conditions than that system.
Application frameworks are a useful design management pattern, providing predictability without imposing excessively rigid requirements. An application framework is a collection of cooperative pieces that may be partially or totally reused to reduce the amount of original design needed to realize a particular application. The three major parts of a framework are architecture, components, and documentation. All three pieces are needed. Many failures result from a missing piece, especially documentation. Without existential documentation, it is often not clear how to reuse a component even in an anticipated form. Without essential documentation, minor discrepancies between a new application environment and the environment originally envisioned can appear as fundamental flaws rather than easily corrected oversights.
We know that large imprecise data sets sometimes contain valuable information that cannot be determined from the individual data points alone. The challenge is how to “see the forest through the trees”. If we know what we are looking for, we can usually devise a band-pass filter to extract it or a band-stop filter to remove interfering data. But we can’t devise a method to extract unknown “useful” information. Some people naively think statistical analysis can reveal new useful patterns of information. The truth is told by the saying “If you torture data long enough, eventually it will tell you what you want to hear”. In fact, given unbounded polynomial complexity, an infinite number of statistical relationships can be “discovered”. Any attempt to rein this in by limiting complexity or reducing the range of acceptable outliers will reject genuinely useful patterns. Turing’s halting problem teaches that it is impossible to devise a program that knows when it has discovered a useful new pattern.
Only humans can discover new useful patterns. In fact, we are very good at it, but the information has to be presented in a form that makes sense to us. Few humans see patterns in words and numbers but we have an acute ability to perceive meaningful visual, auditory, and olfactory patterns. Since we don’t have any operating systems with auditory or olfactory interfaces while the GUI is ubiquitous, visual is likely to be the most convenient means of presenting large amounts of data for humans to discover new patterns. Unfortunately, visual is dimensionally challenged. We have an unobstructed view of only two dimensions and, therefore, of two variables. Typically, a third spatial dimension is used to represent a third variable but this demands compromise. There is no point of view that shows everything and the more data that is presented in given plot, the denser the cloud and greater its opacity. An alternative, which is rarely utilized in big data analytics, is to map a third variable in time. This is especially appropriate if one of the variables happens to be time. Arguably the most common type of static plot is one variable against time. We could show that same variable as it changes in time. In fact this is often done with several one-dimensional gauges, but this presentation does not help a human to see patterns in the relationships between variables.
Hematology analyzers can teach some valuable big data lessons. Unlike flow cytometers, which traditionally use stains (or fluorescent tagging) to identify the type of individual blood cells, or the visual inspection (automated or manual) of individual cells, hematology analyzers use statistical analysis to identify the cluster(s) that each cell belongs to. An isolated cell means nothing. This is clearly a big data analytics problem.
Hematology analyzers acquire multi-variable data for each cell by measuring light refracted by the cell. Variations include multiple incident angles, refracted angles, wavelengths, polarization, etc. A histogram of the number of cells having a particular value reveals the clustering of a single variable, in the same manner as a spectrum analyzer. A scatter plot reveals the clustering of a pair of variables. The instrument can either present these to a human for analysis or perform its own cluster analysis based on known patterns. We know that there are useful patterns in combinations of more than two variables but their discovery is very difficult. A program can easily categorize cells based on clusters of any number of variables but it can’t discover new useful patterns and it can’t produce a visual representation of more than two variables for a human to analyze.
At Abbott we assumed that this problem was intractable and did not try to solve it. However, we had a related problem. The data acquired for each sample is saved in an FCS (Flow Cytometry Standard) file as the instrument operates. Algorithm developers review these off-line to define and refine pattern analyses. The instrument’s own displays are inflexible and not suited to exploring new relationships. The algorithm developers had been using commercial and freeware programs to display the FCS files. All of these programs assume a use model where one file is loaded for the user to examine in various ways. The algorithm developers have a different use model. They need display flexibility but once they have created a particular format, they need to see many (sometimes thousands) of samples in this format. They would type the name of an FCS file, load the file, set up the plots they needed, examine the data, close the file, and then repeat everything for the next file. It was painful just to watch them so I wrote a program for their usage model. I gave them the ability to save and restore any number of display formats, to keep the current format while closing one data file and opening the next, and to iterate backward and forward through directories and lists of files and directories without having to type file names. Simply pressing N or P closed the current file and opened the next or previous. I optimized plotting speed so that a full-screen scatter plot of 10,000 cells was done in 1/4 second. Now the developers could set up or load the display format and the source list and press N or P to quickly examine many samples from a particular viewpoint. For convenience, I made N and P auto-repeat. The principal algorithm developer said that auto-repeat with such fast display transformed the view into essentially a movie, where she not only could see sample-to-sample differences that she couldn’t see before but she also saw new and unexpected patterns emerge from the changes themselves. I had unintentionally used time as a plotting dimension, enabling an unobstructed 3-D plot.
Helping the algorithm developers was not one of my primary tasks and I took many shortcuts in the program, making it unusable outside of our project. When I was no longer working for Abbott, I wrote a more advanced and general-purpose program, which can be used in other situations and produce output more suited to publication. However, the central feature, still not addressed by any other program, is its optimization for analytic review of large cytometric data sets. I released this to researchers as freeware. The Dataman Flash screencast shows some of the capabilities of the program in a convenient way but only its actual use reveals how its extraordinary speed makes it a fundamentally new analytic tool.
Just as we can’t write a program to discover useful patterns in big data sets, there is no standard formula that humans can follow to discover them. However, there are some useful rules of thumb. One of these is that subtle patterns may be evident only when the data is viewed at high resolution. Paradoxically, these patterns may contradict patterns seen at coarser resolution and may actually be more important. This is one of the keys to the success of my hover touch flick interpreter. Another, also important in my flick interpreter, is that meta-patterns, that is patterns of patterns, may be critical to finding meaning in complex data.