Managing Director, Merrill Lynch Equities Trading Development: We looked at everyone we could and didn’t find a fraction of the background, insight, or capabilities. Brad gave a seminar in the didi design methodology that left my development team inspired and motivated.
The Cognitive Engineering Design Methodology
The process we use to apply Cognitive Engineering, honed over decades of research and real-world projects.
[expand title=”Online Class” expanded=”true”]
The online class is five sessions long. It’s a compressed, updated version of the Columbia University Department of Computer Science COMS 6998-005 graduate seminar class, the result of a 2003 appointment of W. Bradford Paley as an Adjunct Associate Professor to teach his own interaction design methodology, giving graduate students an alternative to conventional user interface design processes.
Click here to enroll or email us with questions.
Certification: A certificate is generated; it confirms “[name] has attended a class in the Cognitive Engineering Design Methodology.” If a student excels by completing all sessions and turning in exercises that demonstrate they’ve mastered the material well enough to apply it to real-world projects their certificate will read “[name] has mastered the fundamentals of the Cognitive Engineering Design Methodology.” (Approximately one third of the Columbia students reach that level.)
Format: There is an introductory overview and four primary class sessions, each 2.5 hours long. Professor Paley is available in common on-line “office hours” on consecutive days to be arranged as classes fill. Exercises must be delivered to didi within three days of each class. Certificates or additional one-on-one spot-retraining are returned within three weeks.
The class is always taught by Professor Paley, and he himself is present during online office hours. Here’s a short bio. Grading for certification may be done by colleagues with expertise in the methodology.
[expandsub1 title=”Class Session Synopses”]
One: Extracting the Expert’s Mental Model
Attendees learn to work with domain experts, unambiguously capturing what’s in their minds as they do tasks. CEDM structured outlines rigorously break down expert workflows into goals, tasks, and steps; separately capturing task-related cognitive entities, contexts, states, and transitions.
Two: Paper & Pencil: The Ultimate Rapid Prototyping Tools
We teach how to use standard office supplies fluently, to facilitate conversations with experts that results in “Schematics” which capture everything necessary to accomplish a task, and can be verified to do so. Schematics often result in innovation since they reveal how tasks are organized inside the expert’s mind. They leave an interface’s final layout to an explicit cost/benefit budgeting stage.
Three: Step→Affordance Maps; Information Layering, Managing Attention
Attendees learn to map each mental step in every task to the available affordances, creating a “Release Candidate.” This mapping provides a rational basis for what to prioritize or cut—or can quantitatively justify a bigger budget. The supplied CEDM Feature Evaluation Tool embodies economic techniques to approximate even hard-to-value features. This lets all stakeholders balance concrete, real-world, ongoing benefits (like time saved and reduced risk) against one-time development costs.
Four: Paper Prototyping, 100:1 Savings Finding/Fixing Bugs
We teach how to test Release Candidates, completely and in realistic situations before spec-writing begins, by simulating the system with paper “screens.” This engages the actual mental task steps the expert does in the real world, unlike a PowerPoint walk-through. Students learn to identify exactly what’s wrong and often fix it in a single session—a hundred times faster, assuming design takes 1/10th the time spec-writing does, and specs take 1/10th the time of coding.
[/expandsub1]
[expandsub1 title=”Class Session Topic Details”]
One: Extracting the Expert’s Mental Model
Understanding the actual workflow—directly from the expert—provides the absolutely essential foundation for designing tools that effectively support people executing their tasks. This session steps through social aspects of working with domain experts: detailing the pre-meeting research that will engage experts, defining what should be accomplished in interviews, initiating and respectfully accomplishing on-site observation, and sharing how to get into the most effective tool-design process we’ve seen: an annotated, active apprenticeship.
Structured outlining techniques are taught; they capture the goals, tasks, steps, and cognitive states/transitions necessary for the creation of “schematics” (the methodology’s more rigorous alternative to wireframes) in the next session.
Two: Paper & Pencil: The Ultimate Rapid Prototyping Tools
It’s no easier to use paper and pencil effectively for interface design than it is to code; but fortunately the medium is familiar, and once learned it’s extremely expressive. We teach how to use standard office supplies (paper, pencil, eraser, tape, pens, highlighters) in a structured way to facilitate an “object conversation” with an expert—in which the expert is the one driving the creation of the interface.
Semantic drawing exercises train students in the graphical ability to sketch interfaces as a fluid part of the conversation with experts, bringing the expert directly into the development of schematics. Schematics are annotated images that capture everything necessary to accomplish a task. They are often innovative—even visionary—since they detail how each task is organized in the expert’s mind, where it almost never looks like today’s field-forms or spreadsheet-like grids. Schematics also approximate what task information and actions might look and feel like in the final system, capturing meaning-based physical mnemonics that let the next step shape affordances as maximally recognizable, distinguishable, and expressively able to carry the information needed for each task.
Three: Step→Affordance Maps; Information Layering; Managing Attention
Schematics define all the functional parts of the interface, but don’t dictate its final configuration; that’s driven by mapping each mental step in the task to the available display/action affordances. This mapping also provides a rational basis for fitting within a development budget—or financially justifying a bigger budget: the one-time costs of more sophisticated development efforts can be directly compared with estimates of the benefits they cause in real-world business processes. Economic approximation techniques are taught; they allow stakeholder discussion about even hard-to-value costs (e.g., the cost of disappointing a customer with an incorrect transaction) in terms of time saved, risk minimized, or other metrics. A spreadsheet-based evaluation tool is provided and taught.
Renderings at this stage can be paper sketches, bitmaps, or even working prototypes; they ensure that the UI correlate of each mental step in a task is easy to find, recognize, and act upon. Perceptual distinctions drive our attention in many different ways: drawing the eye to new information; hiding expected, rarely needed labels and affordances in plain sight; grouping objects; hierarchically segmenting windows into simple, possibly re-usable modules; and clearly associating labels with what they label without distracting. Cognitive science concepts such as pop-out search versus sequential search, flagging functional distinctions with distinct “visual vocabularies,” and rapid categorization/recognition of objects based on feature recognition are discussed and tied to design.
Four: Paper Prototyping
One of the most cost-saving parts of the methodology is the ability to test more completely, in more realistic situations, much earlier in the process as compared to most development processes. “Version 1.0s” are rarely fully usable, and coping mechanisms for the failure of early testing—such as a slowly staged roll-out and agile software development—have become institutionalized. PowerPoint walk-throughs are used to get business-side sign-offs, but in practice they don’t actually prevent releases that are missing key things needed to accomplish a task; this still happens because they don’t engage the actual intellectual mechanisms that are in play when a task is being accomplished.
Paper Prototypes combine renderings of every significant state of a system with the intelligent paper-shuffling of the system designer in front of an expert. The designer/tester stands in for the system and engages the actual task-completion mechanisms in the expert’s mind/behavior well before coding begins—even before specs are written. The process also exactly identifies what’s missing or wrong and fixes it in a single session—saving huge amounts of development effort, costs, and calendar time, and improving business/developer relations. Bug detection and correction can happen hundreds of times faster than we see in development processes that catch bugs after coding, assuming conservatively that designing takes 1/10th the time of full specification, and writing specs takes 1/10th the time of coding.
[/expandsub1]
[/expand]
[expand title=”On-Site Introductory Lecture (suspended for now)”]
The lecture is a one- or two-hour lecture introduction to the CEDM.
Format: The lecture is done at a client’s site; it’s a two-hour lecture and includes a 15-minute break that is also time for informal questions.
Material Covered: The entire contents of the class are discussed in reverse order: backwards with respect to how they are applied in practice. We start with easily understood visual layering effects and build to more abstract ideas concerning experts’ mental models. The presentation is intentionally quick; it serves to lay out and integrate the dozen or more disciplines that provide the underpinnings of the methodology—from Typography to Vision’s object recognition, from Attention and Memory to Psycholinguistics’ jargon development.
Staff: The lecture is always presented by Professor Paley. Here’s a short bio.
[/expand]
White Paper
Rather dense, but a pretty comprehensive case study overview of applying the methodology to a trading desktop.
Resources
These books provided the foundations of the methodology.
