Lifelet: A walk in the woods in between rain showers

Day 247 of Self Quarantine      Covid 19 Deaths in U.S.:  245,000 

As I doom scrolled through my morning twitter feed, I clicked through to McSweeney’s for a little morning humor to interrupt the many existential crises:


(2020 EDITION)

In 1820…

“A tree falls in a forest, and no one is around to hear it. It can’t be determined if it made a sound.

In 2020…

“A tree falls in a forest, and no one is around to hear it. However, a surveillance drone captures the tree falling on video.

“A scientist studies the video of the tree falling and then creates a study of why the tree fell. The scientist publishes a report, concluding that the tree fell prematurely due to accelerated soil erosion driven by climate change.

“The Sierra Club tweets out the report with the attached video of the tree falling to draw attention to the effects of climate change. It soon goes viral with fans of the environment everywhere.

“The video is reposted on a conservative Facebook page calling the report bogus. They put out a statement that the tree fell of its own free will and was not co-opted by some leftist climate movement.

“In response, a new Facebook group, “Friends of the Fallen Tree,” is created to counter misinformation spread by conservative media.

“Rush Limbaugh catches wind of the story. He claims that, rather than climate change, the tree was strangled to death by excessive tree-hugging and blames environmentalists for it falling over. The President retweets the claim. . .”

I think you get the idea.  But now the meme was firmly lodged in my feeble brain.  I knew I had to get into the woods for my morning walk to see if there were any trees that had fallen in the previous night’s wind storm.

I have to be careful this time of year as the fallen wet leaves obscure the many roots that are waiting to trip me up.  I have to move slowly and stop to see if there are any fallen trees near the trail.

As the trail widens I look forward to greeting the gnome homes and seeing what decorations adorn them.

Just a few yards from the Black Lives Matter gnome home, I stop at a recent fallen tree.

Did anybody hear it fall?  Did a drone capture its demise?  Fortunately, the magic trail clearing sawyer cut up the tree and moved the tree chunks.  Does this constitute a crime in 2020?  Do I have to be on the lookout for young men practicing “renegade tree falling”?

Now I am truly stumped.  What do I do with this travesty against nature?  The tree didn’t fall, the car did.

This tree just jumped out and crashed this old car.  Clearly, the car has been here for generations.  Did anyone hear the crash?  Did it really happen?  Inquiring 2020 minds want to know.

My trek through the woods on a rainy day is fun now.  How easily a little humorous essay can change my walk as I continue my quest to see fallen trees.

I stop on all three sides of the Wacky Nut Horse Farm Storm Water Ponds noticing how full both ponds are.  We must have had a lot of rain the last couple of nights.

I continue my search for fallen trees as I walk through the Labyrinth.

I really want to walk the slick stone labyrinth but the fallen leaves and slippery rocks caution me to wait for another day.  I stop and say a few prayers in the prayer wheel garden while listening to a seaplane take off below me in Blakely Harbor.  I absorb the patterns of nature and the rearrangement of nature by the women and men who created this little oasis.

I reflect on Marcia Bates definition of “information”:

“Information 1 is defined as the pattern of organization of matter and energy.

“Information 2 is defined as some pattern of organization of matter and energy that has been given meaning by a living being.

“Knowledge is defined as information given meaning and integrated with other contents of understanding.”

Bates, Marcia J.. Information and the Information Professions: Marcia Bates Selected Works, Volume I (p. 3). Ketchikan Press. Kindle Edition.

I am awash in Information 1 and Information 2 as I try to make meaning of the flow of cycles of nature and man.

I keep moving in my search for more fallen trees.  Alas, fallen trees are everywhere.  They’ve been falling along this trail for a 100 years.  Most are covered in moss and ivy and blackberry bushes.  Did anyone even notice?  Or hear?

Did anyone hold a DiCaprio ceremony for these fallen trees?

“In a tragic turn of events, the tree and square miles of the surrounding forest are completely destroyed in a fire caused by a gender-reveal party stunt gone wrong. (It was a boy.)

“Spearheaded by Leonardo DiCaprio, a fund supported by Hollywood celebrities pays for the ashes of the “falling tree” to be collected and interred at Forest Lawn Memorial Park. In his eulogy at the tree’s memorial service, DiCaprio noted, “We have brought the tree here to our special ‘forest,’ where it can remain a symbol to remind future generations that we need to stop trees from needless falling.” They watch the video of the tree falling in silence.”

I am glad I spent time at the gnome homes and at the Labyrinth saying a few prayers for all the fallen trees that made my walk today an inspiration.

Posted in Biodynamic, Citizen, Climate Change, Exercise, Lifelet, Nature | Leave a comment

Good Notes 5 – Surprised by Joy Some More

Day 218 of Self Quarantine  Covid 19 Deaths in U.S.:  219,000   VOTE!!

Certain to Win: The Strategy of John Boyd Applied to Business by Chet Richards is one of the books that Professor David Socha and I are referencing for our upcoming papers on “Observe, Don’t Ask.”  

In “Good Notes 5 – Surprised by Joy” I shared how amazed I was with the ability of the app to read my handwriting and then make it searchable.  In “What is a Book?” I described how many moons ago I completely shifted to Kindle digital books on the iPad for all of my reading.  I described the many benefits of the Kindle app including the ability to highlight and make notes about what I was reading and then have them available to search.

David sent me the following:

“I spent way too long reading Certain to Win last night. And then again early this morning, after which I went back to bed to sleep some. . . 

“Attached are my notes. Both from our session yesterday, and from my reading last night. One of the best parts of my reading session last night, was that while reading and annotating Certain to Win in Goodnotes, I came to a point where I wanted to add more notes than I could in the margins of the page. And then I realized that, wait, this is just a set of pages in Goodnotes, so I could add a page in the middle of my book! See the attached image. I was so used to thinking about this as a book that I am marking up, that I had forgotten / had not realized that the material of the book was different. That I could take advantage of all of the aspects of the materiality provided by Goodnotes. So I added a blank page into the middle of the book. Then I used the “Take Screenshot” feature to capture snippets of the text and pasted them into my notes, sizing them appropriately. What fun! And during this process I felt surprised that doing this was a revelation. “

Wait, What?

You can inhale a whole book length PDF into GoodNotes and make handwritten notes which are then recognized and they become searchable just like the rest of the book.  And both the original book and my notes show up in the same file?  You mean I don’t have to search inside the Kindle book AND then go to the highlights on the Amazon web site.  

I immediately have to try this for myself.  I download the Certain to Win PDF and search for the OODA loop page.  I then make some notes.

Way cool.  Writing notes in the margins is exactly what I used to do with paper based books.  Now I can do the same thing with a digital book.  Can I search what I just wrote?

Are you kidding me?  Even with the text written sideways, GoodNotes 5 was able to recognize “people” and then include the search result in with all the digital text search results.

I trust David’s report but I have to see for myself that I can add an extra page to write more extensive notes.  It is just that easy.

With the combination of being able to have my collection of digital Kindle books with handwritten notes in a single app, I am in tech nerd heaven.  Because it is saved in PDF format, the searchable text is available on any of my devices.  It’s taken 20 years since I first glimpsed this kind of capability and now it is here.  Not only is it here, but it is far better than I imagined.

This new capability is a “back to the future” of matching digital affordances with how I want to work.  I can work like I want to and yet my work is fully digital and searchable and malleable and remixable just as Lawrence Lessig talks about in Remix: Making Art and Commerce Thrive in the Hybrid Economy.

Thank you Amazon (Kindle eBooks), and Apple (iPad with Apple Pencil), and GoodNotes 5.  You’ve made my day on a rainy fall day in the Pacific Northwest.

And thank you David Socha as always.

Posted in Content with Context, Design, Know Now, Learning, Lifelet, Software Development, User Experience | Leave a comment

Good Notes 5 – Surprised by Joy

Day 212 of Self Quarantine  Covid 19 Deaths in U.S.:  214,000   VOTE!!

As Professor David Socha and I were collaborating on a paper about “Observe, Don’t Ask” and its role in the software product development life cycle, he pulled out his iPad and Apple Pencil and asked if he could share his notes.

For 20 years, our medium of collaboration was a white board.  We would find a conference room with the biggest expanse of white board and begin a collaboration session.  We’d both bring several colored markers to separate our notes while we visualized our thoughts.

At the end of a session we would take photos of our notes and use those images as fodder for our next round of collaborative writing.

Yet, since March and the Pandemic quarantine, we’ve been severely limited in our ability to collaborate around a white board.  Now, Zoom is our medium of communication.  Instead of a white board we are usually restricted to sharing a web page or a powerpoint presentation and talk around the shared screens.

The good news about Zoom is that we can continue our collaboration remotely AND capture a searchable transcript through

Yet, something magical happened when David started writing notes on this iPad.  For me, it transformed the experience into being back at the white board.  We were fully collaborating again.  Could it be as simple as the pixellated bits on the screen or was it being able to observe and think at the speed of writing.  Instead of a complicated slide being blasted into my feeble brain, the beauty of handwritten notes allows me time to absorb and process the meaning.

Meaning and understanding at the speed of writing.

While Zoom has the annotation capabilities, it is hard for me to write using a mouse.  It is far more natural to write with an Apple Pencil on an iPad.

I am giddy being able to almost experience whiteboarding with David again.

I finally ask David what application he is using on the iPad.  Goodnotes he shares.

I download it.

I decide to do my morning notes on my iPad this morning instead of in my current Moleskine notebook.  For over 30 years, I’ve carried a Moleskine notebook with me where ever I go.  Today, I will try something different.  

I wanted to jot down some thoughts about a blog post on searching and a tool I want to build – Personal Patterns.  I create a notebook from a library of templates and start writing.  Unlike with a Moleskine, I can change the colors of my ink, or type something, or drop a diagram from my photo library onto a page.  I can write my notes or annotate a diagram.  It is fluid and easy.

I continue making notes for an hour without thinking about the transition to this new medium.

As I stop, I need to somehow save the notebook I am writing in.  I find the save button and also notice a search icon.  “Nah,” I say to myself.  “They couldn’t possibly be OCRing my hand writing and making it searchable.”  So I try searching for “patterns.”

Now I am bouncing in my seat being surprised by joy.

Then, I wonder if when I sent the PDF of the notebook to myself that it created a searchable PDF.  On my desktop, I open up the PDF image file.  Up pops all the instances of “patterns.”

Once again the iPad platform and a terrific app like GoodNotes helps me to shift away from paper to digital.  I was overjoyed when I was able to shift from paper based books to the Kindle digital format.  Now, I can do the same thing with my note taking.  My back pack just got lighter once again. 

As with all of the advantages of shifting from paper books, everything I write is now saved in multiple places (iPad, desktop, the cloud) and it is searchable.  I can find notes in my notebooks, just like I can find notes and annotations across my Kindle digital books and within Evernote.  If I could only get 30 years worth of Moleskine notebooks into GoodNotes, I would really be a happy digital native.

In the midst of our pandemic and all of our existential crises, I am surprised by joy once again with how much easier my work and my collaborations will be going forward.  

Thank you, David Socha!

Posted in Content with Context, Design, Know Now, Learning, Lifelet, Software Development, Uncategorized, User Experience | 3 Comments

Observe, Don’t Ask. Show, Don’t Tell. Part 3

Day 200 of Self Quarantine             Covid 19 Deaths in U.S.:  205,000

Rest in Peace Justice Ruth Bader Ginsburg                                   VOTE!!

Professor David Socha asked me a second question after “where do you observe different parts of the software product development life cycle – what is the diagram that shows where to observe?”

I am continuously sketching the answer to David’s question.  As usual, I start with the complexified version of the diagram.  Over the last week, David and I discussed, debated, added and subtracted a lot from this diagram.  This blog post is the first attempt to come up with a diagram of an Agile OODA (Observe – Orient – Decide – Ask) HCD (Human Centered Design) Loop (virtuous cycle) for software product development.

As I stare at this diagram, I realize it is more of a macro diagram and is heavy with framework systems that require a lot of description and unpacking.  And this diagram is “simple” because I would really mess it up more if I added all the loops of iteration throughout the diagram.

David did a nice job in “Observe, Don’t Ask Part 2” of enumerating all of the activities in the software product development life cycle where you either need to Observe or Show.  The more interesting question is where you wouldn’t do Observe or Show?  We both realized the challenge of trying to document what we mean versus showing a more complete context of what we mean.

As I tweaked the above diagram, I wondered if there was a diagram that showed the feedback and feed forward loops  or maybe a Venn diagram to illustrate relationships in a clearer way.  As I doodled with different loops, I realized that I had not referenced the OODA Loop that John Boyd developed as a fighter pilot.

Boyd’s work in developing the Observe-Orient-Decide-Act loop was an important step forward in military tactics.  This strategy was the basis for our winning so quickly in Gulf War I and was adapted for business and commercial use in the books Winning in Fast Time and Certain to Win.  While Boyd did not write any books, he was famous for putting his materials into an extensive slide deck and giving the presentation to anyone who would listen.

Boyd’s original OODA loop diagram looked like this:

I love that there are both Feed Forward arcs and feedback arcs.

The folks at IMARC AI provide a simpler update of the original OODA loop.

As I play with this simplified diagram, I realize it fits nicely underneath all of the steps and frameworks in the complex diagram.  The “observe” step is obviously the “observe, don’t ask” step.  The “orient” step fits nicely with what I mean by “show, don’t tell.”  The “act” step needs a reminder of don’t dilly dally with “Act, don’t delay” which is the fundamental speed insight of Boyd.  The decide step fits a mantra of “prototype, don’t guess.”  So many times in the product development process, the software engineers are forced to guess what is meant by a user story or a requirements document or a short discussion during a daily stand-up.  What human centered design practitioners learn early on is to do rapid prototyping even if it means doing paper prototyping.

I can’t help myself and realize that I have to add to the simple OODA diagram.  While I love the original icons, I didn’t like the ACT step as it implied thinking to me.  Acting is about moving and I wanted to have something that had hand movement.  I also felt that there needed to be something in the middle that was an indicator of the “so what.”  Where does the OODA loop for software product development lead – to value.

For each step in the complex diagram (and the many sub-steps), the work is about figuring out what to observe, and then what insights arise from the observation, that need to be carried to the next step to “show” the developers.   At each step there are one or more decisions.  Some of those decisions can be made based on “patterns that repeat.”  Others require some rapid and iterative prototyping.  The prototyping takes the guess work out of deciding what needs to be built – the “act” step.  Then once something is built, that prototype needs to be observed in use “in the wild” of an actual user’s environment.

Then, the OODA loop repeats until there is a measure of good enough as described in the Part 2 blog post.

“To claim that any given thing is Good Enough is to agree with all of the following propositions:

              • It has sufficient benefits.
              • It has no critical problems.
              • The benefits sufficiently outweigh the problems.
              • In the present situation, and all things considered, further improvement would be more harmful than helpful.

The micro level OODA loop steps for software product development are:

Now that we have an idea of the core process underlying the more complex model, let us look at the components of the larger model.  The descriptions that follow assume that we are in an early stage startup where the product and the company are mostly the same thing.  Inside a large company the equivalent is intrapreneuring where a new product and new business are created together.

The macro OODA loop for software product development has several internal and external components.  The Observation step in the macro OODA loop is:

Over the years I’ve learned to start any new product idea by trying to fill out the nine boxes of a lean canvas.  The lean canvas below is an early draft I did for the Know Now book I am writing:

The lean canvas is a paper prototype of your product and business idea.  Filling in each box gives you a quick way to see what you know and what you think you know (The Four Boxes of Knowing).  The work is to figure out which box of content you know the least about.  That box becomes the priority to learn more about.  Most early stage entrepreneurs leap right into the Solution Box and have a wide range of ideas of how to solve for some problem.  Over the years I’ve found that I rarely understand the problem or the existing alternatives and who the early adopters might be.

For the Know Now book project, my hypothesis was that the early adopters would be product managers.  While I have done product management off and on for 20 years, I realize how little I remember about how I educated myself about product management.  So the first thing I needed to learn about is product managers and how they learned to become a product manager.

The key components of the Observe step come from the Human Centered Design process that I learned about at the Institute of DesignIdeo offers a range of online courses and extensive consulting services that help companies implement a robust human centered design process.  A starting point for learning about Human Centered Design is Vijay Kumar’s book on 101: Design Methods.

As I interviewed several product managers, I was also coaching several early stage startup CEOs.  I realized that one of the challenges in an early stage startup is that the CEO acts as the product manager due to lack of funds.  In an early stage startup, the product and the company are one and the same.

In parallel with identifying users to observe, the CEO as product manager is also developing a business plan for the product and trying to understand the external environment as depicted in the micro OODA loop.  Useful frameworks for observing the external world are captured in the Scenario Planning Process and in collections of useful frameworks.  The kinds of information that the CEO is looking for are social dynamics, economic issues, political issues, technological issues, culture trends, legislative and regulatory trends, and ecological issues.

Along with observing external factors, the CEO is always trying to estimate TAM, SAM and SOM for the potential product.

Now that you have identified the users to observe and performed the observations (where possible using video ethnography), then it is time to ORIENT.  The Macro OODA Loop for Orientation is:

As you make sense of your observations there are several frameworks that guide what to look for.  The observations should include the full flow of work.  Within the flow of work you are looking for the hassles that a user encounters.  Adrian Slywotzky describes hassle maps in his book Demand: Creating What People Love Before They Know They Want It and in a short monograph on The Art of Hassle Map Thinking.

An example of a hassle map is the one that Zipcar created:

No matter how manual or automated a given work flow is there are still hassles that a user must overcome.  This step of the Orient process is to identify all of the hassles and to start looking for solutions to eliminate the hassles.

Service Science emerged from IBM as their business transitioned from hardware to software to services.  Jim Spohrer in “The Emergence of Service Science: Toward Systematic Service Innovations to Accelerate Co-Creation of Value” provides a diagram of the evolution of automation of a manual task.

After identifying the hassles of a user, the next step is to understand where the user’s work environment is in the sequence of work evolution.  Each step has a different set of software development activities and degree of difficulty.  For most of my professional life, I focused on human systems that are mostly manual and then looked for ways to augment that system.  Spohrer points out that is just one step along the path.  Over the last 20 years of software product development in the eDiscovery space, we’ve seen the progression to Augment with our Attenex Patterns product, and then to outsourcing with companies like FTI Consulting and Lighthouse, and now with full automation with the AI/ML capabilities of NUIX Discover.

The next step in orienting to what you observe is understanding whether there are particular steps that have the most costs or that lead to the most revenue.  In the eDiscovery market the EDRM model is the generic view of the work flow.  When we did our early studies, we realized that most of the costs are incurred in the four boxes of processing, review, analysis and production.  We could make a huge difference on the cost structure of a matter if we focused our augmenting efforts just on those four boxes.  While all of our competitors tried to solve for each of the boxes, many of which weren’t generating costs, we focused on where the costs were.

Usually when finding out where the costs are in any workflow, you will also identify the bottlenecks or constraints to the work process.  Eli Goldratt with his Theory of Constraints asserts that in any give work process there are only 2-3 bottlenecks or constraints.  He recommends that the automation work focus on the bottlenecks.

Now that you have gone through the orientation steps of the Macro OODA model for software, it is time to build a prototype solution to test your understanding of what you have observed and oriented.

These prototypes can be as simple as a paper prototype as described in Vijay Kumar’s 101 Design Methods or as complicated as a software prototype.  Fortunately, our tools for building software prototypes have gotten so good that even complicated prototypes can be produced quickly.  Each week that goes by show cases another low code or no code tool for building sophisticated applications.

The Lean Canvas is a paper prototype of a business.

With the first pass of the Orientation loop producing a prototype, it is time to test the prototype with a real user in order to DECIDE what to do next.  The Macro DECISION Agile OODA loop is:

There are four possible actions that you need to decide on to move forward:

    1. Go on to the next step
    2. Recycle within this step
    3. Pivot (take another approach)
    4. Abandon the project

When moving in fast time, the most likely decision is that you need to recycle within this step.  If you have done good work in the Observe and Orient steps, the step 4 of abandoning is unlikely.   If you have recycled several times and don’t seem to be making progress, it is often advisable to go back to the Observe and Orient steps to see if you missed something.

The first step within DECIDE is to commit to doing daily demos rather than agile daily stand up meetings.  The purpose of the daily demo is to SEE the actual progress being made.  It is a combination of understanding what was accomplished the previous day and designing the work for the next day.  The meeting should be rapid fire and last for just 15 minutes.  Once or twice a week, enough problems with the daily design may become evident and a longer stand-up meeting is scheduled to explore in more detail the execution problems with the prototype.  These longer meetings typically use the micro OODA process to go back to basics.

The participants in the daily demo should include the appropriate developers (usually a front end UI developer and a back end data manipulation and storage developer), a senior architect, a product owner or product manager, and a business owner.  An ideal room for a Daily Demo should include a large screen that can be seen by all and a white board for design discussions.  The meetings should be recorded and an audio transcript is generated.  Photos of the white board are also taken and included in the records of the meeting.  The meeting artifacts become the fodder for the weekly and monthly retrospective analysis.

The second step of the DECIDE step is to do a limited release to target users and have them use the prototype in an actual work environment.  The software should be instrumented so that actions the users take can be analyzed.  Video ethnography techniques to record what happens around the software usage are important to capture as well.  The product owner and UX representative check for additional insights through analysis of the telemetry and the video ethnography that are brought to the next day’s daily demo.

A key part of the DECIDE step is evaluating the prototype against the Outcomes and Impact that the product will have for the customer’s business.  An Outcomes focused product development effort is described in a “Product Produces Outcomes.”  As part of the Observe and Orient steps the product team develops a North Start Metric to guide the development.  The North Star Metric should be closely related to the desired outcomes for the product.

After DECIDING what to do next, it is time to ACT and launch a version of your product to a larger customer audience.  The Act Macro OODA loop is:

Now it is time to take ACTION.  As mentioned above, there are relatively few types of decisions to make during the DECIDE step.  “How to decide whether to adopt, adapt or abandon” describes a process for acting on the decisions.

As you progress the first time through the cycle, you are deciding whether it is time to launch an Minimum Viable Product to gather information from a larger sample size of users.

Through telemetry data you quickly identify who is using the product on a regular basis and these users are good candidates to perform more in depth observation.  Opening up your product to a larger audience, particularly in cloud SaaS (software as a service) environment, helps you capture key metrics for product success:

    • How many users invited to participate actually sign up and log in?
    • How regularly do the participants log into the product (ideally multiple times per day)?
    • How much do the participants use the product?

The answers from your telemetry data and your qualitative research should help you answer the key criteria for getting ready to launch your V1 product and your business.  The launch criteria are;

Bringing all the pieces of the macro agile OODA loop together, we have:

The first pass through the macro agile OODA loop gets us to a V1 of a product that becomes the test for a product solution fit as described by Ash Maurya at Leanstack.  The meta agile OODA loop cycles through the macro OODA process as the product evolves from V1 to Vn and the product becomes more of a Whole Product as described by Geoffrey Moore and Philip Kotler.

The Leanstack model identifies the key criteria that it takes to scale from an idea to a product solution fit to a product market fit to a product at scale.  Maurya’s books Running Lean and Scaling Lean describe the journey to scaling.  At Leanstack, Maurya provides tools and coaching services to aid early stage startup teams to navigate the journey to scaling.

For enterprise software products, there is another key element of software product design which Geoff Moore describes as the Whole Product concept to avoid the startup disaster of the chasm.  Moore identified that most startups don’t understand that the earliest customers are not representative of the Early Majority customers.

Moore in his books Crossing the Chasm and Zone to Win illustrates that the Whole Product concept is the method for avoiding the chasm and scaling.

Most enterprise software product development teams only focus on the core and generic product layers.  Yet, in order for a customer to use a product to produce an outcome that will have a business impact the expected and augmented product layers are needed.  These two layers usually involve creating partnerships with other suppliers of related products.  From a technical perspective this means creating application programming interfaces (APIs) within your own product and using APIs from those products you are partnering with.

Throughout this whole process of Agile OODA HCD software product development, you are trying to co-create value for the customer.  The service science researchers have nicely formalized the value co-creation process represented by this diagram from Vargo and Lusch in “Service-dominant logic 2025.”

The diagram provides both a virtuous cycle of value co-creation and a narrative of how the pieces flow together:

“Actors involved in Resource Integration and Service Exchange enabled and constrained by endogenously generated Institutions and Institutional Arrangements  establishing nested and interlocking Service Ecosystems of Actors …”

That is certainly a mouthful of complex concepts from a pair of academic marketing professors.  What is means to me is that in order for any company that produces a product to receive value it must first help co-create value with its customers.

The diagram below summarizes the keys to “Observe, Don’t Ask” in order to reduce the uncertainty of software product development by using the Agile OODA Loop at a micro, macro and meta level in order to reliably and quickly generate successful products.

Observe, Don’t Ask.  Show, Don’t Tell.  Prototype, Don’t Guess.  Act, Don’t Delay.

Posted in Content with Context, Design, Learning, Patterns, Product, User Experience, Wake Up! | Leave a comment

Observe, Don’t Ask. Show, Don’t Tell. Part 2

Day 194 of Self Quarantine             Covid 19 Deaths in U.S.:  201,000

Rest in Peace Justice Ruth Bader Ginsburg                                   VOTE!!

After reading the first post on “Observe, Don’t Ask,” Professor David Socha asserted “this post still begs the question of where do you observe.  Can you create a diagram for where to observe?”

A longtime colleague and collaborator, Eric Robinson shared:

“I like Kelly Franznick’s story.  The implications of that story go all the way back to product architecture and designing for the purpose of observability.  How do you make observability a core component of the product?  What if it had been a client-installed, on premises, monolithic application?  The engineer couldn’t have the quick turn around and put changes in front of the users for the next day’s ethnography cycle.

“And the ‘action logging’ was critical (not just ‘system logging’). The ability for the software engineers to see the problem directly from the logs and prove the existence of the problem is powerful and immediate.  How often have we seen stories come into the product backlog from anecdotal information and have an engineer say ‘that’s not a problem’ and the agile user story gets dropped?  The metric should be ‘how long does it take us to find out the client had a problem?’  The unit of measurement should be seconds or minutes, not days, weeks or months.

“Continuous Integration, Testing and Deployment (CI, CT, CD) are prerequisites for Continuous Observation.  The Daily Demo is a software engineer and product-centered approach — which is probably the best thing you can get quickly without daily access to a real user.  However, these questions are always present:

      • Are we helping the customers (improving their lives, adding value, etc)?
      • How do we know?
      • How do we get the feedback?
      • How soon can we know?

“The opposite of these questions also need asking — did we decrease value, or create a problem, etc?”

After I prodded Professor Socha to tell me more, he went on to explain what he meant about “where to observe?”

“My understanding of waterfall is that is framed around the use of documents (mostly text, sometimes diagrams) to communicate what is important for the people who do the actual work.  Marketing and business analysts would work with customers, business executives, and other stakeholders to figure out what is most important to provide to customers and the business, and then record that in a document. The document is the product of “requirements elicitation”. It seems that business folks often use a slide deck to communicate such things. While there inevitably is some collaboration involved, and some back and forth discussion, the product of this phase is all about “sage on the stage,” broadcast, and “unambiguous” descriptions.

“Part of the frustration that provoked Agile methods is the myth that such documents communicate well. Few people  actually read a hundred page specification document, unless they are embedded in systems like Boeing aircraft design where those documents are taken to a much higher level of rigor and analysis. Bill Erdly talks about how he was involved in using ALL-IN-1 to automate the production of their Boeing proposals so that they could keep making changes up to the last hour before pushing the button and printing a huge document (many thousands of pages) that they would then ship off to DC with one copy on a plane (to get there in time) and the rest on several pallets sent by rail. Their system did all sorts of computation on the data, creating summaries from the raw data, doing cross-checks, etc. It was document automation in the large. And given the regulatory environment and the physics of the domain they were working in, such analyses captured in the documents could represent some goodness.

“This is different when it comes to products that are about delivering adaptive human experiences, the stuff of software-as-a-service etc. Especially different are products that have to scale and adapt to the emergent needs of many varied contexts. In those cases, no document will be correct. But a document represents a hard, reified “truth” that is harder to modify. Or to understand. Which then requires the creation of all of these change-control mechanisms to ensure that any document changes are “correct” and “good”.

“Okay, I’m blabbing about the maladies of documents.  Let me return to your question.

Where to observe?

        • in naturalistic contexts where customers or customers’ customers are doing their thing. That’s the whole decades-old message about taking your software engineers to the field with the customer support team so they can observe and listen to what the actual customer does, and how the customer service representatives address the issues / opportunities / things that come up. But software engineers tend to want to stay in their offices and “be productive”. And there’s always “too much work to do” to justify the day it takes to send them into the field. Besides “it’s too dangerous” sending a software engineer, cause they will say the wrong thing or act “too nerdy” and put off the customer.
        • in the user research lab (vs reading Blink’s report a month later).
        • walking the hallway as a manager to hear the conversations, or to bump into people to have spontaneous conversations (was it Cutler at Microsoft who talked about planning to take hallway walks at just the right time to meet so and so so that he could have such and such a “serendipitous” conversation about an issue at hand?)
        • being in the room with your cross-functional team, both as a team member and as the team manager. To be more creative (see this interview by James Shore about how distributed software development work requires 30% more software developers to get the same level of creativity as when collocated). Software development is a creative industry. Creativity is as important as productivity. Bjorn also highlighted the need for junior developers to be in the same room in order to learn. Remember, Vygotsky notes that all higher-order concepts are social first, so we need to build in the social if we want to get to higher-order concepts. (There, do you feel better now that I’ve mentioned Vygotsky?)
        • when prioritizing features, make sure you can show the current state of a feature, what it actually looks like. That can help the conversation focus on what part of the thing really needs change now, and what parts can be left as they are for the moment.

“Jointly seeing and discussing helps satisficing, aiming for Good Enough. See Bach’s seminal article on A Framework for Good Enough Quality: Beyond the Buzzword (available here along with related articles). That framework generalizes to any context of satisficing; one just has to have the hard conversations about what is critical, etc. Here is the good enough definition from that article:

“To claim that any given thing is Good Enough is to agree with all of the following propositions:

              • It has sufficient benefits.
              • It has no critical problems.
              • The benefits sufficiently outweigh the problems.
              • In the present situation, and all things considered, further improvement would be more harmful than helpful.
        • after doing a change to see what the customer now does with the system. Like you did at Attenex. Cause human behavior is emergent. And customers often don’t do what we expect. It’s theory-of-constraints in the small.
        • Daily demos
        • Scrum and other agile processes that include end-of-iteration demos to team’s stakeholders
        • when needed: “Hey Bob, can you come look at this? Something weird is going on here.”
        • with product owner / business analyst / customer to check that what you have done with respect to a task / feature does what they expect / need / want.
        • with QA person before declaring a bug fixed and throwing it back over the fence to QA; “does this align with what you expected?”
        • when my wife says application X isn’t working, or how can she do Y on her computer, I’ve learned that the first thing to do is ask her to go to her computer and show me what she is doing (and stop talking at me about it). For instance, when she says “I’m using Outlook” does she mean the app running on her computer, or the Outlook running in the web browser.
        • when my student on the other size of the Zoom call says she is doing it correctly, I ask her to share her screen. It’s uncanny how often I’ll find out that she is NOT doing what I asked her to do.

“Okay, gotta go off to do my day’s work. Thanks for the prompt. I’m sure there are lots of more places. But in typing this out, I’m seeing that the need for observe and show is all around. Words just suck. We both think we know what the words mean, but we often (usually) have different meanings. And then you introduce evolving technology where Microsoft Outlook can run in either an app or a browser, or that the version of the Outlook app running on my Mac is completely different from the version running on Caroline’s Mac, even though we both have updated to the “latest” version (perhaps UW Bothell has limited what my Mac views as the “latest”?), and that Mac versions are always different from Windows versions (etc. across any platform) and why should I EVER assume that application X that I see on my system has much to do with what the other person is seeing as application X on their system.

“So, the observe and show is about my day-to-day life at home. It is about teaching, especially now that we are remote and we cannot simply look over the student’s shoulder to see what they see. It’s about working with colleagues to develop a system. It’s about understanding the customer’s experience – cause the experience is a becoming (Ingold‘s term) of their lived experience, their physical materials at hand, their type of device, their version of software, their particular context of use, and the range of “brilliant” to “stupid” ways in which they are trying to do their work. “


After absorbing David’s thoughts on the “where” of user research, I Google searched for more on user observation.  I came across Jim Ross’s “The Role of Observation in User Research.”  He asserts:

“User research consists of two core activities: observing and interviewing. Since we’re most interested in people’s behavior, observing is the most important of these activities because it provides the most accurate information about people, their tasks, and their needs.

“While interviewing is also very important, the information people provide during interviews isn’t always accurate or reliable. Often, research participants don’t know why they do things, what they really need, what they might do in the future, or how a design could be improved. To really understand what people do, you can’t just ask them, you have to observe them.”

Ross goes on to list the types of observation in user research:

      • Usability Testing
      • Contextual Inquiry
      • Naturalistic Observation
      • Shadowing
      • Covert Observation
      • Participant Observation

“A naturalistic observation lets you to see what happens over a longer period of time, whether you’re observing one person or a group of people. You can see how a normal day unfolds without introducing your own interruptions or influencing participants. For example, while you might hear about particular problems during a contextual inquiry, observing participants over a longer period of time provides a better understanding of how often such problems occur and what causes them.”

At the end of Ross article, there is a pointer to David Travis and his book Think Like a UX Researcher:  How to Observe Users, Influence Design, and Shape Business Strategy.

In the first couple of pages, Travis likens UX research to the work of Sherlock Holmes in detecting:

“Opinions are not facts and speculation is not evidence.  Instead, his primary method of collecting facts was careful observation.  ‘You know my method, Watson.  It is founded upon the observation of trifles.'” p. 12

While I absorbed the fullness of the Holmes quote, I thumbed back a few pages to re-read why we do observation – to gain insights.

“The best way of gaining actionable and testable insights is not to ask, but to observe. Your aim is to observe for long enough that you can make a decent guess about what’s going on. Asking direct questions will encourage people to make things up, not tell you what is actually going on.

“There are two ways to observe. We can observe how people solve the problem now. or we can teleport people to a possible future and get them using our solution (a prototype) to see where the issues will arise.

“The key point is: What people say is not as useful as what people do, because people are unreliable witnesses.” p. 4

Travis goes on to provide excellent definitions of the problems of poor user research and the lists of what a user researcher should do.

The Seven Deadly sins of UX:

“The problem isn’t with the quantity of UX research. It’s with the quality: organizations struggle to distinguish good UX research from bad UX research. Here are seven examples of poor UX research practice that we’ve come across in our work with clients—along with some ideas on how to fix them.

• Credulity.

“The dictionary defines credulity as a state of willingness to believe something without proper proof. The form this takes in UX research is asking users what they want (and believing the answer).

The best way of gaining actionable and testable insights is not to ask, but to observe. Your aim is to observe for long enough that you can make a decent guess about what’s going on. Asking direct questions will encourage people to make things up, not tell you what is actually going on. There are two ways to observe. We can observe how people solve the problem now. or we can teleport people to a possible future and get them using our solution (a prototype) to see where the issues will arise. The key point is: What people say is not as useful as what people do, because people are unreliable witnesses.

• Dogmatism.

“Dogmatism is the tendency to lay down principles as undeniably true, without consideration of evidence or the opinions of others. The form this takes in UX research is believing there is one “right” way to do research.

• Bias.

“Bias means a special influence that sways one’s thinking, especially in a way considered to be unfair. UX research is a continual fight against bias. There are a handful of different kinds of bias that matter in UX research, but it’s response bias we want to discuss here. This is caused by the way in which you collect data.

• Obscurantism.

“Obscurantism is the practice of deliberately preventing the full details of something from becoming known. The form this sin takes in UX research is keeping the findings in the head of one person. UX research is often assigned to a single person on a team. That person becomes the spokesperson for user needs, the team’s “expert” on users.

• Laziness.

“Laziness is the state of being unwilling to exert oneself. The form this takes in UX research is in recycling old research data as if it’s boilerplate that can be cut and pasted into a new project. our favorite example of this comes from the world of personas. We find that clients often approach the process of developing personas as a one-time activity.

• Vagueness.

“Vagueness means not clearly or explicitly stated or expressed. In terms of UX research, we see it when a team fails to focus on a single key research question and instead tries to answer several questions at once. This sin is partly caused by the sin of laziness.

• Hubris.

“Last but not least we have Hubris. Hubris means extreme pride or self-confidence. In UX research, it takes the form of taking undue pride in your reports. All UX researchers suffer from this to some extent, but those with PhDs are the worst. And we say that as proud recipients of a PhD.5 UX researchers love data. And when you love something, you want to share it with people. So you create detailed reports packed with graphs and quotations and screenshots and callouts. Look at my data! Look at how beautiful it is!

“Overly detailed reports delay the design process. You don’t need to do extensive analyses in a spreadsheet to find the top problems. That analysis is useful later, when you want to dig into the details, but the critical findings need to be fed back quickly. This is so the design can be modified and so the build-measure-learn cycle can continue. Instead, you need to create information radiators (like usability dashboards and one-page test plans) to get teams understanding the data so they can take action on it. Information radiators are essentially advertising billboards that gradually permeate the team’s awareness of your results. As a general rule, if people need to turn the page, your report is too long. So ask yourself: how can we capture the results in a single glance? This could be a concise visual way of presenting research data, like a user journey map, a persona, or a usability testing results dashboard.

Information Radiator

An information radiator, also known as a Big Visible Chart (BVC), is a large graphical representation of project information kept plainly in sight within an agile development team’s shared workspace.

The term is generic rather than specific: information radiators can include most types of charts used in agile development. Burn down charts, task boards, planning boards and storyboards are among the possibilities. An information radiator is usually hand-drawn or printed but can also include computer-generated charts and electronic displays.

The purpose of information radiators is to help keep the team focused on what really needs their attention and to promote transparency.

Alistair introduced the term “information radiator” in his 2001 book, Agile Software Development.  Martin Fowler is said to have coined the term “Big Visible Chart.”

Once more I feel like I have been asleep.  I have never heard of Information Radiator before.  I love it.

Think Like a Detective

“So what can we learn about doing UX research from the greatest detective of them all—Sherlock Holmes? Holmes was an investigator par excellence, but he was not a super hero (he did not have super powers). Instead, he had well-honed skills and specialist knowledge about a few things. And he was nothing if not methodical. His method comprised these five steps:

        • Understand the problem to be solved.
        • Collect the facts.
        • Develop hypotheses to explain the facts.
        • Eliminate the least likely hypotheses to arrive at the solution.
        • Act on the solution.

“Here are some things we can learn from Holmes’s approach that can help our UX research thinking:

        • Focus on the problem not the solution.
        • Create an explicit research question (actually write it down with a question mark at the end).
        • Don’t start doing any research until you have this question.
        • Don’t assume the question has never been asked before.
        • Find out what your colleagues and your company already knows.
        • Do an archival search—start by reading any prior research reports.
        • Interview team members and stakeholders.
        • Use a checklist to collect background information in a systematic manner.
        • Leave nothing to guesswork.

As I go through these definitions and lists, Travis answers my question of whether to have a hypothesis before hand.  He answers “No.”  However, you need to have a research question.  One good question is better than hundreds of small questions.  He suggests that it is more important to know a lot about a little, rather than a little about a lot.

“You may not get to wear a disguise or crawl about on the carpet with a magnifying glass, but here are some things we can learn from Holmes to improve our observation skills:

        • Watch people actually doing their work—don’t just get a demonstration.
        • Remember that your participants are the experts, you are the “novice.”
        • Focus on the most typical tasks, busiest days, typical days, and critical incidents.
        • Find out what activities precede and follow the task you are observing.
        • Look for inconveniences, delays, and frustrations.
        • Shadow people; follow them wherever they go.
        • Point to things and find out what they are for.
        • Get copies or photos of artifacts, samples, forms, and documents.
        • Make diagrams of the workspace.
        • List the tools people are using.
        • Note people dynamics and interactions.
        • Be alert to things happening simultaneously.
        • Record anything unusual about the scene you are looking at.
        • Ask yourself if anything is missing.
        • Observe behavior at a low level of detail—watch what people touch and what they look at.
        • Pay attention to the sequences and timing of events and actions.
        • Don’t get in the way.
        • Pay attention to trifles.”
    • “Our models, personas, scenarios and stories should include:
        • The primary goals that people have.
        • The workflow of tasks people carry out.
        • The mental models people build.
        • The tools people use.
        • The environments people work in.
        • The terminology people use to describe what they do.”

“Fundamentally, all UX research answers one of two questions:

(a) “Who are our users and what are they trying to do?

(b) “Can people use the thing we’ve designed to solve their problem?”

You answer the first question with a field visit and you answer the second question with a usability test.

I think you get the idea.  Just paying attention to these bullet points helps get to Outcomes and behavior changes that are needed to support Value Co-Creation.

David Socha then followed up with these thoughts and questions:

“Your notes point to the WHERE of user research, but do not extend to the need to use observe, don’t ask and show don’t tell in other aspects of the software development process. Such as in the daily demos, or when deciding if a work item is complete: after a developer makes a change, have the developer sit a customer / product owner / business analyst / etc. in front of the computer with the new system running (the system with the change) and ask that “customer” to give the system a try. Don’t tell them how the new thing works. Don’t walk them through it. Watch them using the new system. See where they do what you expect and, even more importantly, where they do something you had not expected. Ask them how well that works for them. Be curious. It’s not about whether the thing “works” or the bug is “fixed” – it is about whether the new system is the right new system. It may be that you fixed the bug, but the way the system now works actually isn’t what is needed … for whatever reason.

“In other words, I think that the observe+show mindset should be considered and embraced across many more aspects of the software development process. I don’t have a good sense or visualization of what that means yet. I still need to figure that out. My feeling is that it is something about creativity: anywhere there is a need for creativity, or perhaps any discussion about user experience (which can only be understood by experiencing it), one should try to avoid trying to use words to describe the thing. Something like that.”

Observe, Don’t Ask.  Show, Don’t Tell.  Prototype, Don’t Guess.  Act, Don’t Delay.

Posted in Content with Context, Design, Learning, Patterns, Product, User Experience, Wake Up! | Leave a comment

Observe, Don’t Ask. Show, Don’t Tell.

Day 190 of Self Quarantine             Covid 19 Deaths in U.S.:  200,000

Rest in Peace Justice Ruth Bader Ginsburg                                   VOTE!!

Ever since I became acquainted with the human centered design methods of the Institute of Design at Illinois Tech, I preach the gospel of the importance of user observation.  I shared my first exposure to these methods in Observing Users for Software Development.

Then in 1992 while visiting the Institute of Design of the Illinois Institute of Technology, my views on design were transformed by a five minute video from a student class project.  This video was my first introduction to the power of user observation.  Sitting in a miserable concrete walled classroom on the 13th floor of a non-descript research building over looking some of the worst slums of South Chicago, I could barely hear the nervous student introducing his project.  It had something to do with improving the ability of the business traveler to work in a hotel room.  As someone who usually travels 150,000 air miles a year and spends >50 nights per year in hotel rooms, he had my attention, if not my expectation that he could shed any light on a frustrating environment.

The student created a relatively simple task for a male and female pair of business colleagues.  The pair had to create a business report in a hotel room, and then type the results into their laptop PC.  In the process they had to confer with other employees over the phone to get information for the report.  The student would videotape their activities in the hotel room for later analysis.  The first several minutes of the videotape showed the awkward dance of the professional colleagues trying to find a work surface that would accommodate their needs, while avoiding the cultural taboos associated with the only work surface available – the bed.

The pair searched in vain for something that would work and yet the bed remained the only place that is large enough, was convenient to the phone, the power outlets and the available light.  The pair finally concludes that the bed is the only viable place and they start to lay their papers and computers on the bed.  They then realize that there is no comfortable place to sit.  The single chair in the room is too high for the bed surface.  Yet, it hurts to kneel on the floor and it is awkward to sit on the bed without disturbing the papers and computers.  Throughout all of this trial and error, the male and female are trying not to invade each other’s personal space so that they don’t cross the line into intimacy.

After five minutes of trying to work, the pair throws their hands up and quits the exercise.  They cannot get work done in that environment.  I was amazed at how completely the five minute video transformed my experience as a business traveler from unnamed frustration with a hotel room as a work environment to being able to clearly articulate my frustrations.  And in that moment, a solution space opened up for tens of ways to transform the business traveler’s hotel working experience.  No interviews were needed.  No audio was even present on the videotape.  Just watching the interactions said it all.  The student also showed some interviews with business travelers that provided no insights on either the problems or the solutions as a counterpoint to the power of user observation.  Even though we might be experienced business travelers, we are not usually conscious about what bothers us to be specific about the problems.

This short experience of video ethnography led me to the mantra of “Observe, Don’t Ask!

While mentoring a Brandon Fleming, CEO of Chimerocyte, a biotech startup, I encouraged him to use video ethnography while he observed how the lab test that he is commercializing is performed today.  He did the video work and is starting to analyze the video through an extension of the AEIOU method of qualitative observational research.

In our coaching session, Brandon asked “now that I have this video, how do I translate it into a good user story?  How do I write a powerful agile user story?”  In my “bad Skip” coaching style I immediately started yelling and swearing.  Not at Brandon, but at the realization that I had never talked about or written about the next step after video ethnography “showing the video to the software engineers.”  I thought it was obvious.

I need another mantra “Show, Don’t Tell!

While collaborating on “Wide-Field Ethnography: Studying Software Engineering in 2025 and Beyond” for the 38th International Conference on Software Engineering “Visions of 2025 and Beyond,” Kelly Franznick shared a recent story about one of their clients acting on information from the user observation research in real time.

I asked Kelly how many research clients were watching the observation streams live versus coming back to watch the videos at a later “more convenient” time.

Kelly shared that they are surprised that over 80% of the use of Feedback Panel is for real time viewing and there is little use of the stored videos.  They find that many of their clients are doing lab studies at BlinkUX in their observation conference rooms. Their clients facilities don’t have the ease of being able to watch a session as a shared group and collaborate around a white board simultaneously.

He shared an example.

“One of our clients is a software company. When we have a subject using their online Software as a Service (SaaS) product, the key members of the development team (product manager, marketing manager, development manager, and lead software architect) come into one of our conference rooms. During a recent sessions, the team watched the subject encounter a serious problem with their software.

“The team almost in unison exclaimed that they had never seen the problem before. They all realized that it was a serious issue. The developers were asked if they had any evidence of how big an issue this problem was for users. The developers acknowledged that they had no instrumentation (log file captures of the problem) in their application.

“So the lead developer immediately went online into their code and installed some instructions to log whenever the problem occurred and alert him to the problem. Within minutes the alerts started flowing to the developer. After the software ran for a few hours on the live system which had thousands of users accessing the application, they found that 10-15% of users were encountering this problem.

“While users were still being studied in the BlinkUX lab, the development team designed a solution for the problem. The software engineer then implemented the solution and called back to the rest of the team to test the solution. That evening the software changes were made to the operational system.

“When the team came in the next day, the next set of subjects were using the just changed software and with a big sigh of relief the team realized that they had indeed fixed the problem.

“By spending the time to watch the user research in real time, the developers could see the problem at least 30 days earlier than the report that BlinkUX would provide as their final research artifact.

“From a business standpoint, this was a big deal as the problem was in the initial landing page access to their product and they were finding that 10-20% of users were abandoning their investigation into the product. They were losing potentially 20% of their revenue from a problem they didn’t even know existed.”

Kelly’s example illustrates the second mantra “Show, Don’t Tell!

As I talked abut the above example with Brandon, I explained that instead of trying to explain very poorly in writing in a natural language like English the precision you need for a software application, you should show the software engineer what is needed.  You need to show the software engineer the video that you just collected or the subsets of the video that have the insights that you identify.  Natural language is a poor communicator of your insights, but the video both shows needs and opportunities and allows you to re-view the user experience many times.

The most powerful form of communication would be for all the software engineers and product managers to observe real users in their natural work environment but that is usually very difficult, expensive and intrusive.  So video ethnography is the best choice.  Where video ethnography is not possible like in observing lawyers and not wanting to break privacy and client confidentiality or in health care patient settings, then action oriented research where the software engineers do similar work is the next best method.


“So how does this relate to the Daily Demo (versus the Agile Daily Stand-up) that you advocate we use?” Brandon asked.

“It’s the same thing.  The purpose of the Daily Demo is to SHOW your previous day’s work instead of giving a status of green light, yellow light, red light of an agile daily stand-up,” I impolitely bellowed.  Again, I realized that I have never written about the linkage between “Observe, Don’t Ask and Show, Don’t Tell.”

I went on to describe that this is part of the virtuous cycle of observing the need during user research, designing and building a prototype utilizing the daily demo, and then using video ethnography when doing user research with the prototype in the user’s actual workplace.  Then you bring the video ethnography back to the daily demo to help guide the design using your North Star Metric or better yet have the software engineers observe in real time as Kelly Franznick suggested above.

Observe, Don’t Ask.  Show, Don’t Tell.

I discovered the power of the daily demo out of desperation (my usual form of innovation motivation) while I was Chief Product Officer at FTI Consulting.  We were spending a lot of time doing things right in the continuous integration (CI), continuous testing (CT), and continuous deployment (CD) processes of a modern DEVOPS and Agile shop.  Yet, it was clear that we weren’t doing the right things when it came to winning new eDiscovery business or getting customers to do “continuous adoption (CA)” of the features we were building.  We were a feature factory, not a customer outcomes provider.

I started an experiment with one of our ten agile teams to move to a Daily Demo approach to development rather than the standard agile daily stand-up.

At a daily stand-up, the task board is displayed and each person talks about their tasks for the next day (green light, yellow light, red light).  It is more administrative than substantive.

A daily demo is about demonstrating the work from the previous day.  One person demonstrates the current work on the large display.  One or more people take notes.  Somebody drives the discussion. Then a conversation occurs about what seems to be working, what questions arise about how the work evolves, what work then needs to be done (UI and back-end database) for the next day.  What is really helpful for me is that by having a separation of work in two or more software engineers I get a much better feel for the data issues and then the UI, workflow, and use case issues.

The ideal participants for a daily demo are the key software engineers (at least one front end UI developer and one back-end data storage and network “micros” services engineer), a software architect, a product manager (product owner), and a business owner (someone at an executive level who is regularly involved in sales activities).

A daily demo is about a conversation, not administrative status reporting.

A daily demo is group experiential learning.  We are learning together around a customer need and a business goal.  The experiential learning often spans multiple logic levels of understanding.

A daily demo is about designing together in the context of your North Star Metric.

A daily demo is about seeing if we all understand in the same way a user’s real needs.

A daily demo should be about 15 minutes evenly divided between showing the previous days work and designing the next days work.

By having me in the room we also had a different set of expertise from a Product Owner.  A product owner typically acts as the Subject Matter Expert (SME) and should have regular contact with real users.

What I brought to the daily demo is the combination of roles:  Business owner (how do we monetize this and thinking through which features will appeal to influencers, purchasers and users), Product Manager, SME, and UX practitioner.  My twenty years of legal customer experience brought forward usage contexts not just from eDiscovery in the FTI example but also from the perspective of forensic investigations, Early Case Assessment, contract intelligence, intellectual property intelligence, and professional services project management (BCG, McKinsey, other FTI Segments).

We recorded all of our daily demo sessions so that we could put the audio, video and transcripts into our semantic network analytics tool (FTI Ringtail, now NUIX Discover).  These recordings and analytics aided us to prepare for our weekly retrospectives as well as providing context and training examples for our product marketing group, our professional services groups, and our early adopter customers.  They could see and hear how their needs were injected into the software product design and development.  Show, not just tell.

Brandon’s two questions helped me see that the implications of using video ethnography and their potential impact are not at all obvious.

I reached out to Professor David Socha, a long time collaborator, who teaches software development management and evidence based design at the University of Washington at Bothell to see if he connected the dots of video ethnography replacing the need for user stories and the “show, not tell” of the daily demo replacing the administrative daily stand-up.  He was as stunned as I was that he had not made these connections before.

As we talked we realized the extent of the virtuous cycle of Observe AND Show that starts with observing user needs and showing that continues throughout the software development life cycle and the value co-creation frameworks of Service Science.  This virtuous cycle goes from idea to Minimum Viable Product (MVP) to launch of the V1 product to Product Market Fit and beyond as Ash Maurya describes in Scaling Lean.

Agile processes were invented and evolved to deal with this language problem of natural languages that are not precise enough and don’t convey enough to the software engineer to design what a customer actually needs.

Observe, Don’t Ask.  Show, Don’t Tell.

Of course, my colleague David Robinson would just laugh and say “Great, Skip.  You once again discovered my mantra of ‘Experience First, Make Meaning Second.’  How many times are you going to have to learn this lesson?”

Thanks Mr. Robinson.  Evidently I am going to continually learn this lesson.


Observe, Don’t Ask.  Show, Don’t Tell.  Prototype, Don’t Guess.  Act, Don’t Delay.

Posted in Content with Context, Design, Learning, Patterns, Product, User Experience, Wake Up! | Leave a comment

Wait what? A labyrinth? Near my trail?

Day 180 of Self Quarantine             Covid 19 Deaths in U.S.:  190,000

The habits of each day being another Ground Hog day make me wonder if I am sleep walking through life.  “Wake Up!” was my mantra for a while pre-pandemic when I wasn’t paying attention to the world around me.  Time to go back to waking up.

My lovely bride came back from her daily walk with pictures of a labyrinth on Hall’s Hill.  “Wait what?” I asked.

Sure enough there was a labyrinth to the left of where I ordinarily turn right on my walk.  With the surrounding gardens the labyrinth is a peaceful oasis.

I’ve been reading Robin Wall Kimmerer’s Braiding Sweetgrass.

“Purple and yellow are a reciprocal pair.

“Our eyes are so sensitive to these wavelengths that the cones can get oversaturated and the stimulus pours over onto the other cells. A printmaker I know showed me that if you stare for a long time at a block of yellow and then shift your gaze to a white sheet of paper, you will see it, for a moment, as violet. This phenomenon—the colored afterimage— occurs because there is energetic reciprocity between purple and yellow pigments, which goldenrod and asters knew well before we did.

“If my adviser was correct, the visual effect that so delights a human like me may be irrelevant to the flowers. The real beholder whose eye they hope to catch is a bee bent on pollination. Bees perceive many flowers differently than humans do due to their perception of additional spectra such as ultraviolet radiation. As it turns out, though, goldenrod and asters appear very similarly to bee eyes and human eyes. We both think they’re beautiful. Their striking contrast when they grow together makes them the most attractive target in the whole meadow, a beacon for bees. Growing together, both receive more pollinator visits than they would if they were growing alone. It’s a testable hypothesis; it’s a question of science, a question of art, and a question of beauty.

Kimmerer, Robin Wall. Braiding Sweetgrass . Milkweed Editions. Kindle Edition.

As I walk back home, I encounter the paired purple and yellow wild flowers:

As I continue on, I walk by a pond.  Yesterday, it was just a pond.

Today I see the pond through the eyes of Kimmerer as I read her chapter on cleaning out their pond so they can go swimming:

“Being a good mother meant fixing the pond for my kids. A highly productive food chain might be good for frogs and herons, but not for swimming. The best swimming lakes are not eutrophic, but cold, clear, and oligotrophic, or poor in nutrients. I carried my small solo canoe up to the pond to serve as a floating platform for algae removal. I envisioned scooping up the algae with a long-handled rake, filling the canoe as if it was a garbage scow, emptying it on the shore, and then going for a nice swim. But only the swimming part worked out—and it wasn’t nice. As I tried to skim the algae, I discovered that they hung like sheer green curtains through the water. If you reach far out of a light canoe and try to lift a heavy mat of algae at the end of a rake, physics dictates that swimming will occur.”

Kimmerer, Robin Wall. Braiding Sweetgrass . Milkweed Editions. Kindle Edition.

I continue on my path excited to see nature with different eyes.

As I climb a short steep section I remember to look up and see the old car abandoned in the woods.

One of these days I will make up a story of how this car ended up here.

I stop by the Black Lives Matter shrine and see another gift of beauty.

The dreariness of this repetitive walk falls away as I am energized by the beauty and caring of my fellow path walkers.  As I turn a corner on my way home, I see an old man looking at me from another tree.  I’ve walked by this tree for 20 years and never seen the “art.”

As I walk through the labyrinth of life, I am indebted to all those who add beauty to my “ground hog days.”

Posted in Biodynamic, Design, Exercise, Lifelet, Wake Up! | Leave a comment

WUKID Extended

Day 177 of Self Quarantine             Covid 19 Deaths in U.S.:  188,000

Since I met Russ Ackoff thirty years ago and learned about WUKID, I strive to advance from data to wisdom in my learning journeys.  Russ gave me the practical definitions of the stages of learning:

    • Data – the raw stuff of the world. It could be a temperature reading 67 degrees or the price of a book or any of the raw things that we encounter each day.
    • Information – provides structure for data. A weather report puts the temperature (data) in context. The outside air temperature in Seattle, WA on July 10, 2007 was 67 degrees at 2 PM and the sun is shining. Each of the components of the previous sentence is data put together to form a glob of information.
    • Knowledge – is actionable information. Given the above weather information string I would know that it is going to be a nice day but cool for that time of year so I would carry a light sweater or jacket if I were to go outside.
    • Understanding – is seeing patterns in knowledge and information. If the above weather string were combined with 20-30 days of similar strings of information and I had lived in Seattle for 10 or more years, I would be able to see a pattern of it being a cool summer. Understanding has a longer time component than information and knowledge. Understanding incorporates double loop learning as described in Schon’s The Reflective Practitioner.
    • Wisdom – is going beyond the levels of understanding to see larger scale systems and be able to predict behaviors and outcomes in the longer term future (5-15 years) based on seeing the patterns that arise through understanding. When lots of data over many years was refined into information, knowledge and understanding patterns, scientists were able to see long term weather patterns like el nino and la nina. Based on these patterns weather forecasters can predict longer term trends in Seattle and I can act accordingly.

I loved it when the Gapingvoid artists drew their image of WUKID and then added Impact:

What I love about these images is the text they provide to share the context behind the image:

“Semiotics is the study of the signs and symbols that imbue meaning and meaningful communication. It includes the study of signs and sign processes, indication, designation, likeness, analogy, allegory, metonymy, metaphor, symbolism, signification, and communication.

“Semiotics has been recognized as having important anthropological and sociological implications by many of the greatest innovators behind the respective disciplines. Umberto Eco proposed that every cultural phenomenon may be studied as communication; Clifford Geertz, arguably the 20th Century’s most respected anthropologist, once wrote that ‘culture is the culmination of symbols that transmit behavior’.

“It is with these thoughts in mind that Gapingvoid started to study how to impact organizational outcomes by informing culture, mindset, beliefs, and behaviors through semiotic tools.

“We have consistently witnessed and measured how people’s perceptions can be shifted through meaning attached to objects. Back in 2005, we popularized the term ‘social object’ with semiotics in mind: the idea that objects spread ideas faster and more effectively than nearly any other medium We see daily examples in internet memes and Red Baseball Caps, which have come to represent an entire political ideology.”

I’ve been fascinated with the study of conspiracy theories since Mark Ackerman of the University of Michigan introduced me to the Post Truth Economy and conspiracy theory.  As we talked, Mark introduced the notion of micro conspiracy theory.  Mark was studying the interaction of the Post Truth Economy and micro conspiracy theory.  I asked for an example.  He suggested I recall a time when I or others might have shared something we knew not to be true about another business colleague.  He explained that those examples are micro conspiracy theories.  I was depressed thinking about the number of times I propagated a micro conspiracy theory or accepted one.

While on the way to somewhere else on an Internet search, I came across this doctored gapingvoid image posted on Imgur that adds conspiracy theory to my beloved framework of WUKID.

After I stopped laughing, I realized that in today’s Post Truth Economy I will have to update WUKID to WUcKID.  I placed conspiracy theory between knowledge and understanding because there is an evil genius in those who intentionally perpetrate conspiracy theories.  I can’t place conspiracy theory as a full fledged participant in the journey to wisdom and thus the lower case “c.”   Unfortunately in today’s world conspiracy theory has its place.

Posted in Content with Context, Flipped Perspective, Learning, WUKID | Leave a comment

The Map is Not the Territory. Or is it?

Day 165 of Self Quarantine             Covid 19 Deaths in U.S.:  177,000

One of the profound early influences on my thinking was the statement in Alfred Korzybski‘s Science and Sanity “The Map is not the territory.”

A few years later, I came across David Gelernter‘s Mirror Worlds or The Day Software Puts the Universe into a Shoebox … How It Will Happen and What It Will Mean.  The title is the synopsis of the book.

Several years ago, I was reminded of “the map is not the territory” comment as I prepared for facilitating a working session of the GE Power Business Unit by reading several documents on “digital twins.”  The one I enjoyed the most was from Wired’s Kevin Kelly talking about augmented reality (AR) in “AR Will Spark the Next Big Tech Platform – Call it Mirror World.”

“General Electric, one of the world’s largest companies, manufactures hugely complex machines that can kill people if they fail: electric power generators, nuclear submarine reactors, refinery control systems, jet turbines. To design, build, and operate these vast contraptions, GE borrowed NASA’s trick: It started creating a digital twin of each machine. Jet turbine serial number E174, for example, could have a corresponding E174 doppelgänger. Each of its parts can be spatially represented in three dimensions and arranged in its corresponding virtual location. In the near future, such digital twins could essentially become dynamic digital simulations of the engine. But this full-size, 3D digital twin is more than a spreadsheet. Embodied with volume, size, and texture, it acts like an avatar.

In 2016, GE recast itself as a “digital industrial company,” which it defines as “the merging of the physical and digital worlds.” Which is another way of saying it is building the mirrorworld. Digital twins already have improved the reliability of industrial processes that use GE’s machines, like refining oil or manufacturing appliances.

Microsoft, for its part, has expanded the notion of digital twins from objects to whole systems. The company is using AI “to build an immersive virtual replica of what is happening across the entire factory floor.” What better way to troubleshoot a giant six-axis robotic mill than by overlaying the machine with its same-sized virtual twin, visible with AR gear? The repair technician sees the virtual ghost shimmer over the real. She studies the virtual overlay to see the likely faulty parts highlighted on the actual parts. An expert back at HQ can share the repair technician’s views in AR and guide her hands as she works on the real parts.

“Eventually, everything will have a digital twin. This is happening faster than you may think.”

Everything will have a Digital Twin – including you and me.  Sean Buchanan, CEO of Visom Technology, reminds me all the time that they are leaders in creating digital twins from MRI and CT scan data.  My medical digital twin will be a layered 4D model from my genomics to the ecosystem that I live in.  It will be 4D as we will have data over time.  How do my MRIs and CAT scans change over time?  How do my Xrays change over time?  How do my lab tests change over time? How do my chronological age and biological age change over time as I follow scientific wellness protocols?

Mentioned in Kelly’s article is Jorge Luis Borges description of trying to create a map in the physical world that approaches 1:1.  I can remember the fun growing up visiting the 1964 NY Worlds Fair standing on a map of NY State.  I stood in the exact spot on the map that I was standing at the World’s Fair territory.  The experience exemplified the counter example to Korzybski’s “the map is not the territory.”

Lewis Carrol wrote:

“What a useful thing a pocket-map is!” I remarked.

“That’s another thing we’ve learned from your Nation,” said Mein Herr, “map-making. But we’ve carried it much further than you. What do you consider the largest map that would be really useful?“

“About six inches to the mile.“

“Only six inches!” exclaimed Mein Herr. “We very soon got to six yards to the mile. Then we tried a hundred yards to the mile. And then came the grandest idea of all! We actually made a map of the country, on the scale of a mile to the mile!“

“Have you used it much?” I enquired.

“It has never been spread out, yet,” said Mein Herr: “the farmers objected: they said it would cover the whole country, and shut out the sunlight ! So we now use the country itself, as its own map, and I assure you it does nearly as well.”

from Lewis CarrollSylvie and Bruno Concluded, Chapter XI, London, 1895

Philip Evans of Boston Consulting Group (BCG) fame compared what is happening now with digital transformation to what Borges described as a 1:1 mapping of the world.

. . . In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it.

“On Exactitude in Science” Jorge Luis Borges

Evans goes on to describe how health care is quickly showing that “the map is the territory” through the use of big data.

“HEALTH CARE is a prime example of the transformational power of these new business architectures. This huge and dysfunctional industry is at the beginning of a transformation. The cost of sequencing a human genome in 2001 was $100 million, and mapping just one (James Watson’s) took nearly ten years. Today it costs less than $1,000. In two or three years, it will cost $100, and sequencing will take just 20 minutes. The number of sequences has grown as the cost has fallen: the Million Human Genomes Project is up and running—in Beijing. Gene mapping is shifting from an abstract research activity to a clinical one, in which a doctor customizes treatment to the patient’s unique genomic makeup.

“The pattern is clear: big-data techniques will be used to spot fine-grained correlations in a patient’s genomic data, medical history, symptoms, protocols, and outcomes, as well as real-time data from body sensors. Medicine will advance by decoding immense, linked, cheap, noisy data sets instead of the small, siloed, expensive, clean, and proprietary data sets now generated independently by hospital records, clinical trials, and laboratory experiments. These databases will make it possible for practitioners and even groups of patients to become researchers and for breakthroughs to be quickly shared around the world.”

Gregory Bateson describes the problem in a different way in his cybernetic book Mind and Nature:  A Necessary Unity.  He relates the challenge of video capturing the reality of the world:

“Of course, the whole of the mind could not be reported in a part of the mind.  This follows logically from the relationship between part and whole.  The television screen does not give you total coverage or report of the events which occur in the whole televisions process; and this not merely because the viewers would not be interested in such a report, but because to report on any extra part of the total process would require extra circuitry.  But to report on the events in this extra circuitry would require a still further addition of more circuitry, and so on.  Each additional step toward increased consciousness will take the system farther from total consciousness.  To add a report on events in a given part of the machine will actually decrease the percentage of total events reported.”  P.432

The other article (Fortnite is the Future, but Probably Not for the Reasons that you Think) that sparked my interest was on the metaverse which is being developed by the game company that makes Fortnite and the underlying game engine Unreal.  Fortnite could be the platform to host a world of health care Digital Twins.  The revenues from Fortnite are staggering even though it is a “free” game.  I was oblivious to their revenue model.  One of its powers is the built in collaboration.  I imagine the potential for very different coaching visits as a combination of digital twins and a “metaverse” user interface.

It is the “metaverse” stemming from Neal Stephenson‘s Snow Crash and added to by Ernest Cline’s Ready Player One that provide a model for how future health care interactions might evolve.

The term “Metaverse” stems from Neal Stephenson’s 1992 novel Snow Crash, and describes a collective virtual shared space that’s created by the convergence of virtually enhanced physical reality and persistent virtual space. In its fullest form, the Metaverse experience would span most, if not all virtual words, be foundational to real-world AR experiences and interactions, and would serve as an equivalent “digital” reality where all “physical” humans would simultaneously co-exist. It is an evolution of the Internet. More commonly, the Metaverse is understood to resemble the world describe by Ernest Cline’s Ready Player One (brought to film by Steven Spielberg in 2018).

Of course, early versions of the Metaverse will be far simpler – but the foundational elements will go well beyond “gaming”. Specifically, we’d see in-game economies (e.g. trading, bartering and buying items) become more of an industry where humans will literally “work”.

“If you look at why people are paid to do things, it’s because they’re creating a good or delivering a service that’s valuable to somebody,” Sweeney told Venturebeat in 2017. “There’s just as much potential for that in these virtual environments as there is in the real world. If, by playing a game or doing something in a virtual world, you’re making someone else’s life better, then you can be paid for that.”

To this end, a crucial difference between a vibrant game, including Fortnite, and the Metaverse is that the latter “should not simply be a means for the developer to suck money out of the users. It should be a bi-directional thing where users participate. Some pay, some sell, some buy, and there’s a real economy….in which everybody can be rewarded for participating in many different ways”, to further quote Sweeney (A semblance of this has existed for more than twenty years in so-called “Gold Farming”, where players, often employed by a larger company and typically in lower-income countries, would spend a work-day collecting digital resources for sale inside or outside of a game).

To Sweeney, the Metaverse represents the “next version” of the Internet – a matter of when, not if.

These articles remind me that I need to keep widening my lens for what is possible and how it applies to 21st Century Scientific Wellness or starting a new company.  The article “Magic Mirror: The Novel as a Software Development Platform” points out how science fiction often leads to software games which then lead to powerful applications.  Ender’s Game was an inspiration for Autodesk’s Ted Nelson who created hypertext and led Autodesk to create Autocad (which is one of the critical tools for XR).

While the above platforms (Unreal Engine, XR, Digital Twin) may not be available this year, I look to these efforts to inform the kind of architecture and platform we need to be designing to anticipate the future.  The Fortnite model also may give us some clues on how to have a very different revenue model.

This week I was reminded of “the map is not the territory” once again as I coached a non-technical CEO on the importance of “expect what you inspect” when it comes to managing software product development.  With all the different tools we have for “managing” like Zoom, email, Slack, Jira, project management GANTT charts, and project boards, it is easy to be distracted into thinking you know what is being built by software engineers.

The only way to really understand what is being built is to “Expect what you inspect.”  That is, you have to constantly see, demo and use the software your team is building.  Otherwise, all of your “productivity” tools will be the map that is not the territory.

I include at the start of each blog post the statistics about Covid.  These statistics are not the territory of this pandemic.  The politics and disinformation about Covid are not the territory.  The science to date is still only a partial map of the Covid pandemic.  We have so much more to learn in order for our understanding of Covid to be more about the territory.

Keep learning and keep contributing facts to the map of Covid.

Remember to vote those politicians out of office who keep spreading false maps.


Posted in Big Data, Content with Context, Design, Flipped Perspective | Leave a comment

Thoughts on User Research through observation

Day 164 of Self Quarantine             Covid 19 Deaths in U.S.:  177,000

User research with an emphasis on observing users in their natural living and working environments is a key foundation for software product development.  In “Observing Users for Software Development” I shared my early introduction to the power of observing users along with several examples of professional design firms user observation leading to innovations.

Recently, in mentoring several CEOs in early stage software product development, I am spending a lot of time teaching, coaching, and encouraging them to do a lot of user observation.  Doing user observation is difficult in normal times, but especially difficult in the age of the pandemic.  There is only a certain amount of wide-field ethnography that you can do over Zoom with a camera controlled by the user.  I find that entrepreneurs are reverting to doing interviews with users rather than observing.

User research for product building is a bit different than interviewing for business and market research.  With human centered design user research or customer experience research a different process is needed.  One needs to switch from interviewing (talking) to observing the users, purchasers, and influencers in the wild.  The key things that you are looking to observe are the overall workflow that your potential product exists in, the skills and behaviors of the users, and which parts of the workflow matter.

At Attenex, we had the privilege of being co-located with a sophisticated and high volume eDiscovery law firm.  We spent a lot of time observing and doing action oriented research where the software engineers spent a day a month actually doing eDiscovery.  We came out with a workflow that looked like the EDRM model:

We realized that all the costs were in the four boxes inside the Legal Cost on the right above.  Our competitors were spending their time building components for all of the boxes.  We focused on the four boxes where all the costs were going and reduced the costs by a factor of 10 times.  That bought us 8 years of competitive advantage AND 8 years of value based pricing while everyone else was stuck at commodity pricing.

Because we couldn’t use video ethnography due to confidentiality and legal privilege, we had to use pure observation and lots of note taking.  While not my preferred way, it worked.  The action oriented research on real matters (parallel to actual eDiscovery professionals) also helped the software engineers and UX researchers to figure out how to keep improving productivity.

Yet, the only way I’ve found to “teach” observation is through a lot of practice and mentoring/coaching.  The Institute of Design does a reasonable job of teaching and Blinkux does an excellent job with professional projects for observing users.

The problem with interviewing when it comes to user research is that folks Make Stuff Up (MSU).  People don’t want to appear stupid so they MSU.  Worse, most people have no idea what they really do tacitly.  Yet, with observing folks in their natural work habitat (not conference rooms) they are incredibly articulate.  All of the major insights I have made over 50 years came from observing folks in the wild.


In his book on Outcomes and Lean UX, Josh Seiden provides the above model.  So what a good user researcher does is observe all of the above.  The user researcher wants to see what resources the user has (not just in the computer but what is on their desk, who they call on the phone or slack or email or who is around them in the office that they talk to and are collocated with).  What activities lead to what work outputs that lead to what outcomes that have what business impact.  And then ideally do the same for the customer’s customer.  How is your product idea going to work for your customer to produce impact and business results for their customers.

I particularly like simple to remember frameworks like AEIOU.

The bottom axis in the diagram is what observing users is all about – seeing the existing implicit.  Steelcase described this research in their article “Shaping Order from Chaos.”

Another way to look at this process is through the virtuous cycle of user observation (video ethnography where possible so you can look and relook at something 10s to 100s of times), insight generation, then rapid prototyping (including paper prototyping) while observing usage and then repeat.  To get to our V1 of Attenex Patterns we went through this loop at least 100 times including full blown prototypes that we put in users hands.  By Version 4 of Attenex Patterns we’d been through over 350 working software prototypes.  I periodically give a history talk that walks through many of the prototypes and how we got to >10x productivity.  Attenex First Year shows several of the prototypes.

While the professional human centered design firms have a lot of different PhD types doing the video ethnography and user research, I was pleasantly surprised how easy it is to learn in my own work.

The good news 25 years after I started working with the Institute of Design is that there is a growing literature from the ID professors on human centered design methods.

The ID literature I recommend is:

Now that Clive Dilnot and Suzan Boztepe (former ID student of mine) have published Heskett’s unpublished works, I will be publishing my extensions of his work.  John as an economist by training that came over to design through industrial design.  I loved sitting in on his classes when I was teaching at ID and having long lunch conversations with John about design and economics.

Explaining Value Co-Creation Theory is fodder for another blog post or two.

Posted in Attenex Patterns, Content with Context, Design, Entrepreneuring, Outcome, Reflecting, User Experience | Leave a comment