gerrymcgovern - NEW THINKING: Testing a COVID-19 classification

Design Politics: An online conference for change makers, June 16, 17 & 18 2020. I’ll be speaking at this about designing an information architecture for COVID-19
https://live.thisishcd.com/

Join Gerry McGovern, WHO, and a worldwide team in discussing the results from Round 2 testing of the WHO COVID-19 top level classification. We will then prepare for the Round 3—and hopefully final—test. Thursday, 21, 3 – 4.30 PM GMT.
https://attendee.gotowebinar.com/rt/3078570292532611088

WEB LINK FOR THIS ISSUE: https://gerrymcgovern.com/testing-a-covid-19-classification/

TESTING A COVID-19 CLASSIFICATION

Over 800 healthcare providers, academics, and members of the public sorted COVID-19 top tasks into groups. We analyzed the groups and came up with the following hypothetical classification:

Symptoms, Diagnosis, Spread
Mental & Physical Wellbeing
WHO, Government Guidance, Education, Training
Research, Statistics
Vaccine, Immunity, Treatment
Avoiding Infection
News
End Date, New Outbreaks

What is important to understand here is that this is a hypothesis. The results from a sort rarely give definitive data. Lots of interpretation is required. Through experience we have found that we need to test over three rounds in order to design a robust classification.

How do we measure success? By giving people instructions to see how they would use the classification to solve top tasks such as:
• When is a vaccine likely to be available?
• Find the latest updates for COVID-19.
• Can you get infected by COVID-19 through the air?
• If someone has had COVID-19 are they safe from getting it again?
• Find what the WHO recommends in relation to testing.

We recommend having a minimum of 20 instructions and not more than 35. We need at least 20 instructions because otherwise the classification will not get a robust testing. It’s easy to get high success rates if we only test a few tasks. Using lots of instructions allows the classification to be tested from multiple angles.

Over 2,000 people completed the first round. (Normally, we need about 50 for reliable results.) We had an overall success rate of 60%. In other words, the classes we expected to be selected for the instructions were selected 60% of the time. Typically, we find success rates between 40% and 60% for round one, so our hypothesis was working well.

Our target is a minimum of 80% and ideally 90%. It is impossible to achieve a 100% success rate for a complex classification. There will always be around 10% of people or tasks that behave in unusual and unpredictable ways.

To get it over 80% we need to make some adjustments. The classification causing most issues was “Symptoms, Diagnosis, Spread.” This was somewhat expected. The sorting data showed strong connections between symptoms and diagnosis-type tasks. There were some—though weaker—connections between tasks connected with transmission and spread, and symptoms and diagnosis.

There were three tasks in the Transmission cluster:
• Transmission, spread, epidemiology: 62% success
• Virus survival / viability / persistence on surfaces, in air: 30% success
• Virus mutation, new strains: 19% success

It was decided that for round two we would break this class into two:
• Symptoms, Diagnosis
• Spread, Mutation, Survival

Testing-related tasks were strongly connected to the “Symptoms, Diagnosis, Spread” class and the “Vaccine, Immunity, Treatment” class. It was decided that there should be links to testing from both these classes.

The task “At risk, vulnerable” was performing poorly. This, again, was expected. It was the most orphaned task in the sort, meaning that it was the least connected with any other tasks. The reason it was not placed at the top level as its own class was because it was a tiny task, coming 29th. However, data we received from a COVID-19 top tasks survey in Norway indicates it is a top task there. On the national Irish health COVID-19 website and on the UK NHS website, it is at the top level of the classification. It was decided to make “At Risk, Vulnerable” a top-level classification.

Here is the classification that will go into round two and will be tested using the exact same list of task instructions:
WHO, Government Guidance, Education, Training
Mental, Physical Wellbeing
Vaccine, Immunity, Treatment
Research, Statistics
Virus Survival, Spread, Mutation
Avoiding Infection
Symptoms, Diagnosis
News
End Date, New Outbreaks
At Risk, Vulnerable

Gerry McGovern

Round 2 WHO COVID-19 Classification study is still open if you’d like to do it:
https://customercarewords.optimalworkshop.com/treejack/c554y8py-0

How would you organize COVID-19 information?
https://customercarewords.optimalworkshop.com/optimalsort/x13lrtt8

40% off Top Tasks print version at Lulu
http://www.lulu.com/shop/gerry-mcgovern/top-tasks-a-how-to-guide/paperback/product-23824878.html

CUSTOMER EXPERIENCE EXCELLENCE & TOP TASKS
Learn how your organization can deliver a better customer experience through use of the Top Tasks customer experience management model. Top Tasks will tell you what matters most to your customers (or employees), how well they are able to do these tasks, and what needs to be improved. Top Tasks has been used by Cisco, IBM, Microsoft, NetApp Toyota, European Union, various governments, universities and intranets.
info@customercarewords.com

CONSULTANCY PARTNERING WITH TOP TASKS
What you get as a consultancy by becoming a Top Tasks partner. The detailed methods, templates and tools. How the Top Tasks partnership works, commission rates, etc.
info@customercarewords.com

Get updates, insights, news from Gerry McGovern on Twitter
@gerrymcgovern

Subscribing and Unsubscribing to New Thinking
http://www.gerrymcgovern.com/new-thinking

Technical Problems
If you are having any technical problems, please email:
mailto:info@gerrymcgovern.com

www.gerrymcgovern.com
www.customercarewords.com

*****************************************************************