Turning an emotion label into a colour and a tip

The model’s answer is one word: “sad”, “happy”, “neutral”. Right or wrong, a word on its own gives the person in front of the camera nothing to do. So the upload page answers with three things: the label, a colour that fills the screen, and a one-line tip.

One lookup table

Each emotion maps to a pair: a style for the page background and a short tip. The server picks the pair, and the page applies the colour to the whole screen.

• Angry and sad: a calm blue, and “Relax and take deep breaths.”
• Disgust: green, and “Maybe step away from whatever’s bothering you?”
• Fear: black, and a reminder to take a moment and breathe deeply.
• Happy: sunny yellow, and a line about seeing some sunshine.
• Neutral: grey, and “Ask the person to smile a little and show some teeth.”
• Surprise: white, and “What’s got you like that? Please share the tea.”

That’s seven emotions, six colours and six tips.

The choices inside it

• Angry and sad share a response. They’re different feelings, but the same advice fits both: slow down and breathe. One calm blue covers them.
• Contrast is set per colour. On yellow and white, the text switches to black; on the other colours it stays white. Without that, white text on a white page would simply vanish.
• Neutral talks to someone else. Its tip is addressed to whoever is holding the camera, asking for a smile with teeth. A non-result becomes an instruction.
• The tone is light. “Please share the tea” is a long way from clinical language, which suits a demo better than anything that sounds like a diagnosis.
• You see what’s being judged. The chosen photo appears as a preview before upload, so there’s no doubt which image the answer is about.

Where it falls short

Looking back, there are four gaps.

• Only the upload page responds. The live page shows the label as text, without the colour or the tip.
• It shows one answer, not the odds. The model produces seven probabilities and the app keeps only the highest. A face the model barely rates as sad gets the same full blue screen as one it’s sure about.
• Tips assume the label is right. At about 62% accuracy, close to four answers in ten are wrong. A breathing tip for a calm person is harmless, but it shows the app speaking with more certainty than the model has.
• Fear is nearly invisible. Its black background is hardly different from the page’s usual near-black, so a fear reading barely changes the screen.

What I’d change

1. Bring the colour and tip to the live page, with a slow fade so the screen doesn’t flash on every frame.
2. Show the top two emotions with their probabilities, so people can see when the model is torn.
3. Only colour the screen when one emotion is clearly ahead. Otherwise say “not sure” and skip the tip.
4. Write every tip as a suggestion, never a diagnosis.

Rules I took from it

1. A prediction should end in something a person can do.
2. Let confidence set the tone. The less sure the model, the softer the response.
3. Check contrast for every state, not just the default one.