I'm an UI/UX Designer based in NYC
Winrate.png

UX Project: Win/Loss Report

UX Project: Win/Loss Report

 
 

role: ui/ux LEAD designer

TOOL: SKETCH, BALSAMIQ, PHOTOSHOP

Win/Loss report is a L size 4-6 week design project. This project introduces a new type of report in T1.

Win rate is an important metric for users who want to monitor their campaign bid opportunities. Win rate can inform users on whether they need to increase or decrease their bids across a campaign or strategy. 

Design Process Summary

  1. Product Manager holds a kick off meeting with UX Designer to discuss the business requirements and project goals
  2. UX Designer and Product Manager conduct user research with users
  3. Product Manager and UX Designer brainstorm on different ideas for the reporting chart
  4. Product Manager presents their early design ideas during the UI planning meeting to get feedback on technical feasibility 
  5. Product Manager regroups with designer to focus on the design problem and sketch out simple concept for UX Designer to prototype
  6. UX Designer works on very lo-fidelity mockups 
  7. UX Designer writes up questions for validating designs with users
  8. Product Manager schedules users for user testing I 
  9. Product Manager and UX Designer sync up on designs and questions to finalize the mockups for testing
  10. Product Manager and UX Designer conduct user testing and record audio from their session
  11. UX Designer transcribes audio to identify user insights and area which need further design thinking and discussion 
  12. Product Manager and UX Designer run a design session to discuss user researching findings and unresolved design decisions 
  13. UX Designer works on prototypes for user testing II and writes questions, schedules and recruits users for testing
  14. UX Designer and Product Manager discuss the questions and finalize the designs for testing
  15. UX Designer conducts the user testing session
  16. UX Designer to finalize the design
  17. UX Designer uploads design deliverables in the ticket
  18. Product Manager to finalize product requirements and attends poker planning to push the ticket into a sprint 

Kick-off Meeting

Product Manager brief on UX Designer of the business goal and design requirements. Our goal for this project is easily delivering different data includes targeting match, creative match, bid rate and win rate to the users, and allowing them to toggle between strategies. We have 4 views need to design: Campaign Daily and Summary view, Strategy Daily and Summary view

Research on Win Rate Report

After the kick-off meeting, UX Designer did more research on the different chart: bar chart, scatter chart and funnel: Examples found on across Google Image search containing the keyword 'Win Loss Reporting'. Most common uses were in finance, sports and managing sales/client prospecting efforts. Charts varied from bar, spark, lines.

Brainstorm with Product Manager

Inspired by the last two charts, we came up with few ideas: 

  • Campaign Daily: Line chart
  • Campaign Summary: 
    • View by Bar chart: could show bid rates, but no average bid amounts.
    • Scatter + New Funnel A
    • Scatter + New Funnel B
    • Scatter + Old Funnel(Left aligned)
  • Strategy Daily(up to 30 days): stacked bar by each day instead of line chartwith rate on Y (right side)/ Line with two metrics 

We sketched out four different charts to display the data for Campaign Summary page:

  • Vertical stacked bar chart: User can compare the win rate between different strategies, but there is a space limit.
  • Scatter chart + horizontal stacked bar chart/funnel chart: See the waterfall comparison for each strategy, but can't compare between different strategies

Low-Fidelity Mockups with Balsamiq

 Design 1: Campaign Summary - stacked bar chart;  Pros: Compare different strategies

Design 1: Campaign Summary - stacked bar chart;

Pros: Compare different strategies

 Design 2 : Campaign Summary - Scatter + Funnel A  Pros: See the waterfall comparison for each strategy

Design 2 : Campaign Summary - Scatter + Funnel A

Pros: See the waterfall comparison for each strategy

 Design 3: Campaign Summary - Scatter + Funnel B  Pros: See the relations between two data

Design 3: Campaign Summary - Scatter + Funnel B

Pros: See the relations between two data

 Design 4: Campaign Summary - Scatter + Funnel C  Pros: Show the all 4 data's relations and proportion 

Design 4: Campaign Summary - Scatter + Funnel C

Pros: Show the all 4 data's relations and proportion 

 Design 5: Campaign Summary - Scatter + Funnel C  Pros: Show the all 4 data's relations and proportion, and waterfall

Design 5: Campaign Summary - Scatter + Funnel C

Pros: Show the all 4 data's relations and proportion, and waterfall

Discuss with Product Manager about the user testing questions and Product Manager set up user tests

Ask user to introduce themselves, remind them we are testing our designs and not testing them and ask for permission to record the session.

Show user blank M&M Screen

Q1: Where do you expect to find the win/loss report? 

Observe: What will user select? Campaign or Strategy or Chart icon on campaign

Show user either report with A-Bars or B - Scatter

Q2: How would you interpret this graph?

Q3: How would you use these data points?

Observe: Listen for cues that they understand how to read graph, interpret % and tell us how they would change their campaign settings based on this data

Q4: Compare to the first chart, which one is easier for you to interpret? 

Show user all the funnel graph options

Q5: Which visual is easier for you to interpret the results 

Show user any report they saw and ask the following

Q6. How would you switch from daily/summary

Q7: If you want to make a change your campaign based on this chart, where will you go to edit it?

Q8: How do you feel being able to edit here?

We Set up with 8 users in one day, showing bar chart and scatter chart to the users alternatively.

Summary

6 out of 8 users think funnel/scatter chart is easier to read.

3 out of 6 users think funnel/scatter chart is useful after they saw the funnel chart(not the scatter one). 

4 out of 8 users think they would like to compare between strategies. 

6 out of 8 users got confused of the stacked bar.

8 out of 8 users didn’t see the summary/daily toggle until it got pointed out.

For funnel/Chart, users have their favorite: 1, 5 (they want to see the numbers) 

The users think inline edit is nice to have, but not necessary. 

Stacked Bar chart:  Didn’t represent the waterfall relationship between the data points

Someone brought up a table chart idea.

UX Designer brought up a few design questions after discussed with Product Manager

  • Bar chart
    • good for compare at strategy level
    • How to make the bar more clear to read?
      • Group bar(horizontal funnel)
      • Color and interaction 
      • Add Table chart
  • Scatter + Funnel chart
    • What is the default Highest dropdown? 
      • User’s preference?
    • How to make users understand the funnel/scatter chart when there is no strategy selected? 
      • Make the text clear: “ Click a dot/Strategy to view the opportunity funnel”
      • Show the funnel right next to the scatter, make the funnel grey out.  
      • New idea: Add a table chart, have a scatter, funnel, table slideshow 
      • How to show the strategy name in a scatter view?/How to find a specific strategy in a scatter view?
        • Hover to show the name
        • Search box
      • Can we still compare between strategies in this view? 

Design requirements after the meeting: 

User needs:

  • User needs the ability to compare the strategies under one campaign
  • User needs the ability to click in each strategy to see the win rate

Hi-Fi mockups: (Spacing and color)

  • One screen to show scatter and table(color block match to the dots, maybe an icon/something to trigger the funnel)
  • One screen to show funnel(Change colors, and change the arrows) and table
  • One screen to show scatter and funnel

Lo-Fi interaction:

  • scatter and table:
    • When hover on a dot, the strategy in the table will be highlighted; When hover on the strategy in the table, the dot will be highlighted.
    • When click a dot, trigger the funnel
    • when click a strategy in the table, dot disappear.(?)
  • Funnel and scatter:
    • Click two dots, show different funnels.
    • Reset button go back to the initial stage.

Second round of the user test

Main purpose: 

  • Interaction with the scatter?
  • Interaction with the table? (sort/click on the chart? Click on the whole bar?)
  • Interaction with Funnel?

Scatter:

  • Looking at the scatter graph, how would you interact with it?
  • (If no one says they will click on the dot) When you click on the dot, what do you expect to happen?

Funnel:

  • For who saw this graph, please don’t answer it. For whom haven’t seen this graph, please tell us how would you interpret this funnel?

Table: 

  • How would you interact with the table?
  • (Point on the icon) What do you think these icons do?
  • (If no one says) When you click on the row, what do you expect to happen?
  • (If no one says) When you click on the chart icon, what do you expect to happen?

We rounded up with a group of 8 users, 30 mins questions and answers while we went through the clickable mockups. 

Summary

  • What would you do when you first come to this screen?
    • 6 users will click a strategy dot   
    • 1 user will scroll down strategies & win rates box 
    • 1 user will click view by to select strategy/exchange 
  • Scatter view: What do you expect to see when you click on the dot?
    • 8 users expect to view funnel/ details   
  • Funnel view: What do you expect to happen when you click on the toggle? Table icon
    • 3 users have no idea 
    • 1 user expects to see different visualization 
    • 1 user expects to collapse funnel 
    • 1 user expects to see daily view report 
    • 1 user expects to recalculate the win rate through the funnel 
    • 1 user expects to see strategy metrics 
  • Table view: What do you expect to happen when you click on the chart icon?
    • 4 users expect to see the same interaction as click the dot  
    • 2 users expect that the strategy dot is highlighted on the chart  
    • 2 users expect that the strategy data would populate on the chart  
  • Scatter view: What do you expect to happen if you click on another grey dot?
    • 6 users expect that the view will switch to that strategy funnel  
    • 1 user expects that strategy is highlighted in lower MM screen  
    • 1 user expects that the strategy is highlighted on the right side chart 
  • Table view: What do you expect to happen if you click on the strategy row?
    • 6 users expect to see the same interaction as dot  
    • 1 user expect to see dot highlighted on chart 
    • 1 user expect to see Max bid: strategy type

Overall,  for the Funnel View, 8 of 8 users didn't know what to expect when they click on the toggle(table icon). So how do we make the toggle information more clear? Adding a tooltip? 

High-Fidelity Mockups

Results:

After this new feature releases, this has been the second popular report in T1. Users love the chart and the interaction.