...

Usability Testing worth millions

  • Client RBC Capital Markets
  • Platform Internal Web App
  • My Role Lead UX Research and Design

TLDR; or the project in a nutshell

Project Overview

Our client, a leading investment based in New York needed a next generation system for their salespeople. A salesperson's job is to interact with the clients and persuade them for more business:in this case, Securities. Now how do they know which securities to push to the client? This is generally decided by traders and the bank.

Our client, Traders and salespeople gather in the morning in the bullpen, where traders tell the salespeople which securities to push to the client: • The salespeople note them on a paper or an excel-sheet and make calls to the clients assigned to them. • Before making the call they go through CRM systems, Bloomberg Terminal, internal spreadsheets, etc. to get information about the client.

Since we had no interaction with the actual users in the beginning, a lot of ideas in the initial concepts were based on trial and error. Our initial conceptual model of the process and the end user was as follows:

As the lead UX designer for this project, I was responsible for:

  • Conducting secondary research to understand the domain.

  • Conducting a stakeholder workshop to understand the client's requirements and coalesce them into a guiding problem statement.

  • Conducting field research to validate our hypothesis and come up with insights about users.

  • Making wireframes to test with a limited set of tractor drivers (primary user).

  • Working with the visual designer to create a visual language for the app.

Understanding the domain: Secondary Research and a lot of Shooting in the Dark

Since we had no interaction with the actual users in the beginning, a lot of ideas in the initial concepts were based on trial and error. Our initial conceptual model of the process and the end user was as follows:

...
...
...
...
...
...
This is a caption on this photo for reference
  • Salespeople are the bridge between a Bank’s client and the traders.

  • Clients tell them which securities to buy/sell to the traders and vice versa.

  • Apart from the clients' requests, the bank also pushes securities to pitch based on a proprietary algorithm. It considers the client's portfolio and other pertinent data points.The end goal is to increase the bank's business.

  • Salespeople and traders usually plan which bonds to push in the morning, with a paper list or an excel sheet.

  • Unlike traders, salespeople are not concerned with live change in prices, but on the bigger picture. Salespeople are looking for trends and patterns in their client's business behavior. (This was one of the reasons we used visualisations that show trends e.g. spark-lines).

  • They don’t use mobile phones or tablets to access the system (only desktop experience).

...

Understanding the client needs and defining the problem: Field Research

After meeting the stakeholders, I was informed that they had an existing app for tractor drivers, but it had usability issues. As a result, drivers were unable to use it. My primary task was to create an app that was easy enough to be used by tractor drivers with varying levels of ease with smartphone use. The biggest constraint with this project was a short timeline as this whole project was a proof of concept, and the client didn't have enough time for field research and iterations.

Instead of following the standard persona template, we concentrated on task analysis as there is no specified age group for salespeople, and the tasks they hand are quite specific, their end goals are the same.

  • Sales people usually get into the office at about 7–7:30am and finish by 4:30–5:00pm.

  • Start the day with a meeting with traders. The traders walk them through the securities on focus for the day.

  • The sales people may start with the security or a client may call them over the phone inquiring about a certain security. At times a trader or another sales person may also tell them to focus on a security.

  • A lot of communication happens verbally within the office. Everyone sits within a earshot distance.

  • Usually about 20–25 clients are assigned to a sales person’s ‘book.’

  • Prices are not firm and are not always frequently updated. Sales people may ask the trader before making an offer to the client. It may also depend on geographies.

  • Sales people may follow up with a trader, if an e-inquiry by one of their clients is not being executed.

  • The layout of the application must be highly customisable as each trader may have unique tab structures.

  • They only work from the office, and don’t take any work home.

  • They don’t use mobile or tablets.

...

As the lead UX designer for this project, I was responsible for:

  • Trader defines securities in focus at the start of the day.

  • The system ranks these securities based on a weightage. The weightage is combination of client portfolio, holdings, and recent transactions.

  • Sales people usually start at the top of the list by selecting a security.

  • Security matches according to recent client transactions are displayed, along with market activity, live prices, client list, trade history, new issue history, portfolio match and feedback matches are displayed.

  • To get more information about a security, salesperson may get in touch with the trader through phone or IB chat (Bloomberg Terminal).

  • The sales person calls or communicates over a chat with the client. If the client agrees to a trade, a ticket is raised and sent to the trader, who then executes the trade.

  • If the client is not interested, then a manual feedback is submitted. Feedback is categorized under buy, sell, hold and info. Info is kind of a catch all.

  • When the client does not agree to a trade, there is an still an opportunity to obtain information about the client’s view on that security or market in general. This data is not being captured systematically today.

As the lead UX designer for this project, I was responsible for:

  • A list of focus clients is displayed. This list is ordered by a combination of factors — revenue, recent transactions, hit ratio, and current holdings.

  • Selecting a client will reveal a list of securities that the client may be interested in.

  • Associated with each security there may be the bank’s price, market price, aggregate views of Flow (trade, e-inquiries, feedback, and orders/quotes.), and holdings.

  • The sales person calls or communicates over a chat with the client. Sales person has the option to present multiple securities to the client.

  • If the client agrees to a trade, a ticket is raised and sent to the trader, who then executes the trade. If the client is not interested, then a manual feedback is submitted. Feedback is categorized under buy, sell, hold and info. Info is kind of a catch all.

  • When the client does not agree to the trade, there is an still an opportunity to obtain information about the client’s view on that security or market in general. This data is not being captured systematically today.

  • What happens when a sales person is done with a particular client? Do they move to the next client? Can we mark the client ‘done’ like in an email?

As the lead UX designer for this project, I was responsible for:

If a salesperson (a) does not see a client or axe to click in their list, or (b) wishes to quickly search for either, they may enter a free-search.

  • A client calls the salesperson with an inquiry.

  • A trader has directed them verbally to focus on a specific security

  • Results of the free-search are displayed contextually, co-mingling security and client-data.

  • The workflow switches to either Security or Client mode, depending on what the salesperson selects.

...
...
...

Prototyping

I used Axure to create a high-fidelity prototype as the end users had little time to spare. The high fidelity prototype helped them concentrate on their tasks (Prototype link can’t be shared as it contains confidential data).

Usability Testing: Grounding the findings

We conducted 5 minute testing sessions with users on the trading floor. Two users who took special interest in the prototype were frequently contacted during the duration of the project.

Eric’s Feedback

  • Tile view was appreciated.

  • Handles around 50 clients, interface needs to take care of edge cases.

  • Concerned that spreads and pricing data is not updated properly; Ticket processing takes time.

  • Would like to see when was the security last updated.

  • Hit ratio was not that important in his opinion for his case.

  • Would definitely like to see client activity indication, to know which client is going cold.

  • A provision to have feedback on security level with respect to the client.

  • Would like to see Trader's Bloomberg Professional updates.

  • When searching for security with specific parameters, in case of no match, similar securities should be shown.

Amber’s Feedback

  • Appreciated number of recommended securities next to each client.

  • Likes the hit ratio graphs and executed trade.

  • Likes security recommendations and the provision to hover and get more information.

  • Was confused about size and color of the tiles.

  • Would like to see client level trade volume and activity.

Visual Design: Designing the final screens

Conclusion: Limitations and Learnings

In an ideal world we would tested and iterated on the prototype with more users, however as our engagement was limited to 4 months, we were unable to see the impact on the end users. While the project was quite challenging, primarily because of the domain, the working group was extremely helpful and understanding and guided us every step along the way.