It’s been fascinating to watch the evolution of client-focused design over the years. At Vanguard we have long had “usability” practices that studied the preferences of clients for certain features and design treatments on our websites. We would bring clients into labs for demos of prototypes, gather their feedback, and make changes. We added feedback forms to the website.
More recently we brought in eye-tracking software that allowed us to understand customers’ actual behavior when navigating the site. And we’ve invested for years in pilots or beta tests with certain clients or groups of clients before rolling out the features completely, understanding that clients provide better feedback when their own data is used in the beta release. The next step in that evolution was multivariate and A/B testing – the ability to test designs in parallel on full-blown production sites, using clickstream data to measure effectiveness without actually having to ask the user.
Tapping big data for new insights
Today, the real breakthrough is enabled through big data technology that gives us the ability to marry up web data with offline sources of data – call center data, account data, demographic data, social media data – to really get a holistic view of how our technology solutions impact clients. For example, our client insights group has recently created a “suspect index” of web pages that we suspect are causing clients to call because they don’t have the information they need to complete their task online. We can then effectively target those pages and use multivariate testing to get to the best design for clients. Another source of feedback comes from mining our call center recordings and social media sentiment to look for potential opportunities with our services, or understand features clients or prospective clients need.
Improving the internal customer experience
As for our IT organization’s internal business customers, we have done something a bit novel and applied the popular Net Promoter Score system internally. Each year, we send a survey to our business partners – key product owners, stakeholders, and senior leadership – and ask them “If you could choose any IT organization, even outside of Vanguard, how likely are you to recommend your IT partner to your colleagues?” That’s a tough question, and we honestly didn’t think we would learn much from this the first time around.
We were dead wrong! By including survey questions designed to uncover sentiment on several critical-to-quality drivers such as business acumen, delivery on commitment, and quality, we uncovered areas where we thought we were performing well by our internal measures, but were actually missing the mark with our business. Now we follow this survey up with interviews getting at the heart of the answers and create action plans based on the feedback. Since we’ve been doing this survey, we’ve seen our score increase year-over-year. We still have a way to go to achieve our goal, but the big win is that we now have a really effective instrument in place to help us get better and to visibly measure that progress.
