From client labs to big data: How Vanguard has evolved customer experience

From client labs to big data: How Vanguard has evolved customer experience

507 readers like this
CIO Big Data Analytics

It’s been fascinating to watch the evolution of client-focused design over the years. At Vanguard we have long had “usability” practices that studied the preferences of clients for certain features and design treatments on our websites. We would bring clients into labs for demos of prototypes, gather their feedback, and make changes. We added feedback forms to the website.

More recently we brought in eye-tracking software that allowed us to understand customers’ actual behavior when navigating the site. And we’ve invested for years in pilots or beta tests with certain clients or groups of clients before rolling out the features completely, understanding that clients provide better feedback when their own data is used in the beta release. The next step in that evolution was multivariate and A/B testing – the ability to test designs in parallel on full-blown production sites, using clickstream data to measure effectiveness without actually having to ask the user.

Tapping big data for new insights

Today, the real breakthrough is enabled through big data technology that gives us the ability to marry up web data with offline sources of data – call center data, account data, demographic data, social media data – to really get a holistic view of how our technology solutions impact clients. For example, our client insights group has recently created a “suspect index” of web pages that we suspect are causing clients to call because they don’t have the information they need to complete their task online. We can then effectively target those pages and use multivariate testing to get to the best design for clients. Another source of feedback comes from mining our call center recordings and social media sentiment to look for potential opportunities with our services, or understand features clients or prospective clients need.

Improving the internal customer experience

As for our IT organization’s internal business customers, we have done something a bit novel and applied the popular Net Promoter Score system internally. Each year, we send a survey to our business partners – key product owners, stakeholders, and senior leadership – and ask them “If you could choose any IT organization, even outside of Vanguard, how likely are you to recommend your IT partner to your colleagues?” That’s a tough question, and we honestly didn’t think we would learn much from this the first time around.

We were dead wrong! By including survey questions designed to uncover sentiment on several critical-to-quality drivers such as business acumen, delivery on commitment, and quality, we uncovered areas where we thought we were performing well by our internal measures, but were actually missing the mark with our business. Now we follow this survey up with interviews getting at the heart of the answers and create action plans based on the feedback. Since we’ve been doing this survey, we’ve seen our score increase year-over-year. We still have a way to go to achieve our goal, but the big win is that we now have a really effective instrument in place to help us get better and to visibly measure that progress.

John T. Marcante is Vanguard’s chief information officer and managing director of Vanguard’s Information Technology Division. Vanguard is one of the world’s largest investment management companies. As of December 31, 2017, Vanguard managed more than $4.9 trillion in global assets.

7 New CIO Rules of Road

CIOs: We welcome you to join the conversation

Related Topics

Submitted By Sachin Gupta
January 22, 2021

Remote work brings new challenges to the hiring process. These interview questions can help you gain insight into a candidate’s communication skills, initiative, and more

Submitted By Yoav Kutner
January 22, 2021

Think you know how to motivate your team? You might be surprised. Consider this advice on what inspires people to work hard.

Submitted By Lee Congdon
January 21, 2021

By establishing flexibility, cost-effectiveness, business partnerships, and a culture of ownership and accountability, we successfully dealt with great change.


Email Capture

Keep up with the latest thoughts, strategies, and insights from CIOs & IT leaders.