Productivity measurement in big data or any other data team is a management exercise that requires careful control. Most development and data management teams are culturally fairly independent as they are expected to be in order to be effective. Nonetheless, managing their respective productivity is a crucial step to ensuring the successful integration of the big data process into the overall business strategy mandate.
Back in the ancient history of five years ago, you (the CIO) might get away with a "no news is good news" mentality, which meant you could utilize a sort of laissez-faire approach to team management and focus your CIO endeavors and calendar in a more strategic-oriented venue. In the current and modern era this approach might put a new CIO in dangerous and possibly cold waters.
When we refer to CIO-level big data team productivity measurement, we imply two cardinal pieces: Internal team productivity measures and enhancements, and second, but much more complex and just as critical, is how the big data organization lifts overall business opportunity.
Big data is about distributed data and knowledge such that processes and products can deliver improved value for customers. Hence efforts in this area are not just about data stream quality but velocity as well. Executives will need to shift perspectives into data streams and not reports or data sets.
Big data team managers should not aim for a specific result (short or consistent) but rather should look to maintain balanced approaches that are consistent with the overall strategic mandates. The velocity of delivery is another basic area to help guide big data productivity measures.
As such, I suggest to look at the following key areas:
Business team data access velocity
How long does it take for business teams to review data streams after they are produced by the data science teams? This is a crucial measure, especially when related to the diversity data axis. This means the CIO must clearly understand if there are any bottlenecks early on because potential time lags will increase exponentially as the big data initiatives grow in sophistication and data sources. This can ultimately lead to failure of the big data initiatives
Data modeling and preparation velocity
How long does it take data science teams to correctly model and prepare the data for analysis? Does the data modeling process take days, weeks or longer? How long is the competition believed to take? This is another crucial measure that has a direct influence on data access velocity and has to be counterbalanced to the overall business team expectations. CIOs should take a close look at the data analysis tools set and data stream origins to ensure the tools in use are adequate and not hindering the data model stage.
Business big data output absorption
How long does it take for business teams to access and absorb data insights? These measures are highly relevant but need to be reviewed from a senior management or executive-level perspective to avoid unproductive internal competitive behaviors.
How long does it take for the business organization to produce changes or insights from
the available data streams? How useful is the data once produced? Not all data insights will produce revolutionary products or services. Some data streams will fall into the "I knew it already" category, and this is fine. CIOs should keep in mind that big data is about small but incremental shifts over time.
Getting a good handle on the above metrics will help ensure success in your big data team endeavors.
