Three ways for CIOs to handle big data

Three ways for CIOs to handle big data

471 readers like this


June 13, 2014
Big data assessment

Implementing new technologies in order to solve existing issues is part and parcel of the CIO's world. As new technologies appear and become validated as useful and safe for enterprise deployment, active IT departments take advantage of the new opportunities to put the technologies to work.

Big data is one of the technologies that is touted to be a solution for many issues. Technology executives who are able to move from initial deployment to a fully optimized and elegant implementation that delivers more than what was initially envisioned help to move their companies ahead of the pack.

Austin Park, senior vice president and chief technology officer for Paragon Development Systems, works with projects that involve big data and offered his thoughts on moving toward optimizing big data projects.

"Optimizing a big data environment can be a bit daunting simply due to the sheer volume of data we have to be concerned with. But you can break it down to some key areas of concern and approach them sequentially," Park said.

"One of the first considerations is the storage costs. Most organizations make storage hardware decisions based on a cost/performance formula they perform once every three years during a refresh. They need to adopt a far more granular approach when selecting appropriate storage for big data. Often making quarterly or even monthly adjustments to the storage architecture. Many are looking at  software defined storage to make this process easier, but ultimately it needs to support the tools you plan on using.

"Secondly, big data involves big compute. Optimizing that inevitably involves taking a look at new infrastructure architectures that incorporate lots of automation to ensure that you squeeze out every last bit of computational capacity. This usually involves building your own open stack environment or leveraging a cloud service.

"Thirdly, given the amount of data we are working with, the better organized the information is, the better optimized the solution will be. This, of course, is much easier said than done. Therefore, many organizations have started the work of creating data lakes. This is a relatively new term to describe a data-centered architecture, where silos are minimized, and processing happens with little friction in a scalable, distributed environment. This means the data itself would not be encumbered by poorly-organized schema decisions that may have happened years before knowing the type of data we are working with."

Clearly the path to optimizing the use of big data is not a simple one. But since we can expect that big data will become even bigger over time, it's a journey that is better started as soon as possible, and with a solid plan in place.

Comments 0
Scott Koegler practiced IT as a CIO for 15 years. He also has more than 20 years experience as a technology journalist covering topics ranging from software and services through business strategy. He has written white papers and directed and published video interviews.

Harvard Business Review: IT Talent Crisis: Proven Advice from CIOs and HR Leaders

CIOs: We welcome you to join the conversation

Related Topics

Submitted By Carla Rudder
September 21, 2018

Hiring managers are moving candidates with this skill to the top of their list. Here are 4 ways to highlight your adaptability .

Submitted By Peter Bendor-Samuel
September 21, 2018

Traditional change management practices weren’t built for this level of change. Here’s how to rethink two key aspects of your approach.

Submitted By Kevin Casey
September 20, 2018

How can you get a blockchain job – even without a ton of experience? Experts share advice on how to stand out and land a blockchain job.


Email Capture

Keep up with the latest thoughts, strategies, and insights from CIOs & IT leaders.