Too many IT operations are trying to cope with today's data using yesterday's methods, says Christian Beedgen, CTO of machine data analytics company Sumo Logic. In an interview with The Enterprisers Project, he explains what approaches work now.
The Enterprisers Project (TEP): What must companies do to meet the challenges of today's data?
Beedgen: We already know that current data tends to be voluminous, or big. But we also need to focus on data arriving in streams to process in real time with minimal latency. This process can be hard to do and it differs quite a bit from traditional data management and the ad-hoc query approach common in the database world.
And in contrast to the clean structures found in relational data in traditional databases, today's data comes from all sorts of places. It's usually not overly clean and is almost always poly-structured. Modern data strategies need to aim for integrated solutions that cover streaming acquisition, cleansing, and transformation of data flowing into real-time analytics engines that are required to produce insight with very little latency. If your competitor can do it better and faster, you will be out of business.
TEP: M2M (machine-to-machine) data is an increasingly significant part of any organization's data set. What are the best strategies for using this data?
Beedgen: Most M2M, or machine data, is created by the applications that support and often enable business. There is a maturity model here.
In short, it starts with using machine data to enable efficient operations of the applications that run the business. With machine data, ideally, the strategy would be to not only maintain the applications but also enable agile development on top of them. It means constant updates and a required counterbalance for the risks involved in constantly changing systems.
Machine data can make the business more efficient in running foundational applications and systems.
TEP: Why is it important for today's organizations to be able to work with both traditional RDMS and unstructured data?
Beedgen: It is important because data knows no bounds, and because you often have to play with the hand you're dealt. Businesses have a significant amount of data in classic relational databases, and there's nothing wrong with that — it is already there! But modern data is mutating into a different beast. Modern data is predominantly poly-structured, and accordingly, systems have to be able to seamlessly blend data of any kind — with varying degrees of structure, coming from traditional data stores as well as from real-time data streams.
TEP: How can IT leaders make sure to get the benefit of data in time to act on the insights it provides?
Beedgen: There are many secondary options. You can bring in a contemporary machine data analytics platform, or establish big data competency. But the first step is to create what is at the heart of it all. Decide what applications and systems drive the business? What data are they processing? What data do they create during the processing? And how can this data be used for optimization? For this, you have to know what needs to be optimized. Start there and work your way backward.
TEP: What advice would you give CIOs and CTOs about making the best use of data?
Beedgen: My advice? The first step to understanding the imperative of modern business is optimization. Whatever you do with modern data, if your competitor does it better and faster, you will lose. The systems that support the business create the data that must be used to optimize the business!
