More and more teams are beginning to use Kubernetes in production to run their containerized workloads and applications. “Kubernetes is quickly becoming the easiest way to operate large-scale, complex applications in production when zero downtime is essential and security is paramount,” says Gou Rao, CTO and cofounder at Portworx. “Kubernetes is complex enough that it is not yet for everyone, but for apps with a high level of scale and complexity that need to run across multiple clouds and data centers, there is nothing close to Kubernetes.”
But it’s not a set-and-forget tool, no matter how much automation power it packs. Ongoing management tasks deserve attention.
“Like any platform, Kubernetes requires administration, management, and maintenance,” says Ravi Lachhman, DevOps evangelist at Harness.
Right alongside, there’s security - which should be baked into your processes as early as possible. “New technologies can challenge existing approaches to security,” says Red Hat security strategist Kirsten Newcomer. “For example, data collection and network security solutions need to adapt to an ephemeral environment where applications come and go. And best practice is never to patch a running container – if you do, the next time that container instance is deployed from an image, the patch is lost. This means that organizations need to revisit their investment in DevOps and in shifting security left.”
[ Want to learn more? Get the free eBook: Getting Started with Kubernetes. ]
7 tips for managing Kubernetes for the long haul
We asked experts for their advice on what teams and practitioners should know at the start about managing Kubernetes over the long haul. Dig into these seven pieces of wisdom:
1. You have to treat Kubernetes as a serious investment
It’s one thing to tinker with Kubernetes on a laptop or sandbox environment. (Many teams start this way, and Minikube and similar tools make doing so relatively straightforward.) It’s another to run it in production.
“Doing Kubernetes right requires that enterprises take it seriously,” Rao says. Indeed, all of the tips in this article essentially spring from this: It’s a powerful software engine, as Rao puts it, so treat it accordingly.
[ Related read: Kubernetes: 6 secrets of successful teams. ]
“This engine wants to be able to run, and move, and move your apps again at any time, for any reason, subject to the parameters that you give it,” Rao says. “Basically, you tell Kubernetes what you want to happen, and it makes it happen. If you are going to benefit from this relentless operational efficiency, you need to provide it with an environment in which it can operate effectively.”
That means you’ve got the right people, infrastructure, processes, and culture in place. Cutting corners, or hoping Kubernetes will paint over existing structural problems, will make managing your environment unnecessarily messy.
“You wouldn’t hire Martin Scorsese to direct next summer’s big blockbuster, then halve his requested budget, reject his preferred actors, and make edits to his final script,” Rao says. “That wouldn’t be a Scorsese movie anyone wants to see.”
[ Read our deep dive for IT leaders: Kubernetes: Everything you need to know. ]
2. Even orchestrators need some TLC
Some teams (especially those just starting out) may think Kubernetes will run itself. That misnomer actually has a pseudo-basis in reality, given the orchestrator’s declarative nature and emphasis on automation. But Kubernetes needs some TLC, like any other system.
“Every organization is different, and having new learnings captured for the first time will have a long-lasting impact on your organization,” Lachhman says.
Lachhman points to examples of common maintenance and management tasks, such as:
- Patching or upgrading a cluster and its underlying infrastructure
- Adding and subtracting worker nodes
- Plugging in new functionality, such as service mesh tools like Istio and Traefik
- Developing a clearly defined Namespace taxonomy, especially as more teams begin working with Kubernetes
Why are namespaces important? They help secure access to resources in containers among multiple users. Our sibling site Opensource.com offers a great technical primer: Kubernetes namespaces for beginners.
3. Kubernetes needs proper infrastructure to succeed
One of the fundamental needs for successfully managing Kubernetes over time: the right infrastructure. Think modern here – hybrid cloud and/or multi-cloud environments, for example.
“The mistake I see teams making is moving to Kubernetes because they want to be agile but then hampering Kubernetes’ ability to provide the operational outcomes that drove their initial decision. It is self-defeating,” Rao says.
“Any system is only as agile as its least-agile element, so if you run Kubernetes in a static data center environment stocked up with the same hardware and software powering your much less agile VM environment, then you are not taking Kubernetes seriously and you won’t get the benefits you are after.”
Storage is one specific subset of your infrastructure that may require planning, according to Maksim Yankovskiy, VP of engineering at Zettaset. Stateful containers, for example, come with particular storage considerations.
“Planning storage for workloads is critical,” Yankovskiy. “Some workloads may require specialized storage or per-application tuning of storage parameters. Not all storage should be managed by Kubernetes.”
[ Kubernetes Operators can be a boon for stateful apps. Learn more about them: Kubernetes Operators: 4 facts to know. ]
Rao also points to storage as an example of a specific area that a team might ignore as it adopts Kubernetes in an effort to become more agile, only to run into management headaches.
“I work with a lot with customers running data services on Kubernetes, and one of the things that I keep hearing is ‘I tried running MongoDB on Kubernetes but I keep having issues with my SAN,’” Rao says. “SANs are great, but they weren’t designed for the scale, density, and dynamism of Kubernetes. To get the most out of Kubernetes, you need to give it the infrastructure it needs to operate efficiently. It can’t work its magic if you give it static infrastructure.”
[ Get the free eBook: O’Reilly: Kubernetes Operators: Automating the Container Orchestration Platform. ]
4. Kubernetes' reliability can make you overconfident
Kubernetes’ self-healing and reliability features are a big part of its appeal, as Rao noted at the outset. Just don’t use them as an excuse for ignoring your operational health. You’ll still want to ensure you have monitoring in place, for example.
“Don’t rely on Kubernetes reliability features to work around faulty services,” Yankovskiy says. “Just because Kubernetes does a great job automatically restarting failed pods doesn’t mean that you should not be monitoring your applications for run-time errors.”
Logging is another critical management need as you grow.
“Kubernetes clusters are highly distributed, so your first step in deploying one should include planning for centralized logging facilities,” Yankovskiy says. “Accessing logs on individual nodes when monitoring performance or diagnosing a problem is not scalable. [A] great log management system is one of the foundations of capable Kubernetes management.”
Red Hat technology evangelist Gordon Haff adds, “Cloud-native applications deployed to Kubernetes are fundamentally distributed applications, an architecture that IT shops accustomed to running monoliths may not have a great deal of experience with. The ecosystem of open source projects in Kubernetes’ orbit includes monitoring, logging, distributed tracing, and more. But you probably want a platform that already integrates that tooling with Kubernetes rather than embarking on a big DIY project. Your staff may also benefit from training in new cloud-native app development practices."
In general, teams tend to run into problems when they treat Kubernetes as a magic wand rather than a tool that must be maintained and improved just like any other important system. In this vein, Lachhman points out that fallacies of distributed computing still apply in the Kubernetes landscape.
“You likely will have multiple administrators, a regularly changing topology, and not be immune to network latency while using Kubernetes,” Lachhman says.
Let's explore three more important things to know:
