Docker and other Linux containers

Virtual machines are mainstream in cloud computing. The newest development on the this arena are fast and lightweight process virtualization.  Linux-based container infrastructure is an emerging cloud technology that provides its users an environment as close as possible to a standard Linux distribution.

Linux Containers and the Future Cloud article tells that as opposed to para-virtualization solutions (Xen) and hardware virtualization solutions (KVM), which provide virtual machines (VMs), containers do not create other instances of the operating system kernel. This brings advantage of containers over VMs is that starting and shutting down a container is much faster than starting and shutting down a VM. The idea of process-level virtualization in itself is not new (remember Solaris Zones and BSD jails).

All containers under a host are running under the same kernel. Basically, a container is a Linux process (or several processes) that has special features and that runs in an isolated environment, configured on the host.  Containerization is a way of packaging up applications so that they share the same underlying OS but are otherwise fully isolated from one another with their own CPU, memory, disk and network allocations to work within – going a few steps further than the usual process separation in Unix-y OSes, but not completely down the per-app virtual machine route. The underlying infrastructure of modern Linux-based containers consists mainly of two kernel features: namespaces and cgroups. Well known Linux container technologies are Docker, OpenVZ, Google containers, Linux-VServer and LXC (LinuX Containers).

Docker is an open-source project that automates the creation and deployment of containers. Docker is an open platform for developers and sysadmins to build, ship, and run distributed applications. Consisting of Docker Engine, a portable, lightweight runtime and packaging tool, and Docker Hub, a cloud service for sharing applications and automating workflows.
Docker started as an internal project by a Platform-as-a-Service (PaaS) company called dotCloud at the time, and now called Docker Inc. Docker is currently available only for Linux (Linux kernel 3.8 or above). It utilizes the LXC toolkit. It runs on distributions like Ubuntu 12.04, 13.04; Fedora 19 and 20; RHEL 6.5 and above; and on cloud platforms like Amazon EC2, Google Compute Engine and Rackspace.

Linux containers are turning to a way of packaging up applications and related software for movement over the network or Internet. You can create images by running commands manually and committing the resulting container, but you also can describe them with a Dockerfile. Docker images can be stored on a public repository. Docker is able to create a snapshot. Docker, the company that sponsors the Docker.org open source project, is gaining allies in making its commercially supported Linux container format a de facto standard. Red Hat has woken up to the growth of Linux containers and has begun certifying applications running in the sandboxing tech.

Docker was last week a lot in IT news because Docker 1.0 has been released. Here are links to several articles on Docker:

Docker opens online port for packaging and shipping Linux containers

Docker, Open Source Application Container Platform, Has 1.0 Coming Out Party At Dockercon14

Google Embraces Docker, the Next Big Thing in Cloud Computing

Docker blasts into 1.0, throwing dust onto traditional hypervisors

Automated Testing of Hardware Appliances with Docker

Continuous Integration Using Docker, Maven and Jenkins

Getting Started with Docker

The best way to understand Docker is to try it!

This Docker thing looks interesting. Maybe I should spend some time testing it.

 

280 Comments

  1. Tomi Engdahl says:

    Creating small containers with Buildah
    https://opensource.com/article/18/5/containers-buildah?sc_cid=7016000000127ECAAY

    A technical problem drives home the power of open source collaboration.

    Reply
  2. Tomi Engdahl says:

    10 layers of Linux container security
    https://opensource.com/article/17/10/10-layers-container-security?sc_cid=7016000000127ECAAY

    Employ these strategies to secure different layers of the container solution stack and different stages of the container lifecycle.

    Reply
  3. Tomi Engdahl says:

    Just say no to root (in containers)
    Even smart admins can make bad decisions.
    https://opensource.com/article/18/3/just-say-no-root-containers?sc_cid=7016000000127ECAAY

    User namespace is all about running privileged processes in containers, such that if they break out, they would no-longer be privileged outside the container.

    OpenShift is Red Hat’s container platform, built on Kubernetes, Red Hat Enterprise Linux, and OCI containers, and it has a great security feature: By default, no containers are allowed to run as root. An admin can override this, otherwise all user containers run without ever being root. This is particularly important in multi-tenant OpenShift Kubernetes cluster

    Sadly one of the biggest complaints about OpenShift is that users can not easily run all of the community container images available at docker.io. This is because the vast majority of container images in the world today require root.

    Why do all these images require root?
    If you actually examine the reasons to be root, on a system, they are quite limited.

    Container runtimes can start applications as non-root to begin with. Truth be known, so can systemd, but most software that has been built over the past 20 years assumes it is starting as root and dropping privileges.

    the main reason apache or nginx start out running as root is so that they can bind to port 80 and then become non-root.
    Container runtimes, using port forwarding, can solve this problem. You can set up a container to listen on any network port, and then have the container runtime map that port to port 80 on the host.

    Bottom line
    Almost all software you are running in your containers does not require root. Your web applications, databases, load balancers, number crunchers, etc. do not need to be run as root ever. When we get people to start building container images that do not require root at all, and others to base their images off of non-privileged container images, we would see a giant leap in container security.

    There is still a lot of educating needing to be done around running root inside of containers. Without education, smart administrators can make bad decisions.

    Reply
  4. Tomi Engdahl says:

    10 tasks for running containers on Atomic Host
    https://opensource.com/article/18/5/10-tasks-running-containers-atomic-hosts?sc_cid=7016000000127ECAAY

    Learn what you can do with Atomic Host, a container operating system that provides a secure foundation for running containers.

    Reply
  5. Tomi Engdahl says:

    Tom Krazit / GeekWire:
    AWS now fully supports Kubernetes with the general release of Amazon EKS, following other cloud vendors, after announcing the service last year at AWS re:Invent — Amazon Web Services was the last of the major cloud vendors to go all-in with the Kubernetes container-orchestration project …

    Amazon Web Services now fully supports Kubernetes with the general release of Amazon EKS
    https://www.geekwire.com/2018/amazon-web-services-now-fully-supports-kubernetes-general-release-amazon-eks/

    Reply
  6. Tomi Engdahl says:

    Getting started with Buildah
    https://opensource.com/article/18/6/getting-started-buildah?sc_cid=7016000000127ECAAY

    Buildah offers a flexible, scriptable way to create lean, efficient container images using your favorite tools.

    Reply
  7. Tomi Engdahl says:

    Introducing conu – Scripting Containers Made Easier
    https://developers.redhat.com/blog/2018/03/07/introducing-conu-scripting-containers-made-easier/?sc_cid=7016000000127ECAAY

    There has been a need for a simple, easy-to-use handler for writing tests and other code around containers that would implement helpful methods and utilities. For this we introduce conu, a low-level Python library.

    This project has been driven from the start by the requirements of container maintainers and testers. In addition to basic image and container management methods, it provides other often used functions, such as container mount, shortcut methods for getting an IP address, exposed ports, logs, name, image extending using source-to-image, and many others.

    Reply
  8. Tomi Engdahl says:

    Running integration tests in Kubernetes
    https://opensource.com/article/18/6/running-integration-tests-kubernetes?sc_cid=7016000000127ECAAY

    Verify how your application behaves with your full solution stack by using multiple containers to provide a whole test environment.

    Linux containers have changed the way we run, build, and manage applications. As more and more platforms become cloud-native, containers are playing a more important role in every enterprise’s infrastructure. Kubernetes (K8s) is currently the most well-known solution for managing containers, whether they run in a private, public, or hybrid cloud.

    With a container application platform, we can dynamically create a whole environment to run a task and discard it afterward. In an earlier post, we covered how to use Jenkins to run builds and unit tests in containers.

    Reply
  9. Tomi Engdahl says:

    How to create a cron job with Kubernetes on a Raspberry Pi
    https://opensource.com/article/18/3/kubernetes-cron-job-tasks?sc_cid=7016000000127ECAAY

    Find a better way to run your scheduled tasks efficiently and reliably.

    Reply
  10. Tomi Engdahl says:

    Why Kubernetes is The New Application Server
    https://developers.redhat.com/blog/2018/06/28/why-kubernetes-is-the-new-application-server/?sc_cid=7016000000127ECAAY

    Have you ever wondered why you are deploying your multi-platform applications using containers? Is it just a matter of “following the hype”? In this article, I’m going to ask some provocative questions to make my case for Why Kubernetes is the new application server.

    Reply
  11. Tomi Engdahl says:

    Container storage for dummies
    https://www.redhat.com/en/resources/container-storage-dummies?sc_cid=7016000000127ECAAY

    Although many organizations still use traditional storage appliances, they don’t offer the agility needed by containerized environments. Containers are highly flexible and bring incredible scale to how apps and storage are delivered; traditional storage can be the bottleneck that stops this progress. The underlying storage should be highly elastic, easily provisioned by developers and admins, and, ideally, managed using the same orchestration framework (like Kubernetes).

    Reply
  12. Tomi Engdahl says:

    Building tiny container images
    https://opensource.com/article/18/7/building-container-images?sc_cid=7016000000127ECAAY

    Here are 5 ways to optimize Linux container size and build small images.

    Reply
  13. Tomi Engdahl says:

    Unleash powerful Linux container-building capabilities with Buildah
    Posted by Ben Breard
    https://rhelblog.redhat.com/2018/07/11/unleash-powerful-linux-container-building-capabilities-with-buildah/?sc_cid=7016000000127ECAAY

    Balancing size and features is a universal challenge when building software. So, it’s unsurprising that this holds true when building container images. If you don’t include enough packages in your base image, you end up with images which are difficult to troubleshoot, missing something you need, or just cause different development teams to add the exact same package to layered images (causing duplication). If you build it too big, people complain because it takes too long to download – especially for quick and dirty projects or demos. This is where Buildah comes in.

    In the currently available ecosystem of build tools, there are two main kinds of build tools:

    Ones which build container images from scratch.
    Those that build layered images.

    Reply
  14. Tomi Engdahl says:

    Linux containers, virtualization, and services: “Three’s a party” in modern IT
    https://www.redhat.com/en/blog/linux-containers-virtualization-and-services-threes-party-modern-it?sc_cid=7016000000127ECAAY

    Linux containers are often positioned as disruptive to traditional virtualization, frequently culminating in the question: Will containers kill virtualization? It’s a fair question, given the shared similarities in workload isolation, resource utilization, and so on, but the answer is a hard “no.” They’re complementary, each solving a unique challenge for the enterprise; that said, historically they don’t actually integrate or work well together. This means separate application stacks, separate developer workflows, and so on.

    Reply
  15. Tomi Engdahl says:

    Learn Kubernetes using Interactive Browser-Based Scenarios
    https://www.katacoda.com/courses/kubernetes

    Solve real problems and enhance your skills with browser based hands on labs without any downloads or configuration

    Reply
  16. Tomi Engdahl says:

    From 0 to Kubernetes
    https://opensource.com/article/17/11/kubernetes-lightning-talk?sc_cid=7016000000127ECAAY

    Clear up your confusion about containers and Kubernetes in this Lightning Talk from All Things Open 2017.

    Reply
  17. Tomi Engdahl says:

    Why Kubernetes is The New Application Server
    https://developers.redhat.com/blog/2018/06/28/why-kubernetes-is-the-new-application-server/?sc_cid=7016000000127ECAAY

    Have you ever wondered why you are deploying your multi-platform applications using containers? Is it just a matter of “following the hype”? In this article, I’m going to ask some provocative questions to make my case for Why Kubernetes is the new application server.

    You might have noticed that the majority of languages are interpreted and use “runtimes” to execute your source code. In theory, most Node.js, Python, and Ruby code can be easily moved from one platform (Windows, Mac, Linux) to another platform. Java applications go even further

    Reply
  18. Tomi Engdahl says:

    Conclusion
    Container images have become the standard packaging format to distribute cloud-native applications. While containers “per se” don’t provide real business advantages to applications, Kubernetes and its related projects, such as OpenShift and Istio, provide the non-functional requirements that used to be part of an application server.
    https://developers.redhat.com/blog/2018/06/28/why-kubernetes-is-the-new-application-server/?sc_cid=7016000000127ECAAY

    Reply
  19. Tomi Engdahl says:

    Kubernetes Services By Example
    https://blog.openshift.com/kubernetes-services-by-example/?sc_cid=7016000000127ECAAY

    When explaining Kubernetes to people new in the space I noticed that the concept of services is often not well understood.

    In a nutshell, Kubernetes services are an abstraction for pods, providing a stable, virtual IP (VIP) address. As pods may come and go, for example in the process of a rolling upgrade, services allow clients to reliably connect to the containers running in the pods, using the VIP. The virtual in VIP means it’s not an actual IP address connected to a network interface but its purpose is purely to forward traffic to one or more pods. Keeping the mapping between the VIP and the pods up-to-date is the job of kube-proxy, a process that runs on every node, which queries the API server to learn about new services in the cluster.

    But what makes the VIP 172.30.40.155 forward the traffic to the pod? The answer is: IPTables.

    Reply
  20. Tomi Engdahl says:

    History of containers on unix like system:
    1. chroot 1982
    2. Freebsd jails 2000
    3. Linux vserver 2001
    4. Solaris zones 2004
    5. OpenVZ 2005
    6. LXC 2008
    7. Systemd-nspawn 2010
    8. Docker 2013

    #sysadmin #linux #unix #macos #devops

    Reply
  21. Tomi Engdahl says:

    A Practical Introduction to Container Terminology
    https://developers.redhat.com/blog/2018/02/22/container-terminology-practical-introduction/?sc_cid=7016000000127ECAAY

    You might think containers seem like a pretty straightforward concept, so why do I need to read about container terminology?

    I’ve encountered misuse of container terminology that causes people to stumble on the road to mastering containers. Terms like containers and images are used interchangeably, but there are important conceptual differences. In the world of containers, repository has a different meaning than what you’d expect. Additionally, the landscape for container technologies is larger than just docker. Without a good handle on the terminology, It can be difficult to grasp the key differences between docker and (pick your favorites, CRI-O, rkt, lxc/lxd) or understand what the Open Container Initiative is doing to standardize container technology.

    Reply
  22. Tomi Engdahl says:

    William Denniss / Google Cloud Blog:
    Google to hand over operational control of the Kubernetes project to its community and donates $9M in Google Cloud credits to Cloud Native Computing Foundation — Kubernetes, the container orchestrator created and open-sourced here at Google, has experienced incredible development and adoption since it was introduced in 2014.

    Google Cloud grants $9M in credits for the operation of the Kubernetes project
    https://cloud.google.com/blog/products/gcp/google-cloud-grants-9m-in-credits-for-the-operation-of-the-kubernetes-project

    Reply
  23. Tomi Engdahl says:

    Code Execution in Alpine Linux Impacts Containers
    https://www.securityweek.com/code-execution-alpine-linux-impacts-containers

    A security researcher discovered several vulnerabilities in Alpine Linux, a distribution commonly used with Docker, including one that could allow for arbitrary code execution.

    Based on musl and BusyBox, the Alpine Linux distribution has a small size and is heavily used in containers, including Docker, as it provides fast boot times.

    APK, the default package manager in Alpine, is impacted by several bugs, security researcher Max Justicz has discovered. The most important of them, the researcher says, could allow a network man-in-the-middle (or a malicious package mirror) to execute arbitrary code on the user’s machine.

    Reply
  24. Tomi Engdahl says:

    A sysadmin’s guide to containers
    https://opensource.com/article/18/8/sysadmins-guide-containers?sc_cid=7016000000127ECAAY

    What you need to know to understand how containers work.

    Reply

Leave a Comment

Your email address will not be published. Required fields are marked *

*

*