
Hypernetes: Bringing Security and Multi-Tenancy to Kubernetes - scprodigy
http://blog.kubernetes.io/2016/05/hypernetes-security-and-multi-tenancy-in-kubernetes.html
======
switchbak
They mention that this helps cure some issues with regards to resource sharing
/ memory usage, etc. But does each VM still have a static allocation of
memory?

One of the main benefits I have now is that if I run a number of containers
that all take various amounts of memory, I can just throw them on and they
share memory amongst each other quite efficiently. If I have to make a static
allocation of memory for a VM, I'll typically choose a conservative memory
number, and usually under-utilize the machine, wasting a lot of memory per-
instance. Not so bad since I chose per-pod, but still an issue.

As it happens, this same issue is why I'm leaning towards lightweight native
applications these days instead of an aggressive greedy virtual machine that
grabs a bunch of heap. Golang/Rust in particular.

~~~
scprodigy
Actually, scaleup is pretty easy for both VM and Linux container. But scale-
down is very troublesome for both.

And the scheduler will need the mem size (not right now, but inevitable)

------
sarnowski
That is what rkt with kvm is essentially doing as well, correct?

[https://coreos.com/rkt/docs/latest/running-lkvm-
stage1.html](https://coreos.com/rkt/docs/latest/running-lkvm-stage1.html)

~~~
mjg59
Not quite - rkt will bring up a new VM for each container, this approach only
brings up a VM per pod (ie, a set of functionally related containers).

~~~
philips
This isn't correct. rkt does a VM per pod.

------
andrewstuart2
This really does not appeal to me at all. The major point of docker containers
is not the image format, it's that the kernel can allocate resources more
intelligently. VM images work just fine for "shippable images."

What I'd rather see is an allocation layer for physical resources that just
cordons off the whole machine (physical or virtual) by tenant as soon as
previous tenant resources have been fully consumed, then reclaims hosts after
usage subsides. So as a provider I still only have one cluster to manage, but
as a consumer I still don't worry about _another_ layer of abstraction slowing
things down or pre-allocating resources.

~~~
chatmasta
I'm interested in the economics of (docker) containers vs. virtual machines.
Containers can run within a VM, but a VM can only run within a hypervisor.

Currently, if you want to resell computing resources, you need to rent or buy
a dedicated server, and run a hypervisor on it.

Containers enable a new class of reselling computing resources. Because you
can run a container within a VM, you can resell computing capacity on a VM.

I think we are going to see another abstraction on top of "the cloud," due to
this additional layer of reselling (new russian doll on the inside, new doll
on the outside).

The physical abstraction is:

Datacenter > Floor Space > Server Rack > Server

The virtual abstraction is:

Server > VM > Container > Kubernetes|MESOS|...

Virtual is a 1:1 inverse of physical. Next step is datacenter interconnections
(i.e. multihost kubernetes or whatever flavor of the month IaaS software
people use).

~~~
derefr
People have been reselling containers without needing an intermediary VM
abstraction forever; look at any cheap VPS host offering OpenVZ-based "virtual
machines"—which are actually [resource-quota'ed] containers.

~~~
chatmasta
It's not a question of need; it's a question of ease of opportunity. It's now
easier to virtualize via container, and there are more opportunities, since
it's easier to get a VM than a dedicated server with hypervisor access.

------
markbnj
Can someone clarify how this compares to LXD, released by Canonical in Ubuntu
16.04? A lot of the keywords and concepts seem similar.

~~~
resouer
And another different side of HyperContainer is that it follows OCI spec,
check the runv project here:
[https://github.com/hyperhq/runv/](https://github.com/hyperhq/runv/), so
technically speaking, it's a hypervisor version of OCI, just like docker is a
linux container version of OCI. Seems rkt/clear linux or LXD does not.

~~~
shykes
Actually it's _runc_ that is the linux container version of OCI. Docker is a
higher-level abstraction which calls runc by default but can call any OCI-
compliant runtime, including runv.

See [https://blog.docker.com/2016/04/docker-
engine-1-11-runc/](https://blog.docker.com/2016/04/docker-engine-1-11-runc/)

~~~
resouer
It will be great to see if I can use dockerd start runv containers!!!!

------
Veratyr
Speaking of security on Kubernetes, it's worth noting that most of the
"Getting Started" guides (e.g. [0]) to help you set up a cluster result in
completely unauthenticated API servers.

This means that by default, anyone can do anything they want with your
cluster.

There are no warnings, no suggestions that turning on the much better TLS
based authentication would be a good idea (or even how to do it), no nothing.

Be __very __careful with Kubernetes.

[0]: [http://kubernetes.io/docs/getting-started-
guides/ubuntu/](http://kubernetes.io/docs/getting-started-guides/ubuntu/)

~~~
robszumski
Maintaining community-led documentation is a hard and time consuming process,
especially with a young project. I encourage you to get involved if you have a
few free cycles.

Sometimes you have to take a stance on these types of things, as we have done
with the CoreOS + Kubernetes community guides [0]. The guides are open source,
but full TLS, passing conformance tests, etc is required for contribution.

(I work at CoreOS)

[0]:
[https://coreos.com/kubernetes/docs/latest/#installation](https://coreos.com/kubernetes/docs/latest/#installation)

~~~
praneshp
The CoreOS documentation is a lifesaver (even when setting up k8s on a non-
coreOS system. Thanks a lot (and agree with the comments on
documentation/involvement).

------
jldugger
At a brief glance, this looks comparable to Magnum:

\- containers \- openstack \- multitenancy

~~~
scprodigy
Totally not!

------
resouer
Aha, the
chancellor([https://www.youtube.com/watch?v=PivpCKEiQOQ](https://www.youtube.com/watch?v=PivpCKEiQOQ))
will feel great to see that finaly he can eliminate IaaS/vms and use docker in
production env.

------
xbeta
Anyone knows whether there's similar thing for Mesos ?

~~~
scprodigy
A patch is available:
[https://issues.apache.org/jira/browse/MESOS-3435](https://issues.apache.org/jira/browse/MESOS-3435)

