The Platform Engineer's Quest

Hoo hoo! Welcome, brave platform engineer. I’ve been expecting you.

Your CTO has given you a critical mission: “Provide Kubernetes to the company. Don’t use the cloud. A certain administration across the pond keeps me up at night. Do it on-prem. Here’s the budget, go buy some servers.”

First, you talk to the teams. Your platform team wants shared staging and prod clusters (isolated from each other for security), plus a central one for observability and GitOps. Data science needs its own for their higher-privilege operators. That’s 4 clusters minimum.

You waddle over to the infra team. Datacenter hardware has settled on big virtualization hosts: smaller nodes waste power, cooling, and network ports. Best price/performance starts at 512 GiB of RAM upwards. They order 10 bare metal servers, 64-core beasts with 512 GiB of RAM each, and rack them.

In the cloud, you’d just spin up EKS or GKE clusters as needed. But on-prem, buying dedicated servers per cluster doesn’t scale: each box is way too big for a single cluster’s workers or control plane, and every new cluster means another hardware order (or expensive spares lying around). Cramming everything into one big shared cluster kills the isolation your teams need.

So, how do you split up the big beefy hosts into Kubernetes clusters? Reach for VMware, Proxmox, OpenStack so you can create VMs? Or is there a cloud-native way to get GKE-style Kubernetes on-prem?

Let’s enter the quest. By the end, those 10 servers will quietly power a multi-cluster Kubernetes platform that rivals the cloud, on your own metal. Your CTO sleeps easy. Not a single packet crosses the Atlantic.

Before You Begin

You’ll need:

• A laptop with at least 26 GiB free RAM (i.e. a 32 GiB laptop) and nested virtualization support:
  • macOS: Apple Silicon M3 or later with macOS 15+ (M1/M2 and Intel Macs don’t support nested virtualization)
  • Linux/Windows: most modern x86_64 CPUs (Intel VT-x or AMD-V)
  • No suitable laptop? Use a cloud VM with nested virtualization instead. Launch a recent Ubuntu on Azure Standard_D8ds_v5, AWS r8i.xlarge, or GCP n2-standard-8 and follow the Linux (QEMU/libvirt) guide. On AWS and GCP, make sure to explicitly enable nested virtualization on the instance.
• A virtualization tool for your OS. We have step-by-step guides for each:
  • macOS: tart (recommended)
  • Linux: QEMU/libvirt
  • Windows: Hyper-V
• Your owl USB stick containing the Nest installer ISO
• A stable internet connection: the Nest installation downloads around 10 GB of data. Don’t do it on a train.
• About 35 GiB free disk space on your laptop
• kubectl and helm installed on your machine
• About 40 minutes of your time

Copy the Nest installer .iso from the USB stick to your laptop. Pick the right architecture: nest-installer-v*-arm64.iso for Apple Silicon Macs, nest-installer-v*-amd64.iso for Intel/AMD Linux and Windows laptops.

For this quest, we’ll simulate your bare metal servers with 3 VMs on your laptop. Don’t worry: every click, every concept is identical to what you’d do in production. Everything works the same on 10 Dell R7625s or three VMs on your MacBook. (Your MacBook’s battery may disagree.)

Chapter 1: Hatch Your Nest

Estimated time: 15 minutes

Every great platform starts with a Nest: the management appliance that hosts your Kubernetes control planes and Foundry, the web UI. Think of it as the command center for your 10 servers. Without it, they’re just expensive space heaters.

The Nest takes the first of your 10 servers; the other 9 will join the platform cluster in Chapter 2.

Create and boot the Nest VM:

1. Create a Nest VM by following your hypervisor guide (macOS, Linux, Windows):
   • CPU: 4+ vCPUs
   • RAM: 10 GiB
   • Disk: 200 GiB (thin-allocated, won’t use more than 20 GiB)
   • Boot: Attach the Nest installer .iso
2. Start the VM: the Nest TUI installer will appear.
3. Follow the installer prompts:

On a local VM the installer is designed to be a next → next → finish click-through: confirm the defaults and suggested IPs/hostnames. If you get stuck on any screen, check the Install Nest reference for a detailed walkthrough.

Except Hyper-V users: in the Networking tab, select Static IPs instead of DHCP. Hyper-V’s Default Switch doesn’t persist DHCP leases across reboots. The suggested static IP settings work fine.

4. Wait for the installation to complete (~10 GB download + cluster setup). This can take 10–15 minutes.
5. Once finished, the installer shows you the Foundry URL (typically https://app.192.168.x.y.d.meltcloud.io, where x.y is your VM’s IP). Note it down!

Access Foundry:

6. Open your browser and navigate to the Foundry URL.

Splendid! You’re looking at Foundry, your meltcloud command center. From here, you manage every cluster and every machine. Think Azure Portal or GCP Cloud Console, only the metal is yours.

Notice the sidebar: Clusters, Machines, Enrollment Images, Elastic Fleets. We’ll touch all of these.

From here on, each step offers two paths: Foundry (the web UI) and Terraform (infrastructure as code). Look for the tab switcher.

terraform {
  required_providers {
    meltcloud = {
      source = "meltcloud/meltcloud"
    }
    time = {
      source  = "hashicorp/time"
      version = "~> 0.11"
    }
  }
}

provider "meltcloud" {
  # Your Foundry URL from Chapter 1
  endpoint        = "https://app.192.168.x.y.d.meltcloud.io"
  
  # From the Foundry URL: app.192.../ui/orgs/<uuid>/...
  organization    = "your-organization-uuid"
  
  # Either export MELTCLOUD_API_KEY env var, or: 
  # api_key = "eyJ"

  # the local Nest installer uses self-signed certificates
  skip_tls_verify = true
}

terraform init

Chapter 2: The Platform Cluster

Estimated time: 15 minutes

Now for the interesting part: your 9 worker servers don’t get divided up front and assigned to disparate clusters. Instead, they all join one Kubernetes cluster: the platform cluster. This is where you’ll run your platform tools (Argo CD, Prometheus, Grafana, your secrets operator).

Later, we’ll build the tenant clusters on top.

Create the platform cluster:

resource "meltcloud_cluster" "platform" {
  name           = "platform"
  version        = "1.35"
  pod_cidr       = "10.0.0.0/16"
  service_cidr   = "10.1.0.0/16"
  dns_service_ip = "10.1.0.10"
}

terraform apply

Notice what just happened: the Nest spun up a full Kubernetes control plane (kube-apiserver, etcd, scheduler, controller-manager) as containers inside the Nest. We call this a Hosted Control Plane (HCP): every cluster you create gets its own HCP, all running in the Nest. The control plane doesn’t eat a single CPU on your worker bare metal. The other 9 servers stay fully available for workloads. In production, you’d scale the Nest out to 3 instances across separate servers for HA.

Create a Machine Pool:

resource "meltcloud_machine_pool" "bare_metal" {
  cluster_id = meltcloud_cluster.platform.id
  name       = "bare-metal"
  version    = "1.35"
}

terraform apply

A quick word on how machines join the platform: enrollment. You generate an Enrollment Image in Foundry (a bootable ISO), boot a bare metal server (or a VM) from it, and the image installs our immutable Linux distro and joins the machine to the Nest. You don’t need to install an OS, SSH bootstrap, or run Ansible.

From that moment, the machine is fully managed, just like in the cloud: patches arrive automatically, every config or version change rolls out as a controlled reboot, and there’s no drift. For the deep dive on bare metal workers, see From Metal to Kubernetes.

Create an Enrollment Image:

resource "time_offset" "enrollment_expiry" {
  offset_days = 30
}

resource "meltcloud_enrollment_image" "quest" {
  name                = "quest-enrollment"
  expires_at          = time_offset.enrollment_expiry.rfc3339
  
  # Pick one: 
  # 1. /dev/vda for macOS (tart) and Linux (libvirt) 
  install_disk_device = "/dev/vda"
  
  # or
  # 2. /dev/sda for Windows (Hyper-V)
  install_disk_device = "/dev/sda"
}

terraform apply

Enroll your bare metal (well, your VMs):

1. Create 2 new VMs by following your hypervisor guide (macOS, Linux, Windows, choose tab Machine on Linux, tab Machine (hosting Elastic Node Pools or KubeVirt) on macOS/Windows). In production this would be your other 9 servers, but for the quest, two is plenty:
   • CPU: 2+ vCPUs / RAM: 8 GiB / Disk: 30 GiB (thin-allocated, won’t use more than 10 GiB)
   • Boot from the enrollment .iso you downloaded above (copy it per VM if your hypervisor locks the file)
   • Enable nested virtualization (in Chapter 4 we’ll run Elastic Node Pools (Virtual Workers) inside these VMs)
2. Start the VMs. They enroll as described above; you can watch the TUI for progress.
3. In Foundry, go to Machines: your VMs should appear with status Ready (Unassigned)

Assign machines to the pool:

import {
  to = meltcloud_machine.worker1
  id = "machines/<worker1-id>" # most likely machines/1
}

resource "meltcloud_machine" "worker1" {
  uuid            = "<worker1-uuid>"
  name            = "worker1"
  machine_pool_id = meltcloud_machine_pool.bare_metal.id
}

import {
  to = meltcloud_machine.worker2
  id = "machines/<worker2-id>" # most likely machines/2
}

resource "meltcloud_machine" "worker2" {
  uuid            = "<worker2-uuid>"
  name            = "worker2"
  machine_pool_id = meltcloud_machine_pool.bare_metal.id
}

terraform apply

meltcloud clusters ship without a CNI by design: you pick whatever fits your team (Cilium, Calico, Flannel, your call). We’ll install Cilium here as a solid modern default.

Install Cilium on the platform cluster:

19. Download the Kubeconfig (Admin) for the platform cluster (from the cluster’s Info tab)
20. Connect and install Cilium as the CNI:

# Linux / macOS / WSL:
export KUBECONFIG=/path/to/platform-admin.kubeconfig

# Windows (PowerShell):
$Env:KUBECONFIG="C:\Path\To\platform-admin.kubeconfig"

# All OS
kubectl get nodes
# NAME      STATUS     ROLES    AGE   VERSION
# worker1   NotReady   <none>   3m    v1.35.x
# worker2   NotReady   <none>   3m    v1.35.x

Both nodes show NotReady - expected, no CNI yet. Install Cilium:

helm install cilium oci://quay.io/cilium/charts/cilium --version 1.19.4 -n kube-system

A minute later:

kubectl get nodes
# NAME      STATUS   ROLES    AGE   VERSION
# worker1   Ready    <none>   5m    v1.35.x
# worker2   Ready    <none>   5m    v1.35.x

Hoo! Your platform cluster is alive. In a real deployment, this is where you’d install Argo CD or Flux to manage your platform GitOps, Prometheus & Grafana for cluster-wide observability, an external secrets operator, certificate management, whatever your platform team uses to operate the rest of the org’s infrastructure.

This single cluster is your platform. Every team’s workloads will eventually run on the same metal, but isolated in their own clusters. Let’s see how.

Chapter 3: The Tenancy Map

Estimated time: 3 minutes

Remember the teams you talked to earlier? Time to turn those conversations into a tenancy map.

Your platform team wants shared staging and prod clusters (isolated from each other for security), plus a central one for observability and GitOps. Data science needs its own for their higher-privilege operators. Here’s the map you end up with:

The naive option: buy more hardware, one set of servers per cluster. Wasteful, and the dedicated boxes sit half-idle most of the time.

Instead of dedicating separate hardware to each cluster, we build them virtualized on top of the platform cluster’s metal. The meltcloud option is Elastic Fleets: turn your 9 platform workers into a set that all your other clusters can carve workers from, virtually. One set of metal underneath, properly isolated clusters on top. Let’s build it.

Chapter 4: Elastic Fleet to Slice the Metal

Estimated time: 8 minutes

Time to build the fleet. An Elastic Fleet turns your platform cluster’s 9 workers into a set of virtualization capacity: the same physical machines can now host virtual Kubernetes workers for any other cluster, on demand.

Under the hood: KubeVirt (the CNCF-graduated project for running VMs on Kubernetes), where each virtual worker runs our immutable Linux distro, just like the bare metal Machines.

Create the Elastic Fleet:

resource "meltcloud_elastic_fleet" "platform_fleet" {
  cluster_id = meltcloud_cluster.platform.id
  name       = "platform-fleet"
}

terraform apply

This takes 1-2 minutes while the virtualization stack starts up. The Nest is rolling out KubeVirt and the virtualization stack across all workers of the platform cluster. Your bare metal nodes can now host VMs. Your platform tools (Argo, Prom, …) keep running side-by-side: containers and VMs share the same metal.

Define Quotas:

resource "meltcloud_elastic_quota" "tenant_clusters" {
  elastic_fleet_id            = meltcloud_elastic_fleet.platform_fleet.id
  consuming_organization_uuid = "your-organization-uuid"
  name                        = "tenant-clusters"
  vcpus                       = 6
  memory_mib                  = 16384
  disk_gib                    = 100
}

terraform apply

Notice this is a quota, not workers. You’ve reserved a chunk of fleet resources for clusters created later. In a true multi-tenant setup, you’d define quotas for other organizations (say, a dev or a data org) and their engineers would self-serve their own clusters from that quota.

For this quest, we’re keeping it simple: the platform team is the only org, and it defines a quota for itself.

Chapter 5: Cluster Magic Starts

Estimated time: 10 minutes

Now the magic. Each cluster on your tenancy map gets a real Kubernetes cluster (its own control plane, its own workers, its own kubeconfig, its own kubectl context, full isolation), but the workers are created as KubeVirt VMs on the Elastic Fleet you just built. All the benefits of virtualization, without the enterprise vendor lock-in (or the bill). The underlying hardware is used efficiently and can be extended with new workers as needed.

Let’s start with shared-staging. The other two are the same playbook.

Create shared-staging:

resource "meltcloud_cluster" "shared_staging" {
  name    = "shared-staging"
  version = "1.35"
}

terraform apply

This is identical to creating the platform cluster in Chapter 2: an HCP control plane spins up in the Nest. No worker assignment yet: that’s the next step.

Create an Elastic Node Pool (the cluster’s workers):

resource "meltcloud_elastic_node_pool" "staging_workers" {
  cluster_id       = meltcloud_cluster.shared_staging.id
  elastic_quota_id = meltcloud_elastic_quota.tenant_clusters.id
  name             = "workers"
  version          = "1.35"
  node_count       = 2

  node_config {
    vcpus      = 1
    memory_mib = 1536
    disk_gib   = 10
  }
}

terraform apply

Within minutes, virtual workers are provisioned on top of your bare metal pool. Download the admin kubeconfig for shared-staging and connect:

# Linux / macOS / WSL:
export KUBECONFIG=/path/to/shared-staging-admin.kubeconfig

# Windows (PowerShell):
$Env:KUBECONFIG="C:\Path\To\shared-staging-admin.kubeconfig"

# All OS
kubectl get nodes
# A brand-new virtual node, sized exactly to shared-staging's needs.

Optional: if you want to install a CNI (and potentially later some workloads) on the new cluster, you can install Cilium here:

# different VXLAN port to avoid conflict with the outer cluster's Cilium
# lower MTU to account for the outer encapsulation overhead

# Linux / macOS / WSL:
helm install cilium oci://quay.io/cilium/charts/cilium \
  --version 1.19.4 \
  -n kube-system \
  --set tunnelPort=8473 \
  --set MTU=1400

# Windows (PowerShell):
helm install cilium oci://quay.io/cilium/charts/cilium `
  --version 1.19.4 `
  -n kube-system `
  --set tunnelPort=8473 `
  --set MTU=1400

Now repeat for the other two planned clusters:

resource "meltcloud_cluster" "shared_prod" {
  name    = "shared-prod"
  version = "1.35"
}

resource "meltcloud_elastic_node_pool" "prod_workers" {
  cluster_id       = meltcloud_cluster.shared_prod.id
  elastic_quota_id = meltcloud_elastic_quota.tenant_clusters.id
  name             = "workers"
  version          = "1.35"
  node_count       = 2

  node_config {
    vcpus      = 1
    memory_mib = 1536
    disk_gib   = 10
  }
}

resource "meltcloud_cluster" "data_science" {
  name    = "data-science"
  version = "1.35"
}

resource "meltcloud_elastic_node_pool" "data_science_workers" {
  cluster_id       = meltcloud_cluster.data_science.id
  elastic_quota_id = meltcloud_elastic_quota.tenant_clusters.id
  name             = "workers"
  version          = "1.35"
  node_count       = 1

  node_config {
    vcpus      = 1
    memory_mib = 1536
    disk_gib   = 10
  }
}

terraform apply

That’s it. Three new clusters in about 10 minutes total. Each fully isolated, each sized for its job, all sharing the same 9 bare metal servers underneath.

Look at what you’ve built. Four isolated Kubernetes clusters (platform, shared-staging, shared-prod, data-science), each with its own HCP hosted in your Nest, one bare-metal worker pool underneath, three elastic node pools carved out of it.

Chapter 6: An Unexpected Request

Estimated time: 2 minutes

Just as you’re about to celebrate: your Slack pings. It’s the CTO:

“Loved the demo. New directive: every engineer in the company should learn Kubernetes hands-on. I want a sandbox cluster where teams can poke around, break things, try new operators, no risk to production. Can you have it ready by Friday?”

You smile. You’ve got a truly elastic multi-cluster platform now, so this is just one more cluster on the same metal.

Create the education cluster:

resource "meltcloud_cluster" "education" {
  name    = "education"
  version = "1.35"
}

resource "meltcloud_elastic_node_pool" "education_workers" {
  cluster_id       = meltcloud_cluster.education.id
  elastic_quota_id = meltcloud_elastic_quota.tenant_clusters.id
  name             = "workers"
  version          = "1.35"
  node_count       = 1

  node_config {
    vcpus      = 1
    memory_mib = 1536
    disk_gib   = 10
  }
}

terraform apply

That’s it. 30 seconds of clicks - or one terraform apply. Five clusters now share the same 9 worker servers.

That’s the whole point of a platform. The next cluster (for whatever the org dreams up) is just a click or a terraform apply away.

Chapter 7: The Aftermath

Estimated time: 3 minutes

The good platform engineer you are knows the truth: when the CTO presentation ends, the work is done for everyone except you. Welcome to Day 2.

Patching, scaling, backups, capacity planning: you know, the boring work. All of it, automated by the platform. That’s what we meant back at the start when we said GKE-style Kubernetes on-prem that rivals the cloud.

Stays current, automatically:

• OS patches every month. Your immutable Linux distro gets a fresh image; machines reboot in a controlled rollout during the maintenance window you configure.
• Kubernetes patches every month, plus new minors as they ship. Both HCPs and workers stay on a supported version with a single click in Foundry or automatically in a chosen maintenance window.

Resilient by default:

• etcd backups for every HCP. Snapshots are scheduled and stored automatically; restore is a documented one-click operation.
• Control plane autoscaling. Each HCP scales with its cluster’s load without manual tuning.

Try it now:

1. Enroll another machine. Boot a fresh VM (or a real bare metal box) from the same quest-enrollment ISO and watch it join the platform Machine Pool within a minute. Your Elastic Fleet capacity grows automatically.
2. Scale an Elastic Node Pool. In Foundry, bump the worker count on any pool, or do it with terraform apply if you prefer GitOps. The pool reconciles, a new KubeVirt worker boots in seconds, the cluster grows.
3. Set a maintenance window. Pick when the next OS or Kubernetes patch wave hits your fleet. The platform handles cordoning, draining, and rolling reboots.

Victory!

Hoo hoo hoo! Quest complete, platform engineer.

You took 10 bare metal servers and turned them into a multi-tenant Kubernetes platform that would make any cloud provider jealous. Your devs have shared staging and prod, your data scientists have their dedicated cluster, your CTO has an education sandbox, and your platform team has the home cluster where Argo and observability live: everything managed by the Nest, including the OS on those servers.

What you’ve accomplished:

• Stood up the Nest management appliance from a USB stick
• Built a platform cluster by enrolling bare metal servers with zero OS install
• Hosted 5 HCP control planes in the Nest: no metal wasted on control planes
• Created an Elastic Fleet that turned your metal into a virtualization pool
• Carved Elastic Node Pools for shared-staging, shared-prod, data-science, and an ad-hoc education cluster
• Delivered self-service Kubernetes clusters to your entire company

Where to go from here:

• Platform Overview: the full architecture
• Elastic Node Pools concept: deeper on the fleet/quota/pool model
• Install on bare metal: the same workflow on real hardware
• Terraform Provider: manage everything as code
• Get in touch: we’d love to hear about your infrastructure!