Step By Step: Run Local Kubernetes Cluster, Change Source Code, and Test

Kubernetes is a big project with many contributors. Unfortunately, to contribute the bootstrap for compiling and testing the code with an actual  kubernetes server  up is not easy.  The documentation is complex, not always working, and somewhat outdated. Moreover, it does not give all the details for you to start from zero into a working local  kubernets cluster  with code, with an example of a source file change and compile and run.  This is exactly what we are going to do here!

Step 1: Create a VM and Access It

We have promised to start from zero, right? So, we are going to create a new, clean VM and run it in this first step. 

1. Connect to: https://console.cloud.google.com.

2. Click "Create New Project."

3. Type "kubernetes-test"

4. Click "Compute Engine."

5. Click "VM Instances."

6. Click "Create."

7. Choose  Ubuntu16.04  OS and give it a generous amount of memory. I have chosen  26GB of memory- Kubernetes compilation loves memory, otherwise it would fail.  See the image below for my VM creation.

8. Click "ssh" --> "open in browser."

9. You are now logged into your new Google cloud platform VM instance!

Figure 1: This is what your VM configuration should look like- pay special attention to be sure you have enough memory.

Step 2: Prepare VM for Kubernetes

We are going to install:

1. GCC, Make, socat, and git.

2. Docker.

3. etcd

ssh to your VM and then install GCC, Make, socat, and git: 

# Install compilation and source code git tool
sudo su - 
 apt-get install -y gcc make socat git

Install etcd :

# Install etcd
curl -L https://github.com/coreos/etcd/releases/download/v3.0.17/etcd-v3.0.17-linux-amd64.tar.gz -o etcd-v3.0.17-linux-amd64.tar.gz && tar xzvf etcd-v3.0.17-linux-amd64.tar.gz && /bin/cp -f etcd-v3.0.17-linux-amd64/{etcd,etcdctl} /usr/bin && rm -rf etcd-v3.0.17-linux-amd64*

Install golang (should be 1.8+):

# Install golang
curl -sL https://storage.googleapis.com/golang/go1.8.linux-amd64.tar.gz | tar -C /usr/local -zxf -

# Add it to your PATH
export GOPATH=/gopath
export PATH=$PATH:$GOPATH/bin:/usr/local/bin:/usr/local/go/bin/

Step 3: Get The Kubernetes Source Code

In this step, we are going to clone the Git Kubernetes source code and add it to  GOPATH . We are going to use  --depth 1 so that we do not fetch the whole history, only the latest revision.

# Get kubernetes source code
git clone --depth 1 https://github.com/kubernetes/kubernetes

Now add Kubernetes sources to GOPATH :

# Add kubernetes source code to GOPATH
git clone https://github.com/kubernetes/kubernetes $GOPATH/src/k8s.io/kubernetes
cd $GOPATH/src/k8s.io/kubernetes

Step 4: Compile and Run Kubernetes

This is easy as they have a utility called local-up-cluster.sh 

# Compile and run kubernetes (will take some time)
export KUBERNETES_PROVIDER=local
hack/local-up-cluster.sh

The output should be:

# Successfull output for kubernetes compile and start
Cluster "local-up-cluster" set.
use 'kubectl --kubeconfig=/var/run/kubernetes/admin-kube-aggregator.kubeconfig' to use the aggregated API server
kubelet ( 15806 ) is running.
Local Kubernetes cluster is running. Press Ctrl-C to shut it down.
Logs:
  /tmp/kube-apiserver.log
  /tmp/kube-controller-manager.log
  /tmp/kube-proxy.log
  /tmp/kube-scheduler.log
  /tmp/kubelet.log
To start using your cluster, you can open up another terminal/tab and run:
  export KUBECONFIG=/var/run/kubernetes/admin.kubeconfig
  cluster/kubectl.sh
Alternatively, you can write to the default kubeconfig:
  export KUBERNETES_PROVIDER=local
  cluster/kubectl.sh config set-cluster local --server=https://localhost:6443 --certificate-authority=/var/run/kubernetes/server-ca.crt
  cluster/kubectl.sh config set-credentials myself --client-key=/var/run/kubernetes/client-admin.key --client-certificate=/var/run/kubernetes/clien
t-admin.crt
  cluster/kubectl.sh config set-context local --cluster=local --user=myself
  cluster/kubectl.sh config use-context local
  cluster/kubectl.sh

Step 5: Test That Kubernetes Is Up

Open a new shell to your VM and cd to your Kubernetes source directory in GOPATH  and test it:

# Connect to local k8s in a new shell
cd $GOPATH/src/k8s.io
export GOPATH=/gopath
export PATH=$PATH:$GOPATH/bin:/usr/local/bin:/usr/local/go/bin/
cd $GOPATH/src/k8s.io/kubernetes
export KUBERNETES_PROVIDER=local
cluster/kubectl.sh config set-cluster local --server=http://127.0.0.1:8080 --insecure-skip-tls-verify=true
cluster/kubectl.sh config set-context local --cluster=local
cluster/kubectl.sh config use-context local

# Test local k8s
cluster/kubectl.sh cluster-info
Kubernetes master is running at http://127.0.0.1:8080 # => Great!

Step 6: Change The Source Code

We are going to change an entry point in  kubernetes-apiserver  add a log line to it and see that we see it in logs:

// Edit api server server.go
// root@instance-3:/gopath/src/k8s.io/kubernetes# vi cmd/kube-apiserver/app/server.go

// Search for func Run(runOptions *options.ServerRunOptions, stopCh <-chan struct{}) error {

// Add this log line just before: if insecureServingOptions != nil {

        glog.Infof("HELLO FROM API SERVER") // We just added this!
        // run the insecure server now, don't block.  It doesn't have any aggregator goodies since authentication wouldn't work
        if insecureServingOptions != nil {
                insecureHandlerChain := kubeserver.BuildInsecureHandlerChain(kubeAPIServer.GenericAPIServer.HandlerContainer.ServeMux, kubeAPIServe
rConfig.GenericConfig)

Step 7: Run and Test Kubernetes Source Code Change

Stop your local cluster and restart it with  root@instance-3:/gopath/src/k8s.io/kubernetes# hack/local-up-cluster.sh 

Now tail API server log line and you will see this line:

W0412 16:29:06.603632   15625 genericapiserver.go:305] Skipping API autoscaling/v2alpha1 because it has no resources.
W0412 16:29:06.604382   15625 genericapiserver.go:305] Skipping API batch/v2alpha1 because it has no resources.
I0412 16:29:06.621028   15625 server.go:110] HELLO FROM API SERVER ### THIS IS OURS ### :)
I0412 16:29:06.621082   15625 insecure_handler.go:111] Serving insecurely on 127.0.0.1:8080
[restful] 2017/04/12 16:29:06 log.go:30: [restful/swagger] listing is available at https://10.128.0.2:6443/swaggerapi/

The documentation for starting up a local Kubernetes cluster from scratch then doing a source code update and testing it is somewhat lacking or incomprehensible.  We have just followed a few simple steps and fetched a Kubernetes cluster and compiled and changed its API server, then viewed our change.