OpenStack Edge Hackathon

 

On the weekend before the 2018 OpenStack summit in Berlin, I attended a hackathon held at hub:raum, a German co-working and event space, and startup accelerator. The building where hub:raum is located is one of the key Internet locations in Germany, so it’s apt that the event was held there.

We came together to hack on OpenStack or anything related to OpenStack, “the edge,” or both. Hackathons can be many different things, but generally, the idea is to bring a diverse group of people together to work on brand new ideas and technologies over a short time span, usually a day or two. The goal isn’t to build something that can be immediately useful, instead to explore ideas and see where they go.

Several projects were proposed and were worked on over the course of the event, but I’ll only discuss a couple here.

IoT, AI/ML and Home Heating

The first was a project to perform analysis on data that was gathered from various IoT devices which were monitoring a home heating system, made up of a wood oven, solar power, and several staged water storage tanks. The data was gathered over 1.5 years. The team spent much time cleaning up the data, analyzing it using standard methodologies, and then moved on to applying artificial intelligence and machine learning algorithms. They reported their results, which were promising, but didn’t quite get to the point where they could predict proper settings for the oven to better manage utilization. One could imagine how powerful it would be to have many IoT devices providing data across households and how resources could be conserved by properly managing and predicting settings for these devices.

OpenStackoid – Thousands of Edge Nodes

The second project was the one I participated in. My colleagues from Inria, who have been researching OpenStack and the edge for some time, put forward an idea tentatively termed “OpenStackoid.” The motive for this project was around exploring issues using OpenStack on “the edge.” In this case, by “the edge” we mean having hundreds or thousands of individual OpenStack clouds available for use.

OpenStack has always had the concept of regions, but typically these require shared state, which would work for a few clouds but not thousands. However, thousands of entirely individual clouds can be wasteful regarding managing resources, for example, Glance images. If we were to store the same Glance image in thousands of clouds that would use considerable resources and cause synchronization problems.

As we discussed the project, we settled on a use case where to request resources from different clouds. For example, say you wanted to create a virtual machine in cloud-1 but use an image from cloud-999.

Here’s an example command that might better illustrate what we were trying to do.

openstack server create --image cirros --flavor m1.small --scope “{keystone:cloud-1;glance:cloud-999}” cirros1

The scope option is what we desired to add to the OpenStack CLI, and we would parse that scope and use it for resource requests.

Our team had a lot to do, so we split the project into four main pieces of work:

1. Develop a domain specific language to express resources and scope
2. Configure the OpenStack CLI to be able to provide this information in requests
3. Develop a HAProxy and Lua based system to proxy requests to various clouds
4. Build the infrastructure we could use to test

Together with a couple of other colleagues, I worked on part 3, the proxy component. We were able to intercept requests, interpret the scope, alter the requests (using Lua) and proxy them to the appropriate clouds. This project was the definition of the term hack as we monkey patched the OpenStack CLI and injected changes into requests through HAProxy and Lua.

After two days of work, we completed the initial work on all four significant pieces but unfortunately ran out of time to integrate everything to build a virtual machine in one cloud using resources in another. Over the summit, we hope to continue discussions and work and see where it goes.

Moving to the Edge

OpenStack is a vast project, perhaps second only to Linux in terms of size and scope in the open source world, and it’s in the process of pivoting to “the edge.”  From my perspective, It’s critical that OpenStack support edge use cases, and I think the project we worked on gave us all a better view of what could be done.

It was a pleasure to meet everyone at the hackathon. The event space and staff were terrific, and it was great to collaborate with a happy group of new people and hack on OpenStack. I certainly learned a lot, and hopefully, the lessons we learned will be discussed over the rest of the summit and can help OpenStack with its journey to the edge.