The term Serverless has become synonymous with AWS Lambda. Decoupling from AWS has two benefits; it avoids lock in and improves flexibility.
The misnomer Serverless, is a set of techniques and technologies that abstract away the underlying hardware completely. Obviously these functions still run on “servers” somewhere, but the point is we don’t care. Developers only need to provide code as a function. Functions are then used or consumed via an API, usually REST, but also through message bus technologies (Kafka, Kinesis, Nats, SQS, etc.).
This provides a comparison and recommendation for a Serverless framework for the Kubernetes platform.
Edits:
- v1 02/09/18
- v2 02/09/18: Major edits to OpenFaas info thanks to @alexellisuk, Richard Gee, @kenfdev. Removed Serverless from comparison table thanks to @sebgoa.
- v3 10/09/18: Edits regarding Fn project thanks to @delabassee.
Comparison Table
Below is a table providing a high-level comparison of the different Serverless frameworks for k8s. They are segmented by popularity, stability, tooling, technology and useability.
The problem with opinions like popularity is that it can be hard to quantify. We have to resort to a proxy metic. For example, I’m really interested how many people use this framework on a day to day basis. But it is hard to find those numbers so we use a proxy metric like Github stars or Google searches. These are represented as <desired_metric (proxy_metric)>. See here for more information about project popularity.
Hence, please note that many of these metrics are gross estimates. Individual frameworks may appear better or worse than they really are. Read with a holistic view of all the evidence and features.
| Comparison/Framework | OpenFaas | OpenWhisk | Kubeless | Fission | IronFunctions | Fn |
|---|---|---|---|---|---|---|
| Popularity (Github Stars) |  |
 |
 |
 |
 |
 |
| Popularity (Relative google trends - 100 means most popular) | 37 | 58 | 24 | N/A (conflicts with fission) | N/A (conflicts with serverless) | 3 |
| Popularity (Total StackOverflow.com posts) | 21 | 359 | 15 | 2 | 5 | 9 |
| Stability (Contributors) |  |
 |
 |
 |
 |
 |
| Stability (Contributors > 10 Commits) | 10 | 33 | 7 | 6 | 9 | 19 |
| Stability (Corporate Backer) | VMWare (1) | IBM (Apache Foundation) (3) | Bitnami | Platform9 | https://iron.io | Oracle |
| Stability (Started) | Dec 2016 | Feb 2016 | Nov 2016 | Aug 2016 | Feb 2016 | May 2016 |
| Stability (Development Language) | Go | Scala | Go | Go | Go | Go |
| Tooling (Packaging mechanism) | Docker container | Docker container | Docker container | Docker container | Docker | Docker |
| Tooling (Deployment functions on k8s) | Manifest with custom yaml | Manifest with custom yaml | Manifest | Manifest with custom yaml | Proprietary | Proprietary |
| Tooling (Deployment via Serverless?) | Yes (WIP) | Yes | Yes | No | No | Yes |
| Technology (Underlying technology) | Alertmanager / Prometheus, Nats | CouchDB, Kafka, Nginx, Redis, Zookeeper | None (optional Nats or Kafka) | fluentd (optional Nats) | Postgres, Redis | DB (sqlite3, PostgreSQL, MySQL), MQ (Bolt, Redis), Prometheus |
| Usabillity (Worked out of the box?) | Yes | Yes | No (serverless plugin failed) | Yes (but annoyances) | Didn’t try | Didn’t try |
| Usability (Quality of documentation) | Good | Good | Average (not well organised) | Poor (not well organised, strange navigation and missing documentation) | Average (not well organised) | Good |
| Useability (Slack channel?) | Yes but through email | Yes | Yes (#kubeless channel of the k8s slack: http://slack.k8s.io) | Yes | Yes | Yes |
- Team of developers from VMWare work full time on OpenFaas.
- Serverless included due to the recommendation for SDK.
- Used in IBM’s Cloud Functions offering
Serverless Framework Recommendation
Using this table as a comparison, I can recommend:
- Using the Serverless framework as a software development kit (SDK)
- Using OpenFaas or OpenWhisk as the orchestrator of functions on k8s.
- OpenFaas is mature, easy to use and scalable, but when compared to OpenWhisk has fewer active developers on the core project (according to my definition of active developers) and is less popular (according to my choice of popularity metric).
- OpenWhisk is mature, backed by many active developers, popular, but is complex, written in Scala and backed by IBM/Apache (may be a good thing or a bad thing depending on your opinion)
The resulting technology stack looks like:
The additional recommendation of the Serverless framework allows developers to choose whether they want to deploy to lambda or to a serverless platform deployed on k8s. This helps migration if there are already functions on lambda.
Framework Notes
OpenWhisk
OpenWhisk is a mature Serverless framework supported by the Apache foundation and backed by IBM. OpenWhisk forms the basis of IBM’s Cloud Functions service. The main committers are IBM employees.
There are many underlying components, which adds to the complexity. It leverages CouchDB, Kafka, Nginx, Redis and Zookeeper. The upside is that the developers have clearly focused on scalability and resiliency. The downside is that Developers and Operators would require a working knowledge of these tools. Another downside is that they are duplicating features that are present in a orchestrator like Kubernetes (e.g. autoscaling). Functions are ultimately bundled into Docker containers that run alongside the framework.
OpenWhisk can be installed using a Helm chart but unfortunately requires some manual intervention. Applications can be deployed using the CLI or with the Serverless framework. Prometheus metrics are exported out of the box.
OpenFaas
OpenFaas is a popular (not quite as popular as OpenWhisk according to my estimates of popularity), easy to use Serverless framework. It is not as popular as OpenWhisk and commits are made on an individual basis. VMWare employs a team to work full time on OpenFaas, in addition to the great work performed by individual contributors in their spare time. A company, OpenFaas Ltd., has been incorporated in the UK, although it is unclear how this company relates to the project.
The architecture of OpenFaas is relatively simple. The API Gateway can be invoked synchronously or asynchronously via Kafka, SNS, CloudEvents, CRON and other triggers. Asynchronous invocations are handled by NATS Streaming. The autoscaling is performed using Prometheus and Prometheus Alertmanager; but thankfully it appears that this can be swapped out to use Kubernetes’ HorizontalPodAutoscaler.
Fully supported Kubernetes installer available via Helm or kubectl including an Operator allowing use of CRDs i.e. kubectl get functions. There is also a Kubernetes operator that is WIP, but I found that this worked well.
Applications can be deployed using the CLI or with the Serverless framework (a WIP). A “Function Store” is also provided, a curated list of functions to use with OpenFaas. Prometheus metrics are exported out of the box.
OpenFaaS ® is a registered trademark within the U.K.
Kubeless
I was really excited about Kubeless, the function-native framework for Kubernetes. It works by adding the idea of a “function” as a custom resource definition (CRD) in Kubernetes. Along with some clever code, this means that it turns Kubernetes into a function machine, with no add-on complexities like a messaging bus like the other frameworks.
I love the ability to manage functions like standard Kubernetes objects which means that all the usual Kubernetes goodies work out of the box (Helm, Ark, etc.).
Interaction is through the standard kubectl so no extra tooling and it has Serverless support built in.
It sounds perfect!
But unfortunately it just isn’t mature enough to be used in production yet. The community isn’t large enough, the documentation isn’t up to scratch (have to rely on blog posts) and there were bugs with the Serverless support which meant it didn’t work on EKS.
Given the positive fundamentals, I am sure that this will become the defacto Kubernetes serverless framework within the next six months.
Fission
Fission was interesting because it sits somewhere between Kubeless and OpenWhisk. It’s heavily dependent on Kubernetes features, but it’s not completely integrated. Then benefit of this approach is that it can take advantage of things that Kubernetes is good at, like autoscaling, but does something different to get better performance when needed. For example, it has a quite sophisticated cold-start pooling mechanism.
It is backed by Platform9 and can be installed via Helm. It uses Influxdb to handle state and provides FluentD for logging out of the box. It uses Nats for it’s message bus and Redis for caching. As you can see, other frameworks don’t provide caching and logging out of the box, even though it would be reasonably simple to add it.
Fission has a really nice extra called Fission Workflows. this is a tool that allows developers to compose functions. A la functional programming. This is a really interesting direction and I’m keen to see what can be done with it.
However, there are very few users (only two stack overflow questions - does that mean it’s easy to use?) and very few serious contributors - only 6 people have more than 10 commits. There was a few annoyances, but that’s mainly due to lack of users and developers. And the documentation was poorly organised. This made it hard to figure out how the framework worked. I also didn’t like how the code was responsible for templating and starting pods itself; this is likely to cause breakages in the future. Finally, I am very confused about Fission builders. I could not figure out what they were, or why they are required.
Also, the name makes it difficult to search for.
Fn
Another contender for the silly name crown is Fn. It’s Open Source but the main contributors work for Oracle. The main workflow uses the Fn CLI, but under the hood the functions use Docker containers. There is some information in this blog post and the documentation is here. And it is possible to deploy the framework components with Helm. There is also a new addition called Fn Flow that orchestrates multiple functions similar to OpenWhisk’s workflows.
But the most important difference is the way in which you work. Fn is focused on being easy to use, but this makes it very opinionated. It provides hot functions (all the other frameworks can provide this too) and streaming functions (this is more unique - not entirely clear how this would work with other frameworks).
It was started in 2016, which makes it as old as OpenWhisk and has a decent number of contributors. Despite this I was put off by the amount of opinion enforced on developers and this opinion doesn’t fit well with Kubernetes (you can’t deploy with Kubernetes manifests AFAICT). This was a requirement for me, so I didn’t try it. However, it is compatible with the Serverless framework which could mitigate this a little.
IronFunctions
Iron Functions is supported by a company of the same name. This added some initial complication since the github readme redirects you to the “documentation” which is actually the main page for Iron the company. From there if you click on “docs” then there is no mention of Iron Functions. The real docs are in the docsdirectory of the repository.
It’s Docker based, like many of the other frameworks and one interesting feature is that it has like-for-like support for AWS Lambda functions. You can take your code from Lambda and run it straight away on Iron; that’s nice for migration.
Unfortunately it doesn’t appear to have native support to deploy manifests to Kubernetes, like Fn, and it wasn’t supported by the Serverless framework. This put me of enough not to try it. It also wasn’t popular enough to appear on Google trends.
Severless Framework
I’ve mentioned this a lot through the article, so it deserves a section in its own right.
It is not a platform, it does run any functions. It is a software development kit for serverless. In fact, it is essentially just a packaging mechanism. But the beauty lies in the fact that once packaged in the Serverless way, you can deploy the same code to Lambda, Google Functions, Azure Functions, OpenWhisk, OpenFaas, Kubeless or Fn.
This portability is very attractive. It allows developers to build their code in standard way, but still allow standards, pricing, feature set or usability to decide where to deploy it.
Furthermore, it allows us to defer the choice of framework somewhat. I mentioned earlier that I like the direction that Kubeless is taking with respect to Kubernetes, but it is not mature enough for use. If we use Serverless then we can build code on OpenFaas or Lambda now, and easily port to Kubeless later.
The only downside is the King Kong of stupid names. Every. Single. Time. I have to postfix “Serverless the framework, not the technology”. That and the limited number of supported languages. But other than that, I think this is the safest option to take.
Funktion
Funktion was a defunct solutions from RedHat.