European Cloud Modules
Advanced cloud services are based on good hardware, decent software, and surrounding infrastructure that combines these both into solid solutions that can be provided as a business activity.
Europe is good with operating the hardware. And surprisingly, we are also good with writing software. Much of the software used by the main cloud providers is based on open source, and lots of that open source is authored by European programmers. What we sorely lack here are providers of higher level cloud services, the kind that businesses clamor for.
To create credible advanced cloud providers here in Europe, we can build on our existing hardware/network/storage providers. And one could then base advanced services on existing (open source) software. But a big challenge is converting/completing that software into something you can successfully build a business on.
As a comparison, while I may have a coffee machine at home, that machine is not suitable for use in a coffee shop. At the core it does the same thing as a professional coffee machine, but you need a lot of modifications before a home coffee appliance will “scale” (and descale) to reliably make money in a store.
Individual European (hosting) companies are unable to uplift or author the required software themselves, or perhaps only very slowly. This is partially a matter of scale, but also a matter of not being seasoned software companies themselves. But one thing we can do well in Europe is cooperate.
In my earlier post “A coherent European/non-US cloud strategy: building railroads for the cloud economy”, I wrote:
“The idea outlined above is to create a portfolio of cloud-service-ready software that lots of companies could then turn into great cloud services, based on existing hardware/storage/network capabilities.
If we help create gold standard open source software for the most important cloud service, like key-value stores, S3-like services, AWS Cognito customer IAM (or IAM in general), we deliver a fertile landscape on which service companies could grow.”
Here I’d like to zoom in a bit on how that would work.
This article is part of a series of posts on (European) cloud challenges. In there it is also described how there are many kinds of cloud, and that Europe is good at the lower end of the stack (hardware, network, storage, operating system), but has little to offer at the higher end (AI, identity management, advanced databases etc). We have a big gap to bridge. What follows reads well with recent EuroStack publications that emphasise the role of industry to get us a better European cloud.
The missing features
We have some sterling open source work to start with. Lots of “cloud database services” for example are actually rebadged PostgreSQL services. All clouds are based on Linux, which is already open source.
Every US cloud provider takes these pieces of open source, and both privately and publicly uplifts them. The public part is where for example a company like Microsoft employs people to work on PostgreSQL full time. Amazon, Oracle, Apple and Google do similar things, for example for the Linux kernel, but also for many other projects.
Next to this public work however, all these cloud providers build proprietary infrastructure around these software products. They also add proprietary features that only they have, and which they are not contributing back. In this way, US clouds extract tons of value from public open source. (But it must be said, they also invest in it, MUCH more so than European players).
The infrastructure is where we have work to do. Open source/non-commercial projects love scratching their itch. We programmers gladly solve our own problems. We have very little interest in people’s business needs though. There is a huge difference between “I wrote a high performance web server” and “I also added support for a billing engine so you can make a profit”. The last bit will not happen automatically. Whereas you physically have to stop open source programmers from making software more performant!
There are other things missing too. If you have to support a single database, you are set. If you are operating a fleet of tens of thousands of databases, you need that database to contain specific functionality to make that doable. And in addition you need your own company management software to hook into the management support built into the database software.
A similar story can be told about compliance and audit trails. Many end-users will demand such things, but open source people are loath to spend time on these “enterprise” needs. Again, this will not happen spontaneously.
Open source security tends to not be too bad, but we mostly do this by caring a lot. Periodically someone sponsors an audit, and this helps. But it would be wonderful to have regular reports confirming things are good.
I could tell similar stories about documentation, tutorials, training, certification, logging, observability, (unit) testing, user-interface design & much more. There is a huge gap between “there is software that can deliver the functionality” and “you could build a massive cloud provider with this”.
And currently each cloud provider, including AWS, Google, Azure, has to bridge that gap themselves. Either by starting from scratch, or by uplifting existing open source software. That they all do so individually is massively inefficient, and also makes the big cloud providers’ offerings non-standard and not interoperable (which to them is a boon).
Note that it is not a great idea to “just” clone the big US providers. This would lead to the same problems, but now with a European flag. In what follows are also ways to “take your cloud services somewhere else”, or even take them home/back to your own office.
“European Cloud Modules”
But what if companies cooperated? We all need the same uplift. Cooperation among companies is very difficult in general, but for open source it is a well solved problem. Companies do not need to setup agreements with each other. Instead, they contribute to open source projects. Which is what many companies already do. And these projects then happen to be used by lots of people, who all benefit.
What I propose to do is gather places that want to become serious cloud providers, and have them commit to supply programmers, money and perhaps most crucially, guidance what they actually need to successfully sell a piece of software “as a service”.
With sufficient money and resources, this should enable us to uplift existing products in a way that benefits everyone. Or redo things where no good starting points are available.
Now, I’ve asked around, and here are some key services that are badly/weakly available among Europe’s cloud players:
- S3: I got some push back on this. Folks say ‘we have S3 at home’, but seriously, the big cloud S3 variants beat the pants of whatever we have at home
- AWS Simple Queuing Service (SQS)
- IAM between services (this is the big one)
- AWS Cognito equivalent
- Rock solid managed Kubernetes
- Managed database / RDS
- Reliably sending email/notifications (!)
Now, to anticipate the responses, lots of people will say “we DO have that”. But I urge operators to take an honest look at their offerings. One key thing I hear a lot is “we could build XYZ for you”. But we are competing with providers that don’t need to build anything for you. They offer it at the touch of a button. And that is the level we have to operate on as well.
Similarly, all these modules need to be observable using a common framework (which also allows for security monitoring), something no one offers here right now.
Additionally, the modules need some common capabilities, like for example all being able to use the IAM module. To this end we should do that very hard thing: agree on some standards. But by now this is so important that I trust we can figure it out.
If we work hard, we can uplift existing software to form useful “European Cloud Modules” which will allow lots of operators to turn themselves into credible cloud players.
Stefane Fermigier has written an analysis that delves deeply into this problem, where he covers the need for Technological Building Blocks, which are a more general idea than European Cloud Modules: Position Paper: A Structured Framework for Europe’s Digital Sovereignty: Technological Building Blocks for the Public Digital Infrastructure and Internet Commons.
How to get to these “European Cloud Modules”?
There are movements that think we can just order up a European cloud, and perhaps fund this all from Brussels. I’m not so sure we can do a ‘statist’ approach here, although governments could be very credible “lighthouse” customers of nascent European advanced cloud operators.
In addition, with legislation, European governments could drive business by making it flat out illegal for healthcare providers, national security places, critical infrastructure/data to rely on infrastructure controlled by (would be) tyrants and decaying democracies. Directed procurement, industrial policy, could definitely help here.
Meanwhile, instead of waiting for governments to unleash the cash, industry needs to commit to funding and supporting this process, with money, people and business engagement.
But how do we make sure that the “S3 Module” project actually picks the right software, the right goals, and delivers on the performance, management interfaces, documentation, tutorials, security, compliance, observability etc? That requires some very talented people to drive that all.
Key here is that European companies should stop penny pinching. Up to now, European cloud operators would feel real good about donating some money (“3k”) to an open source project. I’m a bit jaded here. I used to work over at PowerDNS, where we frequently found that billion euro revenue hosters would balk at paying us ten thousand euros for support. This was a common pattern across all of the industry.
If we want to make any headway, we need industry to pitch in millions of euros. And when they do that, the people that can drive a ‘European Cloud Module’ can break away from their corporate jobs, and follow their dream of working in open source. I know for a fact that folks are lining up to do this.
It is frequently said that money is not everything, but we can’t get to where we need to be by hoping good people will work on this stuff in their evenings.
Meanwhile, there are existing “midwives” that could help distribute funds, like cascading funders NLNet, or the German Sovereign Tech Agency. Inspiration can also be found with the Sovereign Cloud Stack.
So in short, pay real money, offer a glorious open source challenge, and real talent will arrive. And then these people can apply their skills to uplifting open source projects into credible European Cloud Modules, while benefiting the open source projects themselves too with higher performance, better documentation, improved security etc.
Some open source projects will already have ’evening talent’ that can be turned into full time staff by providing funding. Other projects will need to attract new people.
Won’t (European) cloud providers just take advantage of all this great software?
Big US clouds have invested a lot in open source, but they’ve also flat out killed a number of projects. Why buy services from an open source company when you can get those same services cheaper from AWS? This destroys the open source company.
This indeed sets up tension. A current example is the open source S3 implementation Min.io, which is now being heavily branded as “AIStor”. It does not look that much like an open source project anymore, and more like a commercial company that has some legacy open source stuff. Now I think this is great for them, go for it. There are bills to pay.
But that likely does not make them an enthusiastic Cloud Module partner.
The best partners are probably projects that can get themselves funded purely by users of their software, and do not have to turn themselves into ‘as a service’ providers.
A great existing example of a European Cloud Module might be Kamaji by the company Clastix, which provices Managed Kubernetes Services. Currently used by OVH, IONOS, Aruba, and others.
And if European cloud operators want to survive, they had better pony up some serious money and play ball. And yes, working on this will also benefit competitors. But individually, everyone will sink.
How much money?
So that’s the interesting thing. People bandy about astounding amounts of money for creating a European cloud. Billions. Dozens of billions. HUNDREDS of billions of euros.
Meanwhile, providing an open source software project with 5 million euros/year in funding would be a totally life altering experience for most projects.
And this level of funding is well within the reach of European cloud hopefuls, and could build the railroads on which the European cloud can run.
Meanwhile, EU and member state governments can easily provide game changing amounts of monetary support, which could certainly speed things up. But it is not feasible to sink billions/year into software development in Europe - we don’t have that many people to type that all in!
A change - don’t clone US clouds to end up with the same problems
US clouds are all geared towards lock-in. Customers don’t just rent servers over at the big cloud providers, they get locked in to proprietary services that are extremely hard work to get out of again. I described this in the post “‘The cloud’ is not just servers. ‘Going to the cloud’ could also mean locking into a forever sub-contractor”. One additional beauty of the European Cloud Module idea is that if need be, you can run the module yourself, or have someone else run it for you. By building on top of relatively standard blocks, customers gain a lot of freedom compared to the current big tech lock-in.
Next up
Crucial at this point is to test the assumptions made above. Will European players commit to supplying money, people, operational experience? Are there open source projects suitable for signing up and turning themselves into European Cloud Modules? Do they want that? Would we be able to find the people that can drive this process in a way that respects the software but also the business needs from the providers?
Interestingly, if it turns out European industry does not want to commit to such an effort, this is a strong signal they don’t want to succeed. Because if we all try to build our individual AWS, we are going to fail. But cooperation is our strength here, so we should be able to make it happen.
If governments, who think about more than the next quarter, show up as customers & moderate scale funders, and if legislation also creates a market for European clouds for sensitive/crucial data, I’m reasonably optimistic we could do all this.
But I’d love to hear your thoughts!
To get updates when I post something new, do subscribe to my mailing list.