Trifecta Technology
Trifecta is a simple open source image sharing site, built with a combination of modern C++, database and web technologies. Intended to both be useful and make some points. This page sets out to explain the underlying technology of this small yet hopefully useful piece of software.
More background can be found on its main page, where you can also read a bit about why I built this software. In short, 1: I need an image sharing site I trust and that does not track its users and 2: I want to show that you can still run a service yourself safely. Code is here.
Trifecta attempts to deliver a lot of functionality using not that much code (1200 lines of C++, 400 lines of JavaScript) and not too many dependencies. In short, something you could fully understand and read. Code that does not pull in the whole world, delivering a fully self-contained standalone web service. There is a Docker image that compresses down to less than two megabytes:
Behold, actual software
To make this happen, you need clever technologies that work well together. In what follows, we’re going to look at modern C++ libraries, a modern build system, a minimalistic JavaScript framework and how to build/distribute tiny Docker images. All in the hope that this might inspire other people to also again write small software.
Note that this is all C++ and JavaScript, but the point is not really about C++. You could write a similar-minded backend in many other programming languages.
SQLiteWriter / nlohmann::json: Storing and moving data
There are complex SQL object relational models that allow you to use native language constructs that then get mapped to a SQL backend. But it turns out that 1) this match is not so great and 2) SQL is actually a pretty good language itself to express what you want. Yet, a bridge between your data and SQL is always needed, even if it does not perform that much magic.
SQLiteWriter (which is another of my projects) helps you populate SQLite, and easily add data to it in a type-safe way, like this:
SQLiteWriter sqw("example.sqlite3");
for(int n = 0 ; n < 1000000000; ++n) {
sqw.addValue({{"pief", n}, {"poef", 1.1234567890123*n}, {"paf", "bert"}});
}
sqw.addValue({{"timestamp", 1234567890}});
This leads to the creation of a table called ‘data’ (by default), with four columns called pief, poef, paf and timestamp. Notably, this interface is type safe and uses a ‘STRICT’ table to enforce that.
You can also get your data out again (with type safety) as a C++ vector, but there is also an interface to get an nlohmann-json object. This makes the following possible:
wrapGet({Capability::IsUser}, "/my-sessions", [](auto& cr) {
return cr.lsqw.queryJRet("select * from sessions where user = ?", {cr.user});
});
We’ll get to the ‘wrapGet’ stuff later, but the key part is that this is a “straight paper path” directly from SQL to a web browser. The web browser gets a JSON array of objects, each representing a row. And again, this JSON is type safe, so a number stays a number. Note that cr.user is passed as a parameter, which gets bound to the statement. There is no escaping here.
By having such an easy and safe bridge between SQL and JSON, we prevent A Lot Of Typing.
As noted, SQL is a pretty nice language, and we can make it do a lot of work for us:
SQLiteWriter sqw(args.get<string>("db-file"),
{
{"users", {{"user", "PRIMARY KEY"}}},
{"posts", {{"id", "PRIMARY KEY"},
{"user", "NOT NULL REFERENCES users(user) ON DELETE CASCADE"}}},
{"images", {{"id", "PRIMARY KEY"},
{"postId", "NOT NULL REFERENCES posts(id) ON DELETE CASCADE"}}},
{"sessions", {{"id", "PRIMARY KEY"},
{"user", "NOT NULL REFERENCES users(user) ON DELETE CASCADE"}}}
});
In this way, when the schema gets autogenerated, instructions are put in there to clean up all images belonging to a post, or all posts belonging to a user etc. As they say in Dutch, you got to let the ball do the work.
SQLiteWriter has helpful and somewhat magic functions, but does not try to hide the SQL from the user. Every other ‘ORM’ technology I’ve ever used managed to not provide access to at least one important feature I needed. For this reason, SQLiteWriter gives you easy and raw access to SQLite if you need it.
- SQLiteWriter for seamless bridge between SQL and JSON, with automated schema generation, SQLite
- nlohmann-json, great C++ JSON library
Alpine.js: Minimalistic JavaScript “framework”
JavaScript. An experiment dating from 1995, grown organically, at times with a lot of thought. And there were also many other times.
Meanwhile in 2024, there is a credible set of functionality in standard JavaScript as universally deployed. Previously, (large) frameworks were an absolute requirements to get anything done. But you can boot a Linux kernel hosted by standard JavaScript these days (for real).
The large JavaScript frameworks are still with us however. In Trifecta, I use a pretty minimal framework called Alpine.js. This is not something that attempts to fix JavaScript for you or take over your life. It does however make it very easy to hook up elements on a web page to JSON data.
For example, the following displays the output of the SQL statement above:
<div x-init="getMySessionList($data)" x-data="{sessions: []}">
<table>
...
<template x-for="s in sessions">
<tr>
<td x-text="s.id"></td> <td x-text="s.ip"></td>
<td x-text="if(s.createTstamp) return new Date(s.createTstamp * 1000).toLocaleString();
else return "></td>
<td x-text="if(s.lastUseTstamp) return new Date(s.lastUseTstamp * 1000).toLocaleString();
else return "></td>
<td style="width: 30%;" x-text="s.agent"></td>
<td class="deleteicon" @click.prevent="doKillMySession($data, s.id)"></td>
</tr>
</template>
</table>
</div>
And here is the getMySessionList() function:
function getMySessionList(f) {
fetch('my-sessions').then(response => response.json()).then(data => {
f.sessions = data;
});
}
Alpine is a single manageable file (around 3500 lines of code) and it is very nice as glue between HTML and JavaScript. The variables you define in there are ‘magic’ in that if you change them, all the elements you tied to that variable also change. And vice versa.
Even though this is nice, this is also where it stops. Alpine is not some kind of ‘React’ framework that takes over your life. It does not require any preprocessing or compiling or translating. You just type it in.
- Alpine.js, a minimalistic JavaScript environment
cpp-httplib: HTTP library
You could do this in two ways - either you try to embed a fully functioning web server and hook it up to the internet. Or you hide behind a feature complete web server that also does TLS, Let’s Encrypt etc. cpp-httplib certainly does all the basic stuff, and if you want, it does TLS as well. But I prefer to run it behind (say) nginx.
The whole library comes as a single file of 9045 lines. The code appears to be pretty clean and it works very well with modern C++.
It is however the part I worry about most for the security of Trifecta. Even though this library does not come across as dangerous, it is not a project that is so popular that you can assume lots of people have looked at its security already.
The core concept of cpp-httplib is that you attach function objects or lambdas to URL patterns, which could be regular expressions. There is also support for parsing of multipart/form-data, which is nice for POSTS originating from html forms.
- cpp-httplib, pretty excellent HTTP library.
“Trifecta-lib”: enabling other projects as well
There is a common set of functionality you need for every web service where people can log in, and where an admin must be able to manage things.
Within the Trifecta project, this common functionality has been split out to support.cc and support.hh. There you’ll find classes that manage users and sessions, plus a common set of API endpoints to create/modify/remove users.
Also, to make it easy to add new endpoints, as much work as possible is automated:
wrapPost({Capability::IsUser}, "/change-my-email/?", [](auto& cr) {
auto email = cr.req.get_file_value("email").content;
cr.users.setEmail(cr.user, email);
cr.log({{"action", "change-my-email"}, {"to", email}});
return nlohmann::json{{"ok", 1}, {"message", "Changed email"}};
});
This creates an endpoint that requires the IsUser capability. In cr the library has already filled out which user this is (in cr.user). In that object you’ll also find the Users class. If any of this throws an exception, a JSON object is created with [“ok”]=0, and a [“reason”].
Meson: Modern build infrastructure
Dependencies are important so we can benefit from very good existing code. However dependencies can also be too easy to pull in. Optimally, adding a dependency should be some kind of work. This forces the programmer to have at least somewhat of a think on what is being added to the project. npm famously makes it way too easy to add over a thousand dependencies to your code, without any work.
Conversely, classic Makefiles make it.. incredibly painful to add dependencies. Far too painful. Autotools help somewhat, cmake helps somewhat, but it is all not quite there.
Meson has been around since 2013. I’ve found it to occupy a sweet spot where you can add even complex projects as a dependency easily. Many projects feature a meson.build file, and meson can also benefit from existing Linux/Unix distribution, pkgconfig and cmake infrastructure. Meson itself also has a repository of objects you can add with ‘meson wrap install’.
It is also pretty easy to make your own projects suitable for use as Meson dependencies.
Here’s what including some dependencies looks like:
sqlitedep = dependency('sqlite3', version : '>3')
thread_dep = dependency('threads')
json_dep = dependency('nlohmann_json')
These are dependencies that Meson successfully finds on my system. If you want to add less well known projects from the Wrapdb, try for example
$ mkdir subprojects
$ meson wrap install pugixml
Installed pugixml version 1.14 revision 1
$ cat subprojects/pugixml.wrap
[wrap-file]
directory = pugixml-1.14
source_url = https://github.com/zeux/pugixml/archive/v1.14.tar.gz
source_filename = pugixml-1.14.tar.gz
source_hash = 610f98375424b5614754a6f34a491adbddaaec074e9044577d965160ec103d2e
patch_filename = pugixml_1.14-1_patch.zip
patch_url = https://wrapdb.mesonbuild.com/v2/pugixml_1.14-1/get_patch
patch_hash = 23ceabbd7bc74a6cc6c2e6f625a9b0840a070e1225d85d3cc29c27b6cc059135
source_fallback_url = https://github.com/mesonbuild/wrapdb/releases/download/pugixml_1.14-1/pugixml-1.14.tar.gz
wrapdb_version = 1.14-1
[provide]
pugixml = pugixml_dep
Note the precision of all this. If your desired depedency is not in the database, you can easily write your own .wrap file for any project, and also put that in the submodules/ directory.
Docker/Podman: Containers
If you tell people they have to install something these days you get a rather odd reaction. People want to install only containers it appears. Now, there are some nice things behind that, like at least some isolation should your thing get hacked. But I did love the old situation where if a bug got fixed in the C library, all programs would benefit automatically.
Containers can also be very large and cumbersome and ship lots of irrelevant stuff. Luckily they don’t have to be.
If you know what you are doing, you can build tiny Docker images that only ship your code, and that is what Trifecta does. The minimalistic Dockerfile we use might be somewhat of a shock for some people:
FROM scratch
ADD build/trifecta trifecta
ADD html html
VOLUME /local-db
EXPOSE 1234
ENTRYPOINT ["/trifecta", "-p", "1234","-l", "0.0.0.0", "--db-file=/local-db/trifecta.sqlite"]
This creates an image that compresses down to less than 2 megabytes.
An interesting issue with (Docker) containers is always how to configure them. To make this easier, the Trifecta process will also read environment variables. These can be set from a Docker compose file for example.
That leaves one final point - how are these containers updated? Because should there be a security issue, you do need to get a fixed container. Back in the age when we shipped packages, you could get your Linux distribution to do this for you. In this brave new container world, Watchtower can be used to automatically update containers from a hub.
Incidentally, Trifecta is on the Docker Hub.
doctest: Unit tests
Unit tests, I can’t tell you how much I love them. You don’t have to go over board to ’test driven development’, but it is quite a fun technique to first write the unit tests and then the actual code. There are many great unit test frameworks for C++, but doctest is small, compiles very quickly and is feature complete.
Of some special note, the Trifecta testrunner actually launches a Trifecta webserver and exercises its API that way. This has caught many bugs already, and I love running a “web framework” with actual unit tests.
It would however be even better to also do emulated browser tests (like with Selenium).
- doctest, very nice and fast unit tests
argparse: Argument parsing
Argument/command line parsers appear to be hard to get right. All of them appear to have some issues. I’ve tried many of them over the years. This one (argparse) is mostly sane, and one of the few where you can clearly specity that “–enable-png” is the same as “–enable-png=true”, and that you could disable this with “–enable-png=false”.
While this is well done in argparse, it lacks the possibility to read configuration from a file. Also, periodically you learn that if you create a default argument value of “something”, this ends up as a char pointer, and you get a weird error if you try to retrieve it as a std::string. All quite understandable, but it does cost you an hour.
- argparse, pretty good argument parser
{fmt}: String formatting
C++ iostreams are pretty painful for precise data output. {fmt} is a project that started externally but is now part of very recent C++ standards, so it barely is a ‘dependency’. We don’t do a lot with {fmt} yet, but it is a very good thing to have in any project. It is better, faster and more expressive than any other string formatting solution I know of.
- {fmt}, excellent string formatting, part of recent C++ standards also
No image parsing?
Somewhat surprisingly, Trifecta does not touch the images it serves. This is a bit sad since it might be useful for the software to create thumbnails for example. The security record of most image libraries however is sufficiently depressing that it is not worth the cost to do any kind of conversion. I’m not a member of the Rust evangelism squad, but I’m not touching any image library written in a memory unsafe language unless they’ve very explicitly built their own memory safety equivalent.
Alternatively, after some suggestions by wise people with a lot of experience, I’ve learned that you can easily run your image processing in a very strict sandbox. I’ve implemented this here, and I’ll probably add this to Trifecta at some point.
Summarising
It is possible to write non-bloated web frameworks using a small set of dependencies. Modern C++ has some nice libraries that make this possible.
Meson is an interesting build system that makes it easy, but not too easy, to pull in dependencies.
Docker and Podman might be known for pretty large images, but this need not be so.
Combined, these technologies hopefully provide us with a robust and secure image sharing site.