Here at the Matchmade engineering team, we value curiosity and constantly learn new things, both professionally and about life in general. On the one hand, this happens on a daily basis through solving interesting problems. But, we also have a team with diverse backgrounds, and there’s a lot we can learn from each other. Hence, every month we host a Today I Learned – or TIL – session, where two team members present and teach something to the rest of the group.
In this blog series, we share the topics and lessons learned in our TIL sessions with y’all. If you have questions or ideas, drop a line to [email protected].
This time, we had Juha talking about trusting strangers on the internet. Alex shared lessons learned from our recent migration to Github Actions.
How to trust people on the internet?
Most online marketplaces have some centralized authority to handle disputes so that two strangers on the internet can trust each other. This is how for example Matchmade or Airbnb work. If you have issues, you can reach out to customer support to settle the case. But what if the central authority doesn’t exist?
Before joining Matchmade, Juha worked on a decentralized marketplace Particl, which solves the lack of centralized authority with a mechanism called Double Deposit, Mutually Assured Destruction Escrow Account. It’s a mouthful, but here’s how it works in practice:
- Both the buyer and the seller deposit collateral money to an escrow account.
- Once the deal has been agreed to, the buyer deposits the amount they are buying the item for, and the seller ships the item.
- The buyer confirms that they’ve received the item as expected.
- The seller confirms that they’ve received the payment as expected.
- Only after both the buyer and the seller have confirmed, the funds from the escrow account are released.
Because both parties deposit collateral to the escrow account, they also lose if the other party doesn’t sign off the completed transaction. If something goes wrong, e.g., the seller sends a faulty item or the buyer doesn’t pay, they are forced to work out their disagreement. If they can’t come to a conclusion, both lose, which is what “Mutually Assured Destruction” refers to. Since both have more to lose than to gain if they act in bad faith, this is an efficient mechanism for keeping the scammers away.
But how to implement this mechanism if there isn’t a centralized authority to trust with the escrow? Scripting Bitcoin is one possibility. Bitcoin has “BIP 65” (OP_CHECKLOCKTIMEVERIFY) opcode, which allows the transaction output to be made unspendable until some point in the future. This locks funds in a secure multi-signature address until all of the parties sign off on the transaction.
While this isn’t directly related to what we do, as we indeed are a centralized marketplace, it’s still interesting to consider the alternatives and understand what we take for granted. Not everything we talk about in TILs needs to be immediately actionable, as long as it’s interesting. And as we’re constantly thinking about new features, it’s good to have perspective and ideas to draw inspiration from.
A brief history of our CI/CD pipeline, how we ended up with Github Actions
We recently switched our CI/CD pipeline from Jenkins to Github Actions. Alex gave a quick recap on our history with Jenkins, what problems we had, how we’ve solved (some of) those, and why Github Actions is the future.
In the early days of Matchmade, about four years ago, Jenkins was the default choice in the CI/CD land. We started by hosting Jenkins on our own machine in Hetzner data center. This worked fine first, but as the team and the number of daily builds grew, the single machine became a bottleneck. We decided to migrate Jenkins over to our AWS Kubernetes cluster, where we have the rest of our services running as well. Ain’t no one got time for slow or crashing (due to insufficient memory) builds.
Moving Jenkins to AWS Kubernetes did help, but we also underestimated the effort required. Jenkins’ elaborate plugin system made the migration process difficult, and scaling concurrent builds isn’t free of problems, even if we’re not constrained by the single machine. Given enough engineering time and effort, all of these problems would probably be solvable. Still, we also don’t want to dedicate a team just maintaining the CI/CD pipeline.
Finally, a couple of months ago, in one of our internal Hack Weeks, Alex gave Github Actions a try. This was easy because of two things. First, we were already running things with Docker, so there was very little to setup with regards to the environment. Second, with the microservice-based architecture, it’s possible to try new technologies with a limited blast radius. If something goes wrong, it doesn’t bring everything down, and it’s also easy to revert changes.
Not that we had to revert, everything worked like a charm. We have stable builds, setup is easy, and most importantly, it has very low people overhead. We don’t need to spend time maintaining anything. The downside is that we’re more dependent on Github, but that is the case already in practice. And since it’s mostly just docker containers, there are very few Github-specific things in the setup anyway. Github does cost money, but so does engineering time as well.
To conclude, Alex summed up the lessons learned from this exercise:
- Engineering time is expensive. Both the time it takes to maintain CI/CD pipeline in-house, and also the frustration due to slow or crashing builds.
- Microservices allow us to adopt new technologies gradually. We don’t need to chase all the shiny new things, but we can try out things very easily.
- Prefer action and small experiments over endless planning. Trying out Github Actions was something we had talked about for quite some time. All that time we could’ve spent just trying it out.
Of course, it’s still the honeymoon phase for us and Github Actions. So far we’ve been extremely happy, but there are probably pitfalls awaiting us. Fingers crossed, but this probably won’t be the last time we hear about CI/CD-adventures in our TIL sessions. Until the next time!
By the way, Juha and Alex are looking for new colleagues! Right now we’re looking to hire for many engineering roles, ranging from a tech-minded Designer (React, Storybook) to a Front-end Developer, Data Engineer and DevOps. If you’re interested, drop a line to [email protected]. What would you be presenting in your first Matchmade TIL?