I saved $150 a month by hosting my own infrastructure

I’d been a long time DigitalOcean user between their low-cost droplets and their managed Postgres databases.

I used their small droplets to host my apps rather than services like Vercel or Netlify partly because I wanted better performance and practice running my own apps on my ‘own’ infrastructure.

Funny enough, DigitalOcean sponsored me in a very broad sense in my early development career when I was working on some random idea. I reached out and asked about getting a some sort of sponsorship. They actually gave me about $20 a month credit for a year, so I became a die hard DigitalOcean fan.

Last year though, I decided I needed something better than their droplets. I was quickly spending much more money than I wanted to (something like $100 USD a month) for a 4-core and 8gb of memory.

When you’re converting to CAD, the USD conversion can become quite expensive.

To save money and get much more performance, I started a 3-node kubernetes cluster that I, and Claude, manage.

The setup

I have a 3-node control-place setup that uses two on-premise nodes and one off -premise node.

The two on-premise nodes are on two computers I own. A VM in my truenas computer. And a VM in Proxmox on my other computer.

Both computers, the 2.5gb network switch and the ASUS router that looks like a spaceship, are plugged into a UPS.

The third VM is on my uncles server rack. While this is off-premise, it’s still in town so if anything were to happen here, I’d be SOL.

While I’m writing this, it would be cool to have a one-touch solution that would replicate a bare minimum setup of my apps somewhere. Because my computers are hooked up into a UPS, it could buy me some time to upload it somewhere in the cloud. And since I have 3gb fiber and fiber doesn’t typically ‘go down’ in the event of a power outage, it would buy me enough time to upload since I will still have internet access.

There were a few key reasons for doing this change that I think most people would have avoided given I was using Kubernetes. But, I really enjoyed the orchestration nature of k8s. I was using Docker compose a lot and it was great and all, but I wanted something that was actually a little better at bringing apps and down bringing them up with little downtime.

Using Tailscale to communicate

All of my apps communicate via Tailscale. Every computer I have is in the same network so that I can manage my cluster from basically any device I own. Rancher is serving its UI in the network which means from my iPad, phone, laptop, I can manage the cluster. I also use Claude for this too.

Every app is secured behind Tailscale.

Granted, I need to look into this and see what other risk factors may exist. But, it’s cool! Anyway.

I have one DigitalOcean droplet that is the smallest one money can buy. All it does is run HAProxy and round-robin requests to my registered K8s nodes via Tailscale IPs. This actually works insanely well and I’m incredibly proud for how this has turned out.

Solving high availability

The main issue I needed to solve was high availability.

DigitalOcean gave me great availability over all even though I had one VPS with them and it hardly ran into issues.

To solve this, I needed to make sure I could secure atleast three nodes. That’s how I landed on two on-premise ones and one off-premise VM.

My uncle has a very intricate setup with a lot of resources available to run cluster nodes.

Cluster nodes are memory hogs which is what I’m figuring out.

Now that all these nodes communicate with each other, I am seeing an improvement in uptime!

When I first started messing with this, my cluster was facing issues all the time. I’m not trying to be an infrastructure expert, I’m just trying to save some money and use the resources I have at my disposal.

This is where Claude comes becomes invaluable. I let it do mostly all adjustments to my cluster with my guidance. Without Claude, I don’t know how well I’d be able to manage this - not well I can tell you.

Moving from a managed database to self-hosted

Alongside my VPS, I had a DigitalOcean managed Postgres, but I figured I could also replace this and save even more money for what I’m getting.

Now, within my cluster I have CNPG managing my Postgres nodes. It takes care of deploying my Postgres nodes, promoting to master and failovers. It’s been a great resource so far. I have a floating Tailscale IP so I can point my apps to one internal IP which points to whichever Postgres node is the master node.

This is great for if I need to bring down the K8s node that the primary Postgres is running on, once the shifting around happens and it migrates to another node, it would break any of my apps as it would have if I was pointing to a fixed IP. This was my rationale anyway.

So now redundancy and high availability is solved. There’s some down time if a primary node needs to migrate, but that is acceptable.

I recently setup Garage in my cluster that I use in conjunction with CNPG. Garage is used by CNPG to take snapshots/backups of my Postgres database and save them to a dataset within truenas via an NFS share.

This runs daily and now I save about $25 a month.

I’ve had an insane amount of fun moving from DigitalOcean to my own infrastructure. I love talking about it because there’s so much more to it than better app performance. Now, I can self-host any self-hostable app.

I have my own Plausible, Signoz, Excalidraw, Baserow, Twenty, Postiz, honestly, somethings I don’t use all that much! But being able to host these apps keeps saving me more and more money.

And I can continue improving upon things! I can keep improving my specs, maybe add more nodes, open up my cluster to my family if they need to host anything. Though, my uncle will have that covered for his family.

This has been the single most cool thing I’ve done in the last few years. Even though there are times where it goes down and some times my users are affected, it’s a fun challenge to keep that up-time… up. The 3GB fiber network has kept up (obviously, it’s fast).

Thanks for reading!