However, although I was not using AI for my FOSS contributions, I kept using it at work. In my workplace, as well as in many of my developer friends’, using AI to work is very, very common. Not without a degree of shame, let me tell you about this experience, and how it was turning me dumber, lazy, and a worse developer.
A friend told me hey, you should try it, you feel so powerful. Once you start using it, you cannot stop. The tasks that took you days, you can have them in hours. No, we’re not talking about cocaine, although to some degree I consider the use of AI, an addiction. I started enabling the enhanced autocomplete in vscode (or was it enabled automatically?), and downloading and using other code generation models just followed. When I was a child, I remember my parents saying be careful with the bad guys who offer candies at the school gate, those are not candies, those are drugs. When I was in my twenties, I’d pull their leg saying yeah, it’s like they give away the drugs now hum? Well, in my thirties I think they may very well do.
Once you start using AI coding agents, things spiral out of control very quickly. At the start, you ask AI to implement a function for you, or to write tests for a certain piece of code. If you’ve been a TDD guy for a long time, you’re inclined to ask it to do the tests for you first, so that you can later write the code. Except you don’t write the code anymore, you just ask AI to do it, and half of the purpose of TDD (not biasing the tests by how you’ve implemented the code) is gone. But you feel you go so fast that you start not caring.
One day, I saw the agent was co-signing the commits I was making, and I ran to opt-out from that setting. Like, I wanted to pretend the code was my own. Gosh.
It would not be so bad if things had stopped there, but that’s not how most human brains work. If you like something and you can have 2x, you’ll have it. I started pasting the whole Jira description of a ticket, and let AI implement it for me. Yay!! So powerful!! Working on a codebase that was relatively new to me, I started losing control over which changes were really needed to fulfill the task. I thought if the AI, which has read every line of code in the project says it’s needed, so be it. Control is an illusion. Other folks I’ve discussed this with agree that they don’t know 100% of what the code they are pushing to production actually does. I bet we not even 20%. Fucking scary.
But it also didn’t stop there. When I had a small task, I gave it to the AI, and in ~30 minutes I had a PR. What was I doing during those 30 minutes? Changing focus. Soon I realized I could have not one, but multiple agents working on different tasks simultaneously, in multiple git worktrees. Full power! Tokenmaxxing! I connected the agents to Jira via ACLI (command-line interface) and just asked them “work on this jira ticket in a separate worktree”. And it would start working on it. I felt ecstatic.
When the task was not so small, you’d think I was working on it myself right? Hell, no. I would of course ask AI to decompose it into smaller tasks, and work on them. AI would then give me a super convincing PR with a bunch of changes I didn’t quite understand. How on Earth was I going to ask for a code review of those changes? What would my peers say? What was I going to say when they (more experienced, specially in that codebase) ask me why I did this or that? How was I going to admit it was all AI, and I was nothing but an AI shepherd? So I forced myself to understand the code. Not that I wanted to understand the architecture of the codebase, I wanted to avoid the humiliation of not knowing what the fuck I was doing.
At some point, I had so many AI agents working, that the focus change was too much for me. There were tasks I could have done in 20 minutes easily, that took 5 minutes of an AI agent, and then 2 days for me to review. Because there were so many other things. And maybe AI and computers are good at context switching, but not me. I was exhausted. Because if it was just about approving PRs that are perfectly written, it would be good. But for each PR I had to look at the code, the tests, the style, give feedback, switch to something else, go back again, push, see if I have any PR comments, if CI passed, wait, now the linter complains. And it takes you 2 days to deliver something that could have been literally 1 hour single-focused. In my personal experience, multiplied AI performance is just an illusion. Multiplied frustration and exhaustion is guaranteed, though.
I think there was not a single task I gave to the AI that I could merge as is. Of course, I didn’t write the commits or the PR descriptions myself (what is this, 2024??), AI was doing it for me, and it made sure there was a 7-paragraph description in the PR, explaining what it had done. I stopped reading those descriptions, like, I told it what to do, I assume it did what I said. The description is for others. Because there has to be a description, right? AI is so fucking good as sounding professional, that you relax and let it be. It’s not that I actually thought the code was good, but you get convinced over time, you get lazy, you become complacent. You stop questioning, and start accepting as good some code you would have never accepted, just because you cannot tell why it’s bad. You have lost control. Most AI developers will deny it, but deep inside, they know it’s true. Just don’t want to face it.
Months passed by, and one day I realized I had not written a single line of code myself in several months. I didn’t even commit myself. Countless times I’ve written in the AI agent chat “commit and push”. What. The. Hell. If that is not laziness, I don’t know what it is. AI was also not able to solve all the tasks that I gave it. Not even half of them. Once and again, I had to babysit the AI agent saying “now, apply this pattern” or “in the specs, separate the test setup in a before block, and add the assertions in a different block”. Like I was mentoring a junior. Don’t get me wrong, I love mentoring, I’ve done it for years, and I think it was good for the mentor and the mentee. But I’m not going to fucking babysit a machine, sorry.
Often times, I would see that the quality of the code written by AI had decreased significantly. Not that it was brilliant at the start, but I got the feeling of “dude, you’ve done this before, why are you uncapable of doing this again?”. So I had to prompt it again to re-explain how to do it. Or link it to a file with my preferences. Alright, I know what you mean now, sorry about the confusion. Let me redo it for you. And again it would do a fucking mess that is not what I was requesting. It started reminding me of the strategy where Google lowered the quality of the search results, forcing users to search several times, and see several times as many ads. And if that worked for Google, how is it not going to work for AI? Specially in the uncertain financial times the AI companies are going through, do you really think they would not do it? Come on. They will do this and more.
Other days, I would see that an agent had been stalled for 30 minutes without giving me an answer. What to do about it? Do you think I took the task and worked on it myself? Absolutely not. I took the verbal whip and, sounding authoritarian, demanded my actual code! The AI agent profusely apologized, and gave me in 20 seconds what it had not given me in 30 minutes. I looked at the token expenses, and it had spent $30 in tokens for absolutely nothing. If I was the one paying the bill, I would feel completely raged.
One month ago, in a code review of one of my PRs, a coworker said something like “this test you’ve written doesn’t actually test the scenario that you’re changing in the application code”. I reviewed it, and he was right. But… it was so convincing… And I felt shameful. Because for me, a developer who had been using TDD for 10+ years, failing on something so basic was a bit of a humiliation. I realized I had lost control, and it was not going to get any better. I think It’s worthless to just try harder, be more picky with the AI, demand the quality you’re paying for! Because despite your efforts, the AI ends up convincing you, and making you complacent again.
So I decided I would stop using AI. Even if that costs me my job.
When you take a draft PR written by AI and have to work on it yourself, you realize to what degree you didn’t know what it was doing. You start questioning if you need this and that, and ultimately discard everything and start over. I felt rusty, like I had not coded myself in ages. Fortunately, I had been programming here and there in my side project, so the effort was noticeable but not that bad.
I went back to TDD, to my PRs with just 5 files changed. To my 2-line PR descriptions that were straight to the point. I went back to understanding what I was doing, to deploying with confidence, to being ready for a detailed code review, because I knew what the changes were doing, and I could justify every line of code. I went back to enjoying human code reviews (not AI-ones, I still hate those. Even more humans that paste AI reviews pretending they are their own), to learning from them. I started asking about architectural decisions, why things were a certain way. I could have asked AI, but I preferred to ask my peers.
I recovered the joy of programming. And I didn’t look back. I won’t pretend that number of lines of code or PRs is a valid metric, especially when AI can be as verbose as it wants to win on every metric. But I can say that I am pushing significant changes every day, getting PRs merged, and at the end of the day, getting the work done. And for that reason, I’m not scared of being fired. Plus, I’m keeping my brain functions intact.
Say no to drugs. Kind of a metaphor, but not quite. The first step is to realize how dependent we you are on AI. A piece of technology that you don’t control, and which costs your employer (if not you) a fortune. You’ll say but my manager expects a high performance from me, how am I going to underperform now? I cannot speak for you, but that didn’t happen to me. I don’t think I underperform compared to 2 months ago. If anything, I am now more aware of what I’m doing, more capable of responding to an incident if there is one (because hey, I wrote the code that fucked up), and the more I (literally, me) work on the codebase, the better developer I become. That is what your manager should want from you.
And you will save them some bucks when the AI budget cutoffs start - because they will, if they haven’t already.
If you’re a developer like me, it was your experience as a developer what got you that job. If you’ve been some years in the market, you know you are as good as difficult were the problems you had to face during your career. If you turn off your brain, and relax babysitting AIs, you’re not getting any better. You’re losing value, because what you can deliver, your actual value as an engineer (not an AI shepherd) decreases as your brain becomes complacent and yields control to the AI.
Wake up. Be brave. Regain control.
]]>Some time after that, I started a personal journey dedicated to replacing all kinds of proprietary software with free software. I made a huge list, and one by one, I ditched dozens of apps and services in favor of free alternatives. When it came to the blog, I moved to Jekyll and started deploying it (along with other stuff like a synapse server) in a VPS.
I was feeling the power of free and open source software. I started hosting dozens of services in several VPS, I set up a homelab and started hosting even more services. It was a lot of fun, and I had to forcefully learn a lot of things. I don’t regret these times. But at the same time, maintaining all these services became very time-consuming. I was tired of my synapse server going down (for whatever reason) and being unable to communicate with my family, which wholeheartedly stopped using WhatsApp with me and followed me in my crazy adventures. I eventually shut down that server, and moved all our accounts to a public one.
This trend of moving services out of my VPS and homelab continued. I stopped having my own docker registry and started using quay.io. I removed my gitea instance and moved all my stuff to Codeberg. Some time later, I removed my forgejo actions runners and started using the ones provided by Codeberg for most of the things (everything except building docker images). Feeling very thankful of Codeberg for all the great service they provide to the community, I became a member of Codeberg e.V..
When I started the self-hosting journey, I learned that free software gives you indeed the freedom to run anything on your own terms. The one thing I realized recently is that, even if one can have all services self-hosted, it’s not efficient for the most part. It’s not efficient to have some forgejo runners idle 98% of the week because I dedicate 3 hours to actual coding. It’s not efficient to have a synapse server for less than 10 users, or have a VPS up and running to host a blog that barely nobody reads. I am confident that with my little usage, the resources I take from Codeberg are far less than the cost of running the services just for me.
This is not just about money. It is about building community. I do save some money if I can shut down a server because Codeberg’s covers my needs now. But more importantly, because I now depend on Codeberg, I do care about Codeberg, it’s resilience, and its future. I feel their pain when AI crawlers try to bring the service down. I value the efforts of the team keeping it up and running. I celebrate every forgejo upgrade, and value forgejo as a project much more than I did before.
The community-based way is also more approachable. To some extent, I have the skills to operate a server, build and deploy my docker images, and debug when something goes wrong. But this barrier is pretty high. If we want more people to use free software, we cannot depend on them being able to do all of this. But building and supporting a platform that lowers the barriers so it’s easier to have a website (like this one), to have a FOSS mail service, or to have private chat conversations is what will ultimately make free software (as a service) adoption skyrocket.
]]>If I’m not mistaken, RVM does not share gemsets between ruby versions, so I had 10 rubies installed, each with a copy of the gemsets of a rails application. Yeah, a lot. I took a quick look to the documentation and came up with a one-liner I’d like to keep for future-me:
rvm remove $(rvm ls | grep ruby- | cut -d " " -f4 | grep -v $(cat .ruby-version) | paste -sd "," -)
Essentially, it uninstalls all rubies except the one in .ruby-version. I had never used paste before, and it was pretty straightforward.
Happy coding!
]]>Things were looking good. I mean, the plugin did the job, Google surveilled us all, we got the metrics we wanted, and we were OK with that. One day, my dad called me and said “son, IDK what I did, but I can no longer see some metrics”. I thought my not-so-techy dad had messed the menus again, but when I looked, the charts were gone. Well, not quite. They were blurred. And there was a banner asking to upgrade to the premium version of the plugin to unlock all features.
The freemium business model is not designed for your benefit as a user or consumer. The sole purpose is to show you a portion of what you can get, and put all the fancy stuff behind a paywall. It’s just proprietary software, and the free part is the bait. The company behind will have no remorse in removing parts of the free features if they want to squeeze some more revenue, or finish the freemium program at all. Once you’re locked in, you’re at their mercy.
I was not surprised to find out there are free software alternatives to Google Analytics and the bridge WordPress plugin. I found Umami, which does the job, has a free software license (MIT) and can be self-hosted. I made the switch, and everything worked smoothly.
Now, many free software projects also have one or multiple vendors that offer the product as a service. They usually impose some restrictions to free usage, such as max storage, historical time period, etc. I don’t have a problem using the free tiers of cloud services as long as they are free software. I don’t have an issue paying for the cloud service either, as long as they are free software. Because I know that I can pull the wire anytime and move to a different provider, or self-host. That is where the freedom kicks in.
Not all free software is built the same. Some projects have some properties I consider yellow/red flags, and I try to avoid when I can:
I’m not against a company making money from free software. Actually, I wish I could make a living out of it. I’m against companies using free software as a bait to attract users, and then restrict their freedoms with paywalled features. I’m totally happy to pay for a hosted free software project that is identical to the one I’d get if I decided to self-host.
]]>And something clicked in my mind.
As a community, we (and by that I mean other people smarter than me) have been able to build sustainable free and open-source alternatives to Twitter/Bluesky, Instagram, TikTok, Windows, Microsoft Office, and countless other proprietary solutions. The fediverse as a whole (including Mastodon, Loops, Pixelfed, etc.) and many other projects (some of which I’ve seen this weekend in FOSDEM) are the perfect example of how something different, better, is possible. Maybe not hegemonic yet, but viable.
So a now-obvious question came to mind. When are we going to replace Amazon?
So, here is my dream. I want to build something that replaces Amazon. But not any replacement. This time, I want it to be a community project, the core principles being:
This is all very cool, but who pays the bills?
First, we need to realize that models like Amazon exploit producers and carriers, and promote low-quality products with high prices. Everybody except Amazon needs an alternative, and it is in their own benefit that this alternative exists.
Second, we need to evaluate the convenience of small VS big instances. We could have few (virtually one) big instances, which would require a lot of resources to run, and be managed by very few hands with a lot of power (even with goodwill). Or we could have many smaller instances, each needing fewer resources, each implementing their own rules and paying their own bills. Both could work. But if you ask me, I’d rather go for the second approach, even if it is collectively more expensive. I think it’s important that each community can build their own set of rules rather than accepting whatever is imposed by the single instance operating on my territory.
Third, we should analyze the level of professionalism each instance may require. Maintaining the instance could be the side project of the cousin of the butcher. Or the community could pay a professional service provider to host the instance for them. Some may require on-call to ensure 24/7 uptime, some may not. Each community should find a balance between resources required, stability, and cost. The yearly costs could go from €50/year (cheapest hosting) to… well, any amount you want to imagine if you need to support dozens of millions of users buying from the same store, an on-call team, etc.
Now, each community is free to organize the funding or their instance the way they wish. Some ideas include:
Multiple instances may coexist in the same physical area. It would be like having several markets on different squares in the same city. Stores should be able to decide in which market they want to sell, and buyers should be able to choose which store to buy from.
But isn’t this too complex? Aren’t products going to be more expensive? Doesn’t goodwill Amazon optimize costs by volume, and therefore can provide more competitive prices? What about compliance with local laws?
Calm down.
Businesses existed before Amazon, and they will exist if Amazon shuts down. Amazon didn’t invent logistics. Some things will indeed be different.
Marketing is the biggest lever companies have to make sure their products get the first positions in the search results. Not better quality. Not lower prices. Higher investments in marketing equals more views and usually more sales. By removing the possibility to bribe the platform, sellers don’t need to invest absurd amounts of money trying to convince you that their product is better, allowing them to set lower prices. Take into account that when you’re buying a product, you’re also paying the indirect marketing cost of selling it (which often is more than the real price of the product itself). After removing marketing from the equation, sellers will have to compete on other factors that are beneficial for the buyers: quality (rating, reviews, etc.), price, and locality.
Logistics companies exist beyond Amazon. We should be able to integrate them into the platform. Not only the big ones, but also smaller, local delivery companies that may operate in smaller areas. Companies already sell off-Amazon. Some sellers may want to offer free-shipping (that is, assuming the cost of the delivery and including it in the price of the product) or charge for the shipping. Some things need to change, though. Maybe the system cannot afford free shipping for a 50-cent sale, delivered from 50 km away to your home. Perhaps this is not the world we should aim for.
Such a big initiative will not solve all the challenges in the first year. Not even in the first 5 years. We will need inputs from hundreds, thousands of buyers, sellers, and carriers. We will need to have tough discussions, discuss what is good or bad. Different instance admins will make different decisions based on their ethics. This will be less of a big store ruled by a multi-billionaire and more of a 21st-century network of digital markets.
Not for venture capitals or billionaires. For us, producers and consumers.
]]>While investigating this issue, I found several resources that suggest modifying Docker config to increase the number of networks available. I’m not going to pretend I understand it all. I did this for my homelab a while ago, but how to do this for a Forgejo Actions docker-in-docker setup?
I didn’t have the time to investigate the issue, but I could get into the dind (docker-in-docker) container, run docker network ls and confirm there were plenty of networks:
NETWORK ID NAME DRIVER SCOPE
0aa32f34b641 WORKFLOW-0febb6edf9ce00f972310629804356d4 bridge local
4d7a7f591a11 WORKFLOW-6ae224aa3687a85d6f25adde9b46c19f bridge local
8cccbbba38ee WORKFLOW-8cf00791dc20765932eaf71b5a5a5d7e bridge local
d01eb0dd2c4e WORKFLOW-9e620af02e1f423ba37887442e7bece7 bridge local
f8463324d26b WORKFLOW-19ca2082193c77dc90be2245ec44f17c bridge local
[...]
this would be fine if there were as many running jobs, but my Forgejo Actions concurrency is limited to ~5 or so, not 32, so something was definitely off. I ran docker ps within dind to see the running containers:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c8cb73fdaee5 moby/buildkit:buildx-stable-1 "buildkitd --oci-wor…" 7 days ago Up 13 minutes buildx_buildkit_builder-1fb2e2c1-4e5a-4974-a542-653d8344b1590
8ce130d2dc43 moby/buildkit:buildx-stable-1 "buildkitd --oci-wor…" 8 days ago Up 13 minutes buildx_buildkit_builder-5262d6e9-4dd9-419d-93f9-14d07fb1ebfd0
eec52d0cc9de moby/buildkit:buildx-stable-1 "buildkitd --oci-wor…" 8 days ago Up 13 minutes buildx_buildkit_builder-86305831-2272-4e52-9bd5-c7d406e46d3e0
e4fba3d99c97 moby/buildkit:buildx-stable-1 "buildkitd --oci-wor…" 9 days ago Up 13 minutes buildx_buildkit_builder-f409254c-15ec-4b0a-afeb-3bb1d32ffaa60
3b92ae7c3fa1 moby/buildkit:buildx-stable-1 "buildkitd --oci-wor…" 9 days ago Up 14 minutes buildx_buildkit_builder-ee01e6d2-642c-4b55-85e1-84cc0db86cb10
04a54a17bc16 moby/buildkit:buildx-stable-1 "buildkitd --oci-wor…" 9 days ago Up 13 minutes buildx_buildkit_builder-51d85866-c4b1-46f8-87f3-523fec1211ad0
ae985b471c3a moby/buildkit:buildx-stable-1 "buildkitd --oci-wor…" 10 days ago Up 13 minutes buildx_buildkit_builder-9c0c35db-5ee2-42ef-b370-b2b7aadb029a0
[...]
and saw many of them had been created days ago. I don’t know if these containers are kept alive for performance reasons, but for sure they were not being used on my CI jobs at the moment. Note the STATUS is recent because I had just installed docker upgrades which restarted the contianers.
So I stopped the containers with docker ps --format= | xargs docker stop and cleared the dangling networks with docker network prune, and then I could rerun my CI jobs.
Note this is just a workaround that worked for me. I’m not endorsing this, nor suggesting it’s a good idea. This is more of a note-to-self than advice for others.
Happy coding!
]]>#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile:
#1 transferring dockerfile: 2.14kB done
#1 DONE 0.8s
#2 [internal] load metadata for docker.io/library/node:24-alpine
#2 ERROR: failed to copy: httpReadSeeker: failed open: failed to do request:
Get "https://docker-images-prod.6aa30f8b08e16409b46e0173d6de2f56.r2.cloudflarestorage.com/registry-v2/docker/registry/v2/blobs/[...]":
tls: failed to verify certificate: x509: certificate is not valid for any names, but wanted to match docker-images-prod.6aa30f8b08e16409b46e0173d6de2f56.r2.cloudflarestorage.com
These errors started happening out of the blue. I don’t know the root cause yet, but here goes what I did to fix it in case I need it in the future. Note to self: TIL posts are not guides but quick notes for future me written in under 15 minutes.
After reading I was being man-in-the-middled, I started making some other tests in my server. This error was happening inside a docker-in-docker container used by Forgejo to run docker-related commands, but I didn’t have this from the docker host, so there was definitely something off with my docker networking.
I continued comparing side-by-side the output of curl -v on the aforementioned URL, and verified the DNS resolution was identical. Against all odds, it was not DNS this time. The DNS resolution returned two addresses, an IPv4 and an IPv6. In the host, the request was made using the IPv6 and everything ran smoothly. However, in the docker container, trying to connect via IPv6 gave some network unreachable error, so it fall back to the IPv4. I have no idea why things worked with the IPv6 but not IPv4, but hey, I had found a difference that most likely explained the problem.
From the host, I ran docker network inspect <my-network>, and observed IPv4 was enabled but IPv6 wasn’t. I had to:
docker compose down
docker network rm <my-network>
docker network create --ipv6 <my-network>
docker compose up -d
After this change, the curl -v from the container started running smoothly, and operations in my codeberg runners were resumed.
I hope I don’t have to come back to these notes, but if I have to, I’ll be glad I ever wrote them.
]]>When my aunt, a non-tech woman in her sixties, told me, “Element was not working; it says something like it cannot connect to the server” for her, my first thought was there was something off with her phone. After all, if it’s working for us all and not for her, it must be something on her end. The day after, my sister-in-law told me she didn’t get a message about a theater night we organized, and I started suspecting there was something off, not just for my aunt, but for some other users.
It was not until I saw this toot by The Matrix.org Foundation that I realized what was going on. The accounts of my aunt and sister-in-law were created in the homeserver run by the foundation, while the rest of my family used mine. This realization brought two ideas to mind:
First things first, this article is in no way a criticism of the Matrix.org Foundation and Element teams, which are doing excellent work to bring and support this technology. I admire and deeply respect their work.
The natural answer to the questions raised above is “Cool, let’s spawn new servers!” but that comes with additional questions related to funding, operational costs, moderation, maintenance, etc. A few months ago, The Matrix Foundation published a blog post explaining why they made the decision to introduce freemium and premium accounts on the server they host:
We’ve kept the matrix.org homeserver around so far, despite its costs, as we consider it essential to seed the network in support of the nurturing part of the Foundation’s mission: despite Matrix being decentralised by design, users need a trusted place to create a free Matrix account to try it out in the first place.
However, we can’t continue to bear the cost of the server as is, and before we get to the extreme position of being forced to turn it off leaving its 370k monthly active users in the awkward position of finding a new home for their account, we’ve decided to try to alleviate some of these costs by setting-up a freemium offering and proposing premium plans in addition to the free ones.
I cannot blame them for decisions like this one. In a world where we’re used to having Internet services like mail, chat, blogging, etc. for free (not quite, as we are often forced to give away our privacy, but you get the point), when a service provider decides to charge for some features, the message is not always welcome. In many cases we’ve seen corporations offering free services until they have a sufficiently large customer base to start monetizing it. I don’t think this is the case, and I trust them when they say this is a survival measure.
We have to realize that digital services have running and operational costs, and as users, we have to pay them one way or another.
In some cases, you just have to go through a paywall. Plain and simple. And it kind of makes sense. Let’s say running a certain service costs €1000 yearly for 200 users. Paying 5€ each may be the most sustainable way to keep the service running. This approach may also be used by a for-profit company, offering free software as a service, and getting a profit margin out of it. Sometimes, the cost per user is so low that one individual can handle the costs, and gift the service to the community. However, this doesn’t escale for millions of users, in which hase funding a foundation that carries the heavy weight may be sensible as well.
All these options come with some drawbacks as well. Paid subscriptions for every service may leave some people out, as not everybody can afford a paid subscription for a mail service, a chat, a blog, social media, etc. You also need to make sure you’re not locking to a vendor, as otherwise you will be at their mercy in terms of security, costs, and features. If a provider doesn’t apply security patches, dramatically raises the running costs, or limits the features in a way you don’t agree with, you should be able to move on to another provider, taking your data with you. Relying on an individual who selflessly provides a service to the community puts, in my opinion, too much pressure on one’s shoulders. That’s the reason why I don’t open my Synapse server to the general public. I cannot guarantee an acceptable uptime because this is a side project for me, and I would not like thousands of users relying on my pet project. The community could be at risk if, at any point in time, that person has to quit (costs, time, death, etc.) putting the users in a difficult situation. The problems of having a single foundation running the service for 99% of users defeats the distributed nature of the protocol and all its benefits, so that option is also not ideal to me.
If you ask me what the solution is, I don’t think I have one. Sad but true. However, analyzing the cost, power, and governance implications of the options depicted above, there is another solution I think may be better than the rest. I think local communities should fund and operate Internet services for the neighborhoods.
When I say local communities, I don’t mean public institutions. Very often we’ve seen how public services have been used to control activism and dissidence, and I don’t think we should have the fox guarding the henhouse. Even though there are some measures that limit what the service provider can do with your data (E2E encryption, limiting the data provided, etc.), it is still too much power in the same hands. Plus, governments change regularly, and as a citizen, I don’t want my services to be impacted by whoever is ruling this year. When you’re using software as a service, even if it’s FOSS, you have to implicitly trust they are not running a modified version of the software. How can I know the software my local administration is running doesn’t have a backdoor for intelligence agencies? We’ve seen many governments that are very willing to have access to encrypted information from their citizens, so again, I would not trust local authorities for this duty.
If it’s not public institutions, what do we have left?
At least where I live, there are multiple organizations and associations fighting for different projects (ecologism, feminism, anti-militarism, housing, childhood, political parties, anti-racism) that co-exist in physical spaces, sometimes occupied, sometimes not, to develop their activism. In some of these spaces, groups of tech people (hackers, makers) gather to work on their projects, organize talks, and in general promote a use of technology that is beneficial for the wider public.
I think hacklabs are the best option to host internet services.
Wouldn’t it be beautiful if a hacklab offered internet services to the different collectives that gather in the social center? What if that extended to the neighbors that participate in the collectives? Or any neighbor that requests it?
In terms of funding, I don’t think this is inherently different from the rest of the alternatives. Just because a hacklab is running the service doesn’t mean it runs for free. But it’s up to the hacklab to decide what they need and how to fund it. A small hacklab with a dozen people may run with donated hardware almost for free. A bigger hacklab serving tens of thousands of neighbors may use membership fees to keep running. Or they may have a scheme where those who can afford it pay more, and those who cannot get it for free.
Hacklabs and social centers are not free of power fights and corruption, and it is possible that at some point you’re no longer aligned with the decisions made in the hacklab, and you want to move somewhere else. If you’re lucky, you may find another social center with a different governance nearby. In my limited experience, these kinds of spaces are run kind of horizontally, so it may be possible to highlight what’s being mishandled and make a change. It may not be easy, but at least you’ve got better chances than trying to make Big Corp change any of their policies.
Maybe we’re a bit too far from this scenario, but it’s definitely one I would feel comfortable with. And I think it’s a sustainable way to bring free software to non-technical people. At the end of the day, if you need self-hosting skills to have an email account, we will always be at the mercy of big corporations.
Have a great day!
]]>As a consumer, I often find it difficult to decide which companies I want to support, and which I don’t. I literally cannot investigate every single company I want to buy from. But at the same time, if I’m aware that a company is involved in what I consider unethical activities, I cannot just ignore it and move on.
When I started my Free Software activism, I made a list with all the proprietary software I use and decided to look for a free, open source software that could give me a similar service. Along the way, I learned about privacy and digital sovereignty, which made self-hosting a very appealing option for me. Since then, I’ve embraced projects like Matrix, OpenStreetMaps, CoMaps, ntfy, Nextcloud, Jekyll, LanguageTools, Kuma, Grafana, Gitea. I even host my own Docker registry. But one of the apps I didn’t manage to get rid of is, precisely, Spotify.
Roughly 20 years ago, I used to have a 512MB MP3 player where I stored pretty much all music I needed. After that, for many years, I didn’t listen to music at all, and it was not until I started working in an office that I created a Spotify account and started enjoying music again. As a software developer, a fair amount of my working hours are dedicated to reading, coding, researching, testing, etc., all of which I typically do own my own while listening to music.
I am not actually sure where that music came from back in the days, but I suspect I didn’t have the skills to download it myself. Or maybe I did, through torrent. Or a friend transferred those files to me. One way or another, most likely the origin of those files was dubious.
The paradigm of music consumption has also changed over time. I remember playing all tracks in an album during road trips, something that few of my friends do nowadays. We’re used to the immediate demand, to the surprise, to the music shuffle. We enjoy not knowing which song comes next. We want to discover new songs, stay up to date with the new releases of our favourite artists, or find new groups. We’re used to all that commodity, and it’s hard to walk away from that.
For artists, things have changed as well, and not for good. I won’t pretend do be an expert on the field, but those who are, are organized in platforms (ref. 1, 2, 3) that condemn predatory practices by Spotify and lower royalties than ever. I see there are some platforms like BandCamp that pay a big share of the benefits to the artists, which may be a more ethical way to acquire music. Plus, you can download the files in MP3 format and use the player of your choice.
The fact that you can have your music in MP3 format is very relevant. When you use a streaming service, you pay for the license to stream (not permanently store) digital media, tied to the contract with a certain provider (Spotify, Apple Music, etc.). In 10 years I’ve been using Spotify, I’ve paid roughly €2000, and I do not own a single copy of a song. However, if you buy music in MP3 format, or rip a CD you own (even if you have to bypass some DMCA protections, which I find unethical on their own), that is something you can later use in a different device, backup, etc.
Provided the origin of those MP3 files is ethical now (unlike my files 20 years ago), you can still get some of the fancy features streaming platforms provide. For example, you can configure an instance of Navidrome to host your personal music library, and apps like DSub2000 to stream songs in your device.
As I mentioned in the beginning, I wanted to ditch Spotify simply because it’s one of the leftover proprietary pieces of software I am still using. But the last nail in the coffin for me was learning that Daniel Ek’s investment group is putting money in the arms industry (ref. 4, 5) (I refuse to name it defense industry) which is something I cannot bear with. I don’t analyze the investments of every CEO of every service I use, but this is something I cannot (and don’t want to) unlearn, and therefore I cannot just sit and wait while the money from my subscription goes to funding this kind of companies.
Again, I won’t pretend everything I’ve ever done is legal or ethical, but at least, I’m trying. And for me, this is one more step towards becoming the person I want to be.
The rhyme in the headers was unintended.
]]>I never managed to recover that data (to this day I don’t think that’s even possible), and I have to admit that was one of the most stressful situations I had in months. In retrospect, it’s obvious I should have had backups from day one, but it’s not until you suffer the pain of not having them that you realize how important they are. Of course, I knew that could happen, and of course I knew I should have made backups, but for some reason you always think that will happen to someone else, and developing xyz is more urgent than having a proper backup system.
After a sincere apology with the team whose work I had thrown to the bin, I worked on a backup procedure, and it was alright.
When I started self-hosting some services, I always kept this story in the back of my mind. Not long after taking it seriously, I got a NAS and started using it for regular backups. In this post, I want to reflect some learnings and pieces of configuration that I found relevant while doing so.
My homelab server was initially connected via Wi-Fi, which I know, is terrible in terms of speed. It was not until I realized there are RJ45 sockets on the wall that I wired the server, getting a dramatic speed increase. Because I had two network interfaces (wired and wireless), I also had 2 IPs.
My NAS was configured to only accept connections from one of them, so sometimes the connection worked, and sometimes it didn’t. When I reviewed the logs, I realized some forbidden connections came from one of the two IPs, so I had to update the config and everything worked like a charm.
I also started getting errors while storing the backups of a little Moodle instance I recently set up. The reason was that I had configured a maximum storage of 10GB for the shared folder, and over the weeks it got full. I followed this forum thread where the author explains how to configure custom scheduled tasks that allowed me to delete old backups, making room for the new ones.
Another thing I had to change is the way I mounted the NAS volume via NFS for the borg backups of my Nextcloud instance. Out of pure laziness, I had first mounted the directory with a mount command, but that volume disappears on a restart. Months later (this morning, to be precise), I replaced that with an entry in the /etc/fstab file, restarted the server, and verified the volume was properly mounted.
This was not particularly difficult, but I suspect every time you face a problem for the first time it looks bigger than it actually is. When I was first mounting the backups, I had enough understanding NFS, SSH tunneling, etc. and I just couldn’t bear with another moving piece.
This was my little adventure for this Wednesday morning, after a sleepless night due to hot weather. I’m glad I fixed this once and (hopefully) for all, but I wouldn’t say no to a straight 8-hour-sleep night either.
Happy coding!
]]>