No, it would actually be quite easy to spot.

Nixpkgs templates the source code url fro the url, and then it injects a variable

Here is an example from bash:

pname = "bash${lib.optionalString interactive "-interactive"}";
    version = "5.3${fa.patch_suffix}";
    patch_suffix = "p${toString (builtins.length upstreamPatches)}";

    src = fetchurl {
      url = "mirror://gnu/bash/bash-$%7Blib.removeSuffix fa.patch_suffix fa.version}.tar.gz";
      hash = "sha256-DVzYaWX4aaJs9k9Lcb57lvkKO6iz104n6OnZ1VUPMbo=";
    };

If the url were to be changed, it would show up as a change in git when someone is reviewing before merging.

weren’t they having access through peertube? you can’t revoke access to something that needs it.

The peertube database did not have the vulnerable extension enabled. They got access but connecting to another database:

We did not have pgcrypto installed in the peertube database, but I overlooked that someone could connect to the main postgres database if they say had a nodejs plugin running.

From that database they escaped. So if they only had access to the peertube database, that path would have been cut off for the attackers.

also at that point, the question arises why not just run everything in qubes OS, each service in its own VM and its own VLAN.

VLAN's suck. They allow for traffic to travel within each VLAN them, unmonitored and unrestricted. Sometimes red teamers call that "hard outside, soft inside, like an eggshell".

Better, is a private VLAN. Private VLAN's enable the firewall to monitor or block all traffic within them. For many usecases, a single private VLAN can replace the complex, many VLAN setups that people spend so much effort setting up. You just block everything within itself from communicating with eachother, and then you can explicitly allow the stuff that actually needs to talk.

Often, there is no reason to allow devices with a VLAN to communicate with eachother. And even if you are allowing intra-pvlan communication within a private VLAN, you can now monitor the traffic, which you can't do with a regular VLAN.

Qubes OS

Yeah. That's basically what virtual machine or application kernel based runtimes are doing. The trick is that they are designed for this usecase, with the goal of making the isolation as cheap and performant as possible. Qubes runs full Linux VM's since it is designed for a desktop isolation usecase.

Anyway. It's a matter of threat model and effort. I like application containers/vm container runtimes due to how simple and easy they are to run, and the clear security benefits they provide. Install in < 30 min, configure the container runtime, restart your containers, done. Switching from a regular Linux desktop to Qubes is not as simple, so it's harder for me to recommend that, and it becomes a matter of threat model.

It's also common to separate services out in different Proxmox VM's, which offers isolation benefits akin to Qubes.

I run 3 separate database servers on the same OS because I made a bad decision at one point, and I regret it. I just don’t have time to merge them.

Congrats on the security isolation!

In all seriousness, you should be able to migrate them into the same database by dumping the database with pg_dump or a similar utility, and then loading them up using pg_restore. Other databases have equivalent tools. I used the Dbeaver open source database GUI to do this a few days ago.

Unfortunately, there isn't really a good solution for remote controlling android or ios devices. Meshcentral can view, but not act. Also, the user must initiate the connection from their end.

I was investigating this (for android tablets), and the solution I came too was to enable android debug tools (adb) over wireless (but in this case, remotely), vpn the phone into a remote server to connect them. Then, you should be able to run adb commands remotely (which lets you uninstall apps). And then over adb, you should be able to stream the screen and control it via genscrcpy.

Actually, the first solution I was going to use was device farmer: https://github.com/orgs/DeviceFarmer/repositories , but the above is basically how device farmer works.

I eventually gave up on remote controllung android devices because it wasn't needed and it would have been a complex deployment.

A simpler solution for your usecase is probably to spend a night cleaning up her phone, and then enable kiddie mode on it. That would disable app installs unless she calls you to approve it. In addition to that, (idk about ios), but you can actually install apps on android devices remotely via the google play website.

except that because of the bug, anyone with query permission could have become postgres superuser.

If a user can't log in to a DBMS, they don't have query permission.

separate db server for each of the services is extreme. it brings much more resource consumption.

Yes. It consumes more resources. But it's not that much more, and you can make it fit easily. Many users using docker compose unwittingly do this since docker composes often bring their own database containers. When done consciously, you make a trade off for peace of mind.

the solution here is being subscribed to security releases and updating soon.

I addressed takes like these in the last part of my previous comment. The linked comment also elaborates on my opinions about manual updates, manually watching security releases, and other forms of security toil.

Me!!

I stopped using it a while ago, and I get all my non arch packaged packages from nixpkgs. Nixpkgs is bigger than the AUR and the Arch repos combined. It has pretty much all of the stuff I would have otherwise gotten from the AUR. But I find Nixos frustrating to use, so I stick to Arch.

I felt extremely vindicated in my decision to avoid the AUR when the AUR malware happened.

For your usecase, I would recommend Rustdesk.

But I would also like to mention Meshcentral. Meshcentral is a hosted application that lets you remotely manage multiple devices. It's different from meshcentral in that it maintains a constant connection, and you can do things like view files, run administrator CMD commands, in addition to being able to remotely connect and control the computer at any time.

It's more designed for managing a small enterprise environment, than individual support like OP is doing. The constant connection is designed to be a cheap, open source alternative to Mobile Device Management (MDM) solutions that do something similar. It is invasive though, since it is essentially a backdoor you put in the computer, whereas rustdesk is temporary, and only works when you have it open and are actively connected and using it. That's why rustdesk is more suitable for individual support than meshcentral is.

Although I wouldn't recommend it for OP, I'm leaving this up for anyone in the future who might be searching for "remote tech support" or similar, and maybe they will find Meshcentral more appropriate.

Late to the party but:

https://wiki.archlinux.org/title/Visual_Studio_Code#Unable_to_debug_C#

"Just patch" is advice for a windows administrator, where updates break everything so you have to sit and baby them and apply them manually.

On Linux, there are ways to enable automatic security updates, including automatic reboots, so you can safely receive the mitigations your distro provides. That way, you don't have to worry about forgetting to patch (until the distro release becomes unmaintained, at least).

Now, dirty frag was a zero day, meaning that it was released and probably in the wild before a mitigation was pushed out to handle it. So you did need to apply an actual configuration patch... unless you had some form of kernel based isolation, which I mention as #2 of my other comment in this thread: https://programming.dev/post/52129409/24414213

Excellent writeup, and I appreciate the transparency. I have some suggestions on how to mitigate something like this from happening in the future.

1. Use a separate DBMS (that is, a separate postgres/mariasql/etc container) for each service. Give each one service unique passwords, which you can define in the docker compose.

This is simpler than trying to control postgres permissions granularity. Even if one application that connects to a database gets owned, it doesn't have access to other postgres databases, preventing data leaks/exfiltration.

2. Use a virtual machine or application container based runtime for your containers.

Kata containers is a container runtime, that is virtual machine.

There is also Gvisor and Syd Box, which are application kernels. Application kernels are reimplimentations of the parts of the Linux kernel needed to run apps, and in this case both Gvisor (Go) and Syd Box (Rust) are in memory safe langauges.

Kata containers are faster, but you will need nested virtualization in order to use them. Application kernels are slower, but you can install them anywhere, including hosts where virtualization is disabled (like a VPS that doesn't let you enable nested virtualization.

Both take a tiny bit more resources intensive due to no longer being able to share the host kernel, but for most part, it is worth it. They don't bring an entire kernel along, just what is needed to run apps.

Both offer similar levels of isolation, and preventing applications running inside them from touching the host kernel directly. They effectively manage to prevent issues like copy fail, dirty frag, and so on, from owning your host.

They are fairly easy to install, docker has some docs here: https://docs.docker.com/engine/daemon/alternative-runtimes/ . But if you are using podman or kubernetes, you can also install them there.

3. Enable automatic security updates (and reboots) on stable distros.

A large part of the draw of stable Linux like Debian or Red Hat, is that they only do security updates. They don't do feature updates, or even bug fixes (except for critical ones). In doing so, there is essentially a guarantee of reliability, where it is impossible for updates to break anything.

This makes it possible to enable automatic security updates, and you can even configure it to automatically reboot in order to load a new kernel that includes mitigations against issues like dirty frag. Make sure your docker containers are configured to automatically restart and everything will be smooth.

"Just patch" is a good but it is never enough, and I am frustrated hearing it so frequently. The way I view it is, any time I have to patch, what I really need to do is to improve my security architecture so I never have to "patch" this specific issues again. Patches are the exact kind of security toil that I complain about in this comment.

Somewhat reminds me of:

https://holdtherobot.com/blog/linux-on-android-with-ar-glasses/

Did you reboot?

It could be that a system service was installed, and activated. This service could stay running even after the packages are removed, since the programs would remain in memory.

Thank you so much. This was so annoying. Although another comment mentions that this appears to be specific to samsung devices, and doesn't work on general android/aosp.

For those the ADB solution another comment mentions probably must be used.