197
Now that vmware is over, what should I move to? (lemmy.world)
submitted 8 months ago by dlundh@lemmy.world to c/selfhosted@lemmy.world
60 comments fedilink hide all child comments

With free esxi over, not shocking bit sad, I am now about to move away from a virtualisation platform i’ve used for a quarter of a century.

Never having really tried the alternatives, is there anything that looks and feels like esxi out there?

I don’t have anything exceptional I host, I don’t need production quality for myself but in all seriousness what we run at home end up at work at some point so there’s that aspect too.

Thanks for your input!

top 50 comments
sorted by: hot top controversial new old
[-] anamethatisnt@lemmy.world 101 points 8 months ago* (last edited 8 months ago)
  • KVM/QEMU/Libvirt/virt-manager on a Debian 12 for minimal installation that allows you to choose backup tools and the like on your own.
  • Proxmox for a mature KVM-based virtualizer with built in tools for backups, clustering, etcetera. Also supports LXC. https://github.com/proxmox
  • Incus for LXC/KVM virtualization - younger solution than Proxmox and more focused on LXC. https://github.com/lxc/incus
[-] vegetaaaaaaa@lemmy.world 18 points 8 months ago* (last edited 8 months ago)

/thread

This is my go-to setup.

I try to stick with libvirt/virsh when I don't need any graphical interface (integrates beautifully with ansible [1]), or when I don't need clustering/HA (libvirt does support "clustering" at least in some capability, you can live migrate VMs between hosts, manage remote hypervisors from virsh/virt-manager, etc). On development/lab desktops I bolt virt-manager on top so I have the exact same setup as my production setup, with a nice added GUI. I heard that cockpit could be used as a web interface but have never tried it.

Proxmox on more complex setups (I try to manage it using ansible/the API as much as possible, but the web UI is a nice touch for one-shot operations).

Re incus: I don't know for sure yet. I have an old LXD setup at work that I'd like to migrate to something else, but I figured that since both libvirt and proxmox support management of LXC containers, I might as well consolidate and use one of these instead.

[-] anamethatisnt@lemmy.world 10 points 8 months ago

I use cockpit and my phone to start my virtual fedora, which has pcie passthrough on gpu and a usb controller.

Desktop:

Mobile:

[-] Voroxpete@sh.itjust.works 6 points 8 months ago

We use cockpit at work. It's OK, but it definitely feels limited compared to Proxmox or Xen Orchestra.

Red Hat's focus is really on Openstack, but that's more of a cloud virtualization platform, so not all that well suited for home use. It's a shame because I really like Cockpit as a platform. It just needs a little love in terms of things like the graphical console and editing virtual machine resources.

[-] Cyber@feddit.uk 5 points 8 months ago

Ooh, didn't know libvirt supported clusters and live migrations...

I've just setup Proxmox, but as it's Debian based and I run Arch everywhere else, then maybe I could try that... thanks!

[-] vegetaaaaaaa@lemmy.world 6 points 8 months ago* (last edited 8 months ago)

In my experience and for my mostly basic needs, major differences between libvirt and proxmox:

  • The "clustering" in libvirt is very limited (no HA, automatic fencing, ceph inegration, etc. at least out-of-the box), I basically use it to 1. admin multiple libvirt hypervisors from a single libvirt/virt-manager instance 2. migrate VMs between instances (they need to be using shared storage for disks, etc), but it covers 90% of my use cases.
  • On proxmox hosts I let proxmox manage the firewall, on libvirt hosts I manage it through firewalld like any other server (+ libvirt/qemu hooks for port forwarding).
  • On proxmox I use the built-in template feature to provision new VMs from a template, on libvirt I do a mix of virt-clone and virt-sysprep.
  • On libvirt I use virt-install and a Debian preseed.cfg to provision new templates, on proxmox I do it... well... manually. But both support cloud-init based provisioning so I might standardize to that in the future (and ditch templates)
[-] Voroxpete@sh.itjust.works 2 points 8 months ago

My understanding is that for proper cluster management you slap Pacemaker on there.

load more comments (3 replies)
load more comments (4 replies)
[-] onlinepersona@programming.dev 3 points 8 months ago

Also VirtualBox.

[-] Voroxpete@sh.itjust.works 7 points 8 months ago

They're obviously looking for a type 1 hypervisor like Esxi. A type 2 hypervisor like virtualbox does not fit the bill.

[-] thanksforallthefish@literature.cafe 4 points 8 months ago

What is the difference between type 1 & 2 please ?

[-] RedEyeFlightControl@lemmy.world 12 points 8 months ago* (last edited 8 months ago)

Type 1 runs on bare metal. You install it directly onto server hardware. Type 2 is an application (not an OS) lives inside an OS, regardless of whether that OS is a guest or a host, the hypervisor is a guest of that platform, and the VMs inside it are guests of that hypervisor.

[-] thanksforallthefish@literature.cafe 2 points 8 months ago

Thank you

[-] Voroxpete@sh.itjust.works 4 points 8 months ago

The previous comment is an excellent summary. It is worth noting that there are some type 1 hypervisors that can look like type 2s. Specifically, KVM in Linux (which sometimes gets referred to as Virt-manager, Virtual Machine Manager, or VMM, after the program typically used to manage it) and Hyper-V in Windows.

These get mistaken for type 2 hypervisors because they run inside of your normal OS, rather than being a dedicated platform that you install in place of it. But the key here is that the hypervisor itself (that is, the software that actually runs the VM) is directly integrated into the underlying operating system. You were installing a hypervisor OS the whole time, you just didn't realise it.

The reason this matters is that type 1 hypervisors can operate at the kernel level, meaning they can directly manage resources like your memory, CPU and graphics. Type 2 hypervisors have to queue with all the other pleb software to request access to these resources from the OS. This means that type 1 hypervisors will generally offer better performance.

With hypervisor platforms like Proxmox, Esxi, Hyper-V server core, or XCP-NG, what you get is a type 1 hypervisor with an absolutely minimal OS built around it. Basically, just enough software to the job of running VMs, and nothing else. Like a drag racer.

[-] BearOfaTime@lemm.ee 3 points 8 months ago

VB is awful.

And I use it every day.

It's like a first-try at a hypervisor. Terrible UI, with machine config scattered around. Some stuff can only be done on the command line after you search the web for how to do it (like basic stuff, say run headless by default). Enigmatic error messages.

[-] garibaldi@startrek.website 3 points 8 months ago

This is what I would recommend too - QEMU + libvirt with Sanoid for automatic snapshot management. Incus is also a solid option too

[-] possiblylinux127@lemmy.zip 39 points 8 months ago

Proxmox works well for me

[-] umbrella@lemmy.ml 29 points 8 months ago

proxmox

[-] else@discuss.tchncs.de 2 points 8 months ago

This is the way

[-] egonallanon@lemm.ee 18 points 8 months ago

If you're running mostly Linux vms proxmix us really good. It's based on kvm and has a really nice feature set.

[-] krigo666@lemmy.world 17 points 8 months ago

Windows guests also run fine on KVM, use the Virtio drivers from Fedora project.

[-] oleorun@real.lemmy.fan 13 points 8 months ago

I've used Hyper-V and in fact moved away from ESXi long ago. VMWare had amazing features but we could not justify the ever-increasing costs. Hyper-V can do just about anything VMWare can do if you know Powershell.

[-] Raptor_007@lemmy.world 3 points 8 months ago

Another vote for Hyper-V. Moved to it from ESXi at home because I had to manage a LOT of Hyper-V hosted machines at work, so I figured I’d may as well get as much exposure to it as I could. Works fine for what I need.

[-] AtariDump@lemmy.world 3 points 8 months ago

Seconded for Hyper-V, and MUCH easier to patch the free edition than ESXi.

[-] purplemonkeymad@programming.dev 2 points 8 months ago

I use it with WAC on my home server and it's good enough for anything I need to do. Easy to create VMs using that UI, PS not even needed.

[-] pineapplelover@lemm.ee 8 points 8 months ago

Qemu/virt manager. I've been using it and it's so fast. I still need to get the clipboard sharing working but as of right now it's the best hypervisor I've ever used.

[-] caseyweederman@lemmy.ca 2 points 8 months ago

I love it. Virtmanager connecting over ssh is so smooth.

[-] exu@feditown.com 8 points 8 months ago

I'm pretty happy with XCP-ng with their XenOrchestra management interface. XenOrchestra has a free and enterprise version, but you can also compile it from source to get all the enterprise features. I'd recommend this script: https://github.com/ronivay/XenOrchestraInstallerUpdater

I'd say it's a slightly more advanced ESXi with vCenter and less confusing UI than Proxmox.

load more comments (1 replies)
[-] Decronym@lemmy.decronym.xyz 7 points 8 months ago* (last edited 7 months ago)

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

Fewer Letters More Letters
DNS Domain Name Service/System
ESXi VMWare virtual machine hypervisor
HA Home Assistant automation software
~ High Availability
LTS Long Term Support software version
LXC Linux Containers
ZFS Solaris/Linux filesystem focusing on data integrity
k8s Kubernetes container management package

[Thread #540 for this sub, first seen 24th Feb 2024, 11:35] [FAQ] [Full list] [Contact] [Source code]

[-] CubitOom@infosec.pub 6 points 8 months ago

If you are dipping toes into containers with kvm and proxmox already, then perhaps you could jump into the deep end and look at kubernetes (k8s).

Even though you say you don't need production quality. It actually does a lot for you and you just need to learn a single API framework which has really great documentation.

Personally, if I am choosing a new service to host. One of my first metrics in that decision is how well is it documented.

You could also go the simple route and use docker to make containers. However making your own containers is optional as most services have pre built ones that you can use.

You could even use auto scaling to run your cluster with just 1 node if you don't need it to be highly available with a lot of 9s in uptime.

The trickiest thing with K8s is the networking, certs and DNS but there are services you can host to take care of that for you. I use istio for networking, cert-manager for certs and external-dns for DNS.

I would recommend trying out k8s first on a cloud provider like digital ocean or linode. Managing your own k8s control plane on bare metal has its own complications.

[-] Dran_Arcana@lemmy.world 8 points 8 months ago

There are also full-suites like rancher which will abstract away a lot of the complexity

[-] TheWoozy@lemmy.world 2 points 8 months ago

K8s is great, but you're chaning the subject and not answering OPs question. Containers =/= VMs.

[-] CubitOom@infosec.pub 1 points 8 months ago

You are right. But proxmox and many of the other suggestions aren't vms either.

[-] Uglyhead@lemmy.world 6 points 8 months ago* (last edited 8 months ago)

I do do not not know— however, that logo is amazing

EDIT: Found it — https://sega-ai.neocities.org/

[-] Xartle@lemmy.ml 6 points 8 months ago* (last edited 8 months ago)

I actually moved everything to docker containers at home... Not an apples to apples, but I don't need so many full OSs it turns out.

At work we have a mix of things running right now to see. I don't think we'll land on ovirt or openstack. It seems like we'll bite the cost bullet and move all the important services to amazon.

[-] GeekFTW@kbin.social 5 points 8 months ago

OOTL and someone who only uses a vm once every several years for shits & grins: What happened to vmware?

[-] anamethatisnt@lemmy.world 12 points 8 months ago

As part of the transition of perpetual licensing to new subscription offerings, the VMware vSphere Hypervisor (Free Edition) has been marked as EOGA (End of General Availability). At this time, there is not an equivalent replacement product available.

For further details regarding the affected products and this change, we encourage you to review the following blog post: https://blogs.vmware.com/cloud-foundation/2024/01/22/vmware-end-of-availability-of-perpetual-licensing-and-saas-services/

[-] GeekFTW@kbin.social 4 points 8 months ago* (last edited 8 months ago)

Whelp..boo-urns. :(

[-] Evotech@lemmy.world 5 points 8 months ago

Minikube and try to get everything on Kubernetes?

[-] ikidd@lemmy.world 5 points 8 months ago

Where does running VMs compare in any way to what Kubernetes does?

[-] Evotech@lemmy.world 2 points 8 months ago

Depends on what you want to self host? Could be worth it to see if what you self host can be deployed as containers instead

[-] Lemzlez@lemmy.world 4 points 8 months ago

Kubernetes yes, but minikube is kinda meh as a way to install it outside of development environments.

There’s so many better manageable ways like RKE/Rancher (which gives you the possibility to go k3s),Kubespray or even kubeadm.

All of those will result in a cluster that's more suitable for running actual workloads.

[-] else@discuss.tchncs.de 2 points 8 months ago

I wouldn’t recommend going K8S only in a homelab. Too much effort and some things don’t fit well (Home Assistant, Gaming VM?)

load more comments (2 replies)
[-] dan@upvote.au 5 points 8 months ago

I know everyone says to use Proxmox, but it's worth considering xcp-ng as well.

[-] nexusband@lemmy.world 10 points 8 months ago

In my "testing" at work and private, PVE is miles ahead of xcp-ng n terms of performance. Sure, xcp-ng does it's thing very stable, but everything else...proxmox is faster

load more comments (1 replies)
[-] slazer2au@lemmy.world 4 points 8 months ago

For home have a crack at KVM with front ends like proxmox or canonical lxd manager.

In an enterprise environment take a look at Hyper-V or if you think you need hyper converged look at Nutanix.

[-] corsicanguppy@lemmy.ca 2 points 8 months ago

I'm moving to oVirt.

Proxmox was out and oVirt was an excellent fit.

Choose carefully ; don't just go with the herd.

[-] rtxn@lemmy.world 6 points 8 months ago

What does oVirt offer that proxmox doesn't? I'm asking because I want to move an ESXi server to another hypervisor, I'm 90% sure it'll be Proxmox, but I'd like to know my options.

load more comments
view more: next ›
this post was submitted on 24 Feb 2024
197 points (100.0% liked)

Selfhosted

40134 readers
485 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
HybridSarcasm@lemmy.world
HybridSarcasm@lemmy.hybridsarcasm.xyz