I have a cluster already. The “enterprise” repo is functionally the same as the free one.
It’s ok. It works. The limitations and UX leave a lot to be desired.
I really don’t like the storage aspect. I’m not using ceph or anything like a DAS or SAN so I’m sort of limited to having identical hardware and configs on all nodes. Migrating from one node to another that doesn’t have identical storage naming convention seems to be impossible, and that is very strange to me. I just want to pick a compute and a storage available defined for VMs and for it to simply move from one named storage to another.
Really don’t like that there is no mechanism to live migrate LXC either, needs a reboot of the lxc. Totally fine for small scale home use cases but still very strange to not have the same behavior. A vm is a vm, storage is storage, take the snapshot and find a way to make it happen ffs.
Maybe a bit of this is just because I’m using prosumer/smb grade gear and I don’t have identical nodes hardware wise, but overall it is very strange. Stuff like not allowing watcher node VMs for HA unless you have an even amount of nodes, but why not let me setup two+ watchers? If I’m removing a third node to replace it I would rather have the watcher on standby until I shut it down or put said node on maintenance mode (don’t think this exists for proxmox, but the concept helps trigger loads of automation and things like DRS.)
VMware is so stupidly easy. I just want proxmox to be a bit different. VMware is totally dead to me, but it hasn’t had a ton of real changes or feature adds that I l would ever use in the past ten years. Hard for me to be so forgiving that proxmox has been around for so long and doesn’t live up to the VMware of ~2012 based on my remembrance anyway.
last thing missing in your post is a complaint for slow migration on a 1gbit link…
you can complain as much as you want because proxmox is not vmware or you can embrace proxmox and make the effort to move/update/refresh your knowlwdge on to the new environment.
I am not sure what you are talking about. I have a an hpc cluster at work that I manage and I set it up with proxmox. 13 nodes with various VMs. If there is no pci passthrough on a VM I can live migrate it on any node without stopping it.i am not using LXC there so I don’t know if that can be done or not.
I have used ceph but we still don’t have the proper infrastructure for it at the moment.
i am not using LXC there so I don’t know if that can be done or not.
Yeah, it’s an lxc thing man. Sounds like you exclusively work with KVMs so you wouldn’t have come across it at all. They’re containers that in the case of proxmox run using a bit of the host’s files. LXD, a competing open source virtualization product, can run and live migrate LXCs. Proxmox devs know about the limitation at least as early as 2017 and in theory it’s on some kind of extreme long running back-burner to enable LXC live migration.
My other VMs can migrate, though the storage thing can be a pain if you have a dissimilar node in the pool, but that’s more of an engineering challenge. Proxmox does not hold your hand with practically anything and their manual pages offer scant advice in terms of best practice.
I have a cluster already. The “enterprise” repo is functionally the same as the free one.
It’s ok. It works. The limitations and UX leave a lot to be desired.
I really don’t like the storage aspect. I’m not using ceph or anything like a DAS or SAN so I’m sort of limited to having identical hardware and configs on all nodes. Migrating from one node to another that doesn’t have identical storage naming convention seems to be impossible, and that is very strange to me. I just want to pick a compute and a storage available defined for VMs and for it to simply move from one named storage to another.
Really don’t like that there is no mechanism to live migrate LXC either, needs a reboot of the lxc. Totally fine for small scale home use cases but still very strange to not have the same behavior. A vm is a vm, storage is storage, take the snapshot and find a way to make it happen ffs.
Maybe a bit of this is just because I’m using prosumer/smb grade gear and I don’t have identical nodes hardware wise, but overall it is very strange. Stuff like not allowing watcher node VMs for HA unless you have an even amount of nodes, but why not let me setup two+ watchers? If I’m removing a third node to replace it I would rather have the watcher on standby until I shut it down or put said node on maintenance mode (don’t think this exists for proxmox, but the concept helps trigger loads of automation and things like DRS.)
VMware is so stupidly easy. I just want proxmox to be a bit different. VMware is totally dead to me, but it hasn’t had a ton of real changes or feature adds that I l would ever use in the past ten years. Hard for me to be so forgiving that proxmox has been around for so long and doesn’t live up to the VMware of ~2012 based on my remembrance anyway.
Your storage issue sounds self inflicted.
Are you using random HSAs in each node or what? Not sure how you can otherwise not have das or san…
last thing missing in your post is a complaint for slow migration on a 1gbit link…
you can complain as much as you want because proxmox is not vmware or you can embrace proxmox and make the effort to move/update/refresh your knowlwdge on to the new environment.
I am not sure what you are talking about. I have a an hpc cluster at work that I manage and I set it up with proxmox. 13 nodes with various VMs. If there is no pci passthrough on a VM I can live migrate it on any node without stopping it.i am not using LXC there so I don’t know if that can be done or not.
I have used ceph but we still don’t have the proper infrastructure for it at the moment.
Yeah, it’s an lxc thing man. Sounds like you exclusively work with KVMs so you wouldn’t have come across it at all. They’re containers that in the case of proxmox run using a bit of the host’s files. LXD, a competing open source virtualization product, can run and live migrate LXCs. Proxmox devs know about the limitation at least as early as 2017 and in theory it’s on some kind of extreme long running back-burner to enable LXC live migration.
My other VMs can migrate, though the storage thing can be a pain if you have a dissimilar node in the pool, but that’s more of an engineering challenge. Proxmox does not hold your hand with practically anything and their manual pages offer scant advice in terms of best practice.