Last week, I intended to replace VirtualBox with QEMU for my personal use. The spartiate documentation made it really hard. I lack the competence to seriously contribute.
The official documentation just explains the numerous parameters. It also points to tutorials (wikibook, etc), which unfortunately are very basic and out of sync. Even Arch Linux does not provide a good documentation (lack of clarity, some obsolete syntax, "Networking" section is a mess...). Like often, Debian's wiki is obsolete, and even their QEMU images are 5 years old. I had to combine the following 3 sources and a lot of try and guesses to build my VMs.
Compare and contrast the number of devices that libvirt adds.
FWIW, I strongly recommend to use libvirt to launch your QEMU-based guests. It is far more effective, and more importantly, provides security infrastructure like running the QEMU process as an unprivileged user, protecting the QEMU process (and its associated disk iamges) via the SELinux-based sVirt mechanism, and so forth.
Check out these[2] slides (recording is online, too) on "Security in QEMU" from one of the QEMU maintainers, Stefan Hajnoczi, at last year's KVM Forum in Edinburgh.
There's a lot to be said for running qemu from the command line, vs the complexity and opaqueness of libvirt. For example you can put the invocation in a script and check it into version control, and it will work on anyone's Linux box. You can easily debug the config and be suire about what options qemu was given.
Yes, there may be many switches in the invocation, but at least they are in one place, the invocation syntax rarely changes and works accross distributions, and they are applied from scratch in each invocation rather than having lots of implicit state hidden in daemons and their configurations / databases.
Your example is nicer on the eyes written in a config file style and leaving out the unnecessary options:
I can see the value in "putting the command-line in a script and launch it anywhere". Many of us use bespoke QEMU command-line scripts for test and development. And I even know a small hosting provider who manages their production guests entirely via QEMU command-line "config file" syntax (-writeconfig and -readconfig options—they're little-known, not least because they don't cover all the options; upstream has some proposals to address it).
I appreciate that libvirt may not be a suitable option for some. That said, you might want to consider the below points on where it adds value (for production VMs; not talking about test and development setups):
• It's not about having all the QEMU "switches available in one place". The dizzying array of switches make it trivial to let you shoot yourself in the shoulder (e.g. getting fiddly details like PCI device addressing right manually).
• Knowing the "best practice" command-line will get you optimal performance out of your (production) guest—libvirt handles that for you. And if the best practices change, libvirt will automatically handle that for you, as it keeps track of "QEMU command-line best practice evolution".
• Launching the command-line is only the first step. If it's a production guest, at some point you might need to live-migrate the guest, or take a live disk backup, or if your guest has a long disk image backing chain, you want to "merge" the disk images into a single, coalesced image—all of this without the guest going offline. These tasks can be done manually (assuming you launched QEMU the 'right away' upfront) via extremely tedious and error-prone ways using QEMU's JSON-based run-time interface, QMP (QEMU Machine Protocol). But that's too risky and a colossal waste of time.
• Speaking of QMP, the run-time interface, it has even more dizzying array of commands—to be more precise, as of 2015: QMP had about 126 commands + 33 events. And more than 700 named arguments and results. Try keeping track of all of that manually. It reminds me of how once a QEMU sub-maintainer memorably compared (in his 2015 talk, "QEMU interface introspection: From hacks to solutions") the volume of QMP schema to old religious books: "QMP schema is larger than "Gospel of Luke", but smaller than "Genesis". :-)
IOW, I'm yet to see anyone who launches multiple guests (on the moderate scale of hundreds of VMs) use direct QEMU in production. There are odd examples, as I noted earlier, but it's certainly not the norm.
But NixOS has a neat feature where any (?) NixOS configuration can trivially instead be built as a "vm", which will assemble an initrd for the described system along with a script which will run a tailored qemu invocation to launch the system, leaving a qcow2 file of any changes you make to the filesystem in your cwd. Makes the whole thing extremely usable, and is done in a very extensible way.
Once you add `-nodefaults`, the list of arguments grows because now you must add a bunch of things that were there by default. If by "minimal command-line" you mean "the simplest list of arguments", then your example is wrong. If by "minimal command-line" you mean "command-line that gives the minimal VM", then you're being misleading, because that's not how people will take it.
Most of the devices added by libvirt would be included by qemu automatically if you didn't say `-nodefaults`. libvirt uses `-nodefaults` and specifies its own defaults, so that it doesn't need to keep track of changes in Qemu's defaults between versions of Qemu (since libvirt cares about the nitty-gritty more than most humans do).
PS: You have too many "t"s in the "htttps://" in one of those URLs.
> Once you add `-nodefaults`, the list of arguments grows because now you must add a bunch of things that were there by default [...]
Partly true (refer below).
> If by "minimal command-line" you mean [...]
I appreciate your corrections, but please don't read too much into the word "minimal". I just quickly pasted it from one of the "subjective scripts" named `min-qemu.sh` lying around on my file system. By the time I realized, it was too late to adjust it.
> Most of the devices added by libvirt would be included by qemu automatically if you didn't say `-nodefaults`
Not quite; I just double-checked with an upstream libvirt dev: when you remove `-nodefaults`, some of the built-in devices get removed, whether those equate to "most" or not depends how many devices you've asked for.
> PS: You have too many "t"s in the "htttps://" in one of those URLs.
> like running the QEMU process as an unprivileged user,
Anyone can run your example "qemu-system-x86_64 -display none -no-user-config -nodefaults -m 2048 -device virtio-scsi-pci,id=scsi -device virtio-serial-pci -serial stdio -drive file=/export/cirros.qcow2,format=qcow2,if=virtio" from the command line/bash, as non-root user, yet libvirt which requires root to install, and is expecting root to configure it, is better? How?
Libvirt is one of those softwares which ads a layer of complexity for the sake of complexity, yet offers nothing of value in return.
its not true that libvirt adds nothing of value, it adds easy networking support, easy snapshots, a nice gui interface, a very nice remote protocol and clients (that seamlessly support both graphical and serial output, aswell as on the fly reconfiguration of VMs without restarting them), template VMs (including one time initial provision setup), easy support for storage pools including remote ones, an easy way to manipulate RAM usage on live VMs, easy PCI and USB passthrough, live migration to other hosts, etc etc.
Sure its possible to do all this without libvirt, but it comes as one integrated, easy to manage package instead of hundreds of disparate parts.
You might not need (all) these features, but to claim libvirt is useless is extremely parochial.
Its also untrue that it needs root, you can virt-manager for instance without a root libvirtd instance at all, and it will run VMs without any permissions needed.
> its not true that libvirt adds nothing of value, it adds easy networking support, easy snapshots, a nice gui interface, a very nice remote protocol and clients (that seamlessly support both graphical and serial output, aswell as on the fly reconfiguration of VMs without restarting them), template VMs (including one time initial provision setup), easy support for storage pools including remote ones, an easy way to manipulate RAM usage on live VMs, easy PCI and USB passthrough, live migration to other hosts, etc etc.
everything libvirt does can be done without it by talking to qemu directly. and yes, Ive done many of the above "by hand", it was easier to read man qemu and other qemu docs than to swallow libvirt.
No, not everything it does can be done by qemu directly, plenty of it lies outside of the scope of QEMU entirely, like managing networks (not just network devices), or storage pools, setting up permissions for device redirection, or doing the legwork for VM provisioning and configuration (including net booting and automatically running appropriate scripts), or managing access to VMs both locally and remotely.
Its fine if your use cases are covered by plain QEMU, but libvirt does a hell of a lot more than plain QEMU alone is capable of.
Again all of that is possible via other means, but when other means involve recreating half of libvirt in shell scripts or other code, id rather let somebody else do the work for me.
QEMU is designed to run virtual machines, libvirtd is designed to manage VM server infrastructure.
For one static network, what about complicated inter VM networking schemes (including firewalling) that need to be dynamically adjusted as VMs go on and offline?
Libvirtd is much better suited to managing that sort of complexity.
I have the luxury of opining that such complicated networking schemes should be reined in, and not served with baroque management software. Complexity is the arch enemy of security, because systems must be kept easy to reason about.
Indeed. Instead of contributing a patch to qemu with better defaults, or even support for config files, someone wrote tens of thousands of lines of complex code.
People forget that software is malleable. Bend it to your needs, instead of adding fragile wrapper layers.
libvirt-the-software is tens of thousands of lines of complex code that doesn't really add much. libvirt-the-project does contribute patches in to qemu, and a lot of the reason that libvirt-the-software doesn't add much of value is because libvirt-the-project worked to get capabilities in to upstream qemu.
The libvirt team definitely does view qemu as malleable, and bends it to their needs.
The problem is that the libvirt authors contributing to qemu look at qemu as a machine-interface (to be consumed by libvirt), not as a human-interface. Many of the more poorly documented or otherwise difficult-for-humans flags were contributed by the libvirt team. And they're useful! They make it so qemu can do things it couldn't do before. But they're also kinda the reason qemu is difficult to use without libvirt.
> Libvirt is one of those softwares which ads a layer of complexity for the sake of complexity, yet offers nothing of value in return.
Isn't that sort of like saying the system startup scripts/subsystem adds complexity for the sake of complexity, and you should just use ifconfig and route manually in some script after startup?
In both cases, that you can fall back to the underlying commands is great. That you should use them instead of the normalized interface provided is debatable.
Unless the default changed one of these last few years, I'm pretty sure if you don't use `-enable-kvm` then qemu emulates the selected architecture. That's probably why it doesn't need root.
EDIT: Nevermind, `-enable-kvm` also doesn't need root. I'm not sure what libvirt uses root for.
Hear, hear. QEMU is excellent software, and I don't find that libvirt adds any value to its use.
Instead, one of the more annoying things I have ever done was foolishly installing libvirt and having it do all kinds of whackiness to a previously excellent QEMU set-up. I still cannot figure out where it managed to change all QEMU invocations to be VNC-based.
That's far more than I typically put in my qemu calls. You can technically be fine with just:
qemu-system-x86_64 foobar.qcow2
but I always add `-enable-kvm` and use `-monitor stdio` or `-nographic` depending if I want a graphic display from the vm or not. `-nographic` is all I need to give me access to a serial console in the guest (besides configuring the guest kernel with the paramater `console=ttyS0`). I also add `-m` because the default amount of memory is just 128 MiB, which is typically not enough for my use. Sometimes I add network related arguments, but we're going beyond minimal then.
There's not really much choice than trying to use qemu command line directly on windows. I didn't find any single useful GUI that was usable or newer than 3 or 4 years ago. It's a pretty sad state of affairs. I spend hours until I discovered that I needed to add some magic flag called machine for acceleration even work.
Using QEMU directly is a bit tricky and it's not really intended to be used in that fashion by most users, unlike VirtualBox.
Both gnome-boxes and virt-manager use QEMU under the hood though, and they provide a point-and-click experience. I use virt-manager myself and I do not think it's any more difficult than VirtualBox is.
We have several windows vms running in virtualbox. All have rdp/vnc for specialised programs. (These are offline).
Everytime I tried to migrate to qemu or virt-manager or gnome-boxes it was a real nightmare especially installing SPICE+virtio drivers+ networking. I read in Phoronix the performance of virtualbox is subpar and all KVM/qemu is the way to go.
Please, could the OSS community build a easy to use GUI that SPICE+virtio drivers+ networking is made simple. Any one from qemu, I really appreciate the work but still very challenging for sysadmins.
About 12 years ago when I had access to Microsoft MSDN I got a big SuperMicro box, set it up with CentOS 4 (RHEL) Linux, and used QEMU/KVM to set up a Windows domain controller and several other Windows VMs, and several Linux VMs, on a couple of virtual subnets, along with iptables port forwarding, Windows Remote Desktop, VNC, and many services. Some of the Linux VMs services authenticated with the Windows domain. I wanted to set up at least one non-x86/x64 emulated-CPU VM but never got that working very well in QEMU.
The exercise was a good (and incremental) learning experience and increased my general networking and sys-admin knowledge, which I've used well as part of my software engineering career. I would have been lost in many professional situations without it. QEMU/KVM are awesome.
Virt-manager seems to be written in Python. So if libvirt supports the bits you want then with some tinkering in GTK you might be able to bolt the controls onto Virt-manager, at least to get things started.
I confess that I haven't run Windows with virt-manager on my own desktop in a long while, but it worked just fine after installing the required drivers at install time. IIRC Redhat distributes signed drivers for Windows for free.
The problems I did have were with VMs I tried to convert from VMware to KVM, and those stemmed from Windows' inability (version 7) to properly handle changing hardware. Perhaps it's better at nowadays.
I mainly use windows + libvirt for gaming. But I also have an instance on a dedicated server for a remote development environment.
I would say forget spice + libvirt graphics layer. Use RDP, if on linux remmina work's great. RDP will outperform in my experience SPICE. I've also had constant troubles with the vga/qxl/virtio graphics.
There is an iso at the above link. Grab that and have it added during install. When you get to the select a drive. None will show up until you load the virtiostorage driver.
There are a lot of nuances to first setting up a windows vm. But the performance is great. It has removed the need for a windows install for gaming at all.
TBH, I tend to give up on the accelerated video drivers on Windows inside of virt-manager. It is often easier to just use RDP for basic office applications.
My recommendation for that sort of use would be to look at using a 'management layer' app that sits above QEMU and handles all the fiddly command line details for you -- https://virt-manager.org/ (which uses/is part of libvirt) is a popular one, I think. The QEMU project has kind of settled on the philosophy of being the low-level tool which provides you "full manual control" and interfaces for a higher-level UI to make use of. User-friendliness is not really our forte and so we let other projects provide that part.
This is the context I was missing when I switched to QEMU. I had not heard about libvirt. A simple "apt search qemu" had shown a few unsatisfying GUIs, but nothing about other layers above qemu.
I suggest QEMU's documentation could mention libvirt and its user-friendly interfaces.
FWIW, QEMU's upstream README does mention libvirt and higher-level managment tools:
"QEMU aims to fit into a variety of use cases. It can be invoked directly
by users wishing to have full control over its behaviour and settings.
It also aims to facilitate integration into higher level management
layers, by providing a stable command line interface and monitor API.
It is commonly invoked indirectly via the libvirt library when using
open source applications such as oVirt, OpenStack and virt-manager."
But I get it, not everyone goes in search of a README buried in a Git repository. I myself am not sure if this REAMDE is rendered in an HTML format somewhere.
actually, having gone through the process of successfully setting up the vm’s make you eminently suitable to contribute documentation knowledge that is sorely lacking in the online resources you have pointed out.
i am sure it will be greatly appreciated and will get improved upon areas that are missing etc.
After a decade of only using qemu via libvirt on the server, and lazily using VirtualBox on the desktop, this post convinced me to start using qemu locally. I run unprivileged guests for development and testing and it fits my workflow perfectly for about six months now.
> There’s really no excuse to be using any other hypervisor [1]. They’re all dogshit compared to qemu.
> [1] Especially VirtualBox. If you use VirtualBox after reading this article you make poor life choices and are an embarrassment to us all.
Well I'm convinced! What a well-presented article!
Appropriate Futurama quote: "Remember when I told you about always ending your stories a sentence earlier?".
Drew was doing well until he ended with this unnecessary condescending and hostile conclusion (and if you think it's funny, note that "the failure mode of smartass is asshole"). It's a trend of his. He's got interesting and informed things to say, but it's hard to really take him seriously because of this.
There are several higher level frameworks that wrap QEMU-KVM and make it easier to use. oVirt, libvirt and proxmox are among my favourites, with the latter being quite polished and with advanced features for HA use.
Rather than using QEMU by itself, it might be advisable to use libvirt..? As somewhat of a newbie to virtualization, that's how I started - googling how to use libvirt and virsh, and I found a lot of resources that helped me to deploy what I needed... What are your painpoints in particular? I.e. which topics would you like to explore deeper and can't due to poor documentation/resources? We could work on improving the resources.
I personally dislike libvirt a lot, and virsh in particular. I think that it's pointlessly complicated and burdensome to use, and that in general the hypervisor-agnosticabstractions provided by libvirt are nullified by the fact that no backend except Qemu really works well, this making them a pointless exercise in futility.
I agree with you whole-heartedly. Instead of running one simple qemu command for each vm you instead have hundreds of lines of xml split into many different files. Files you need to run through libvirts session manager that needs to implements its own pretty pointless state management (this could be handled by systemd instead). If there is a nice feature in qemu you want to use, not only do you need to know how to define it in libvirts xml, it might not even be supported at all.
I believe his pain point is that with Virtual Box you get a very simple GUI where you can get VM up and running very quickly without having to use Google or read any guides at all.
We need to spin up a lot of VMs of various Linux distributions for testing Flockport and we found the existing tooling too heavy and involved.
In the end we rolled our own solution that is much more lightweight, uses QEMU directly and provides better visibility, ease of use and even some potentially useful networking features. Here are a few screencasts [1].
This is mainly designed as a module for Flockport and we will preview this soon.
When I was running a Fedora desktop and was doing a lot of dev/test work on VMs+Foreman+Puppet, I used Libvirt+QEMU as the VM hypervisor.
I ran Foreman inside a CentOS VM, which talked to my desktop's Libvirtd to provision new VMs, hand out IPs via DHCP, boot via TFTP, then kickstart. It worked pretty well, and helped me to spin up a vcenter foreman controlled dev environment for a few hundred users in just a few days.
As opposed to what? Most proprietary tools don't have good documentation either. And then you may not even be able to look at the code to see what it does.
I am on my third confusing error using virt-manager to create an armv7 build environment. First one was some ca-cert error, which meant I forgot to type a slash somewhere in some URL, second was something about not being able to enable firewall, which required firewalld and dnsmasq to be installed, now I'm having an error adding a qcow2 image to my vm setup, it probably doesn't like my mounted storage. We can compare this experience to terrible vendor lockins.
Yeah, but VirtualBox & co. come as a package. virt-manager and Gnome Boxes, as you can see from the other comments, have been missed by various people trying to use these tools :)
I'm impressed nobody mentioned libvirt (https://libvirt.org/) here, an open source framework for managing all kind of virtualization platforms. It is especially well suited to take off all heavy work of managing QEMU. From the features, Virt-Manager (https://virt-manager.org/) can compete with VirtualBox -- and is probably well beyond, thanks to network transparency, the Spice protocol, real guest console access, powerful CLI interfaces such as `virsh` and other goodies.
In Linux distributions, you basically install `virt-manager` and the package manager will resolve all dependencies to provide you a fully working virtualization desktop software which is well capable to be used on servers.
I do this using Proxmox (QEMU under the hood), and it's been mostly a nice experience. I currently run 3 VMs on a single Linux box with 3 GPUs (one per VM via VFIO passthrough). I've used both NVIDIA and AMD graphics cards with success. Before Mojave, NVIDIA cards were easier with their web driver, but Apple and NVIDIA don't get along in Mojave (both companies are blaming each other) and there is no web driver. So for macOS, I've switched to an AMD GPU which works mostly out of the box, apart from needing a small tweak in the Linux host's kernel parameters ("disable_idle_d3=1" to avoid a bug preventing the device from being used after a VM restart).
To get macOS to see my GPUs with full acceleration and audio over HDMI, I use the Clover bootloader to inject a small section into the DSDT, mapping the emulated PCI address to a device that actually looks like a graphics card to macOS. This same DSDT patch works for both NVIDIA and AMD GPUs (the ones I've tried anyway).
The best thing about Hackintosh in a VM is that you no longer need to fear macOS upgrades. Just create a snapshot, try the upgrade, and if it doesn't work out, you can restore, investigate, and try again.
LTT's video is a nice overview, but his claim that "because this is a VM... it will run on any hardware" isn't quite true when talking about GPUs. Unlike the CPU, memory, and some other devices, the GPU is seen (nearly) directly by the guest VM, so guest OS support is in fact required to see the advantages of GPU passthrough.
In summary, it's never going to be super easy, but it's not too difficult either, and the performance (and flexibility) is actually really great.
Thats sounds pretty interesting. Ive been running a hacked together windows VM gaming setup via qemu with vfio passthrough for the past 1.5 years on my local linux server. Im pretty pleased with it. Ive got it down to simply executing a small script and everything comes up. It could be even more automated using evdev potentially (recognizing mouse/keyboard events).
I was thinking about expanding the setup to include a beefier linux host (more gpus and better hardware) and more seats -- windows gaming for me and osx for my wife. But you mentioned several possible osx specific caveats: amd gpus, clover (?) bootloader w/ dsdt (?), disable_idle_d3, etc. Can you link to some info or READMEs regarding your setup or how you came across these solutions?
In case anyone runs across similar AMD GPU issues, the "disable_idle_d3=1" option in the second pastebin link (when used as a kernel parameter) should be prefixed as "vfio-pci.disable_idle_d3=1". Even then, it can still be hit-or-miss with some cards (it works most of the time for me, but some VM restarts still find that the GPU has entered D3 and won't come back without rebooting the host).
I believe it happens for any guest OS (search for "amd reset bug"). Some people are using a workaround that involves removing the device (in the sysfs), suspending the host, and then rescanning, but for me, that's not much better than a host reboot, so when the AMD GPU occasionally does get stuck in D3 (preventing starting the VM after a VM reboot), I just reboot the host. It doesn't happen very often, and when it does, it's only when I'm in the process of rebooting the VM anyway.
Previously, I also attempted using a kernel patch that's been floating around (DECLARE_PCI_FIXUP_HEADER ... quirk_no_bus_reset), but it just gives me corrupt graphics when rebooting the VM.
I am building a VFIO build this weekend. one of the goals is to power a macOS VM for work. if I remember I will update you on my experience.
I had success getting macOS running on QEMU with my existing PC, but its performance suffered without a dedicated GPU and CPU pinning, which my new build will be configured to handle
Note that this isn't a "major" change from v3.x.x; the major version is year-based (${year}-2015). This is v4.0.0 because it is the first release of 2019.
Exactly! We are using this in our compiler test farm for the quick test runs. Sadly, the release prior to 4.0.0 broke floating point handling of NaN on (at least) PowerPC e5500 :( I'm looking forward to test qemu 4.0.0 once I'm back in the office again.
Edit: Tested it, aaaaaand NaN compares are still broken :(
QEMU is one of the core building blocks for many of the KVM-based data center virtualization and infrastructure projects (e.g. OpenStack, oVirt, KubeVirt, etc). And is used by a wide variety of other projects for various use cases (e.g. `systemd` upstream uses QEMU to speed-up development/testing).
When you launch a KVM-based guest, most often you would see QEMU doing the major user space heavy-lifting by providing a PC-like environment—it handles guest memory management, network cards, emulated storage devices, and so forth. QEMU also has a robust Block Layer; not to mention its versatile, and widely-used, native disk image format, QCOW2.
PS: I am biased, as I work with QEMU on a daily basis :-)
I'm using it in conjunction with PCI passthrough instead of dual boot.
I've been running Windows 10 for more than a year now and played all kinds of AAA titles successfully. The performance-loss seems minimal.
Recently I managed to get a second GPU and spin up another VM, so now me and my partner can play on the same machine, heaving each a keyboard, mouse and screen.
Do you have any hints on how to set up a similar system? Links, guides, gotchas, whatnot? I've seen someone mention this setup before (perhaps it was you?) and I'm sure I'm not the only one interested in running something similar.
I do the same, I used to run my VMs off of ZFS with spinning disks, but that was too slow at times, so my last upgrade has been giving the VMs their own SSDs and that has worked out great, I then backup the VMs to ZFS and snapshot them.
You can write a system configuration, generate a QEMU VM and run it. Also all NixOS automatic tests[1] are running in QEMU VMs that are generated from a Nix configuration.
The most beautiful thing is that you can create multiple VMs and run them simultaneously to create a virtual network. For example you can test a client-server application by running the client and the server on two hosts or test some network configuration[2].
I've been using it on my personal laptop for the past two months.
There are a lot of rough edges to the desktop experience, but it generally works. And while that's faint praise, I have no current plans to switch back to Fedora (3 years) or Debian (12 years prior to that).
It's comforting to know that my entire base system configuration is declarative, and it's pleasant to not worry about libraries conflicting or unnecessary packages lingering around.
Immediate, ephemeral access to programs via, e.g., `nix-shell -p pdftk` is extremely convenient: they're only available in that shell session, and they (along with their dependencies) get cleaned up the next time you run `nix-collect-garbage`. Super handy.
I use qemu to run multiple virtualized machines on a single beefy (32-core) headless box; including a shared build-server, and my own remote-workstation.
I got sick of trying to figure out which thing I needed to click in virt-manager to get it to pass the flags I wanted, so I wrote my own minimal qemu-wrapper that covers all of the functionality from libvirt that I care about (vCPU pinning, binding to NUMA nodes, running as non-privileged user), and integrates better with the other system (systemd) monitoring than libvirt does. https://git.lukeshu.com/systemd-qemu/
----
I'm also involved with the Parabola GNU/Linux-libre distribution. Qemu user-mode emulation is essential to our support of non-x86 platforms (ARM and PPC). It lets us run cross-architecture chroots without a full-blown VM.
I have a NUC sitting in a closet that I've set up to serve virtual desktops using LXC or libvirt (depending on what features I need, or if it's a Linux or MacOS or Windows guest). I made some scripts to build the various types of virtual environments I need [1], and then connect to them using x2go or Chrome Remote Desktop.
This has really freed me up because I'm no longer tied to a single machine. My home dirs are mounted volumes that are also backed up, so I can tear down and rebuild the environment with a single command (which means that my environments are also deterministic). I can connect to them from anywhere on the planet, using any machine as a client (even my phone or a tablet). My main laptop is now a weaksauce 1080p chromebook that acts as a glorified thin client and web browser.
We've a shared beefy Threadripper workstation in my lab that has a boatload of RAM and cores to spare. We set up Qemu + KVM in order to run a big Windows VM that we use mainly for MS Office via RDP connections and when we really need Visual Studio. We also have a FreeBSD and a Aarch64 Linux instance that we use for testing. It's amazing how flexible Qemu is, and how reliable the whole stack can be.
I used QEMU for VFIO, GPU passthrough to a Win 10 guest for games. It worked really well. I passed an nvidia card, an ssd with the os/programs, and an hdd data disk from a Linux host (Gentoo).
Setting it up was somewhat difficult. When I did it there weren't many good guides for just exactly my situation, so I had to use many, many different resources to get it running. Based on some of the links others have posted here, there seems to be more out there on it now.
This was my last working startup script. I no longer recall why or even if any of these options were necessary or beneficial. At the time, it was just what worked. (Dual monitor, separate video cards for host and guest, L-CTRL+R-CTRL to pass mouse/keyboard control back and forth.) I haven't used this in quite a while, so I can't say if all of these options are still legit anymore.
We use it with a wrapper[1] for continuous testing of an operating system[2]. The wrapper will perform the installation process as users do, then run the testsuite and upload the results. It's nice because it lets us catch regressions in the installation process, too.
Running linux as Host and optionally starting my gaming vm.
With KVM and various hardware passthroughs (pci network card, usb devices, gpu, etc.) it is as fast a native installation and sometimes even faster compared to my old bloated windows host.
Ofcourse it was not easy to setup and some games needed tweaking (e.g. Just Cause 2 wanted to use unsupported cpu features and therefore needed an extra flag). But if one has the time and patience this is awesome.
I currently have a messy side project[1] where I'm writing bare-metal RISC-V machine code (i.e. writing in hex after assembling instructions by hand) targeting the SiFive HiFive1. QEMU is handy for testing since it supports that board pretty well - it lets me skip the "code upload" step and just run a binary with gdb right away.
I did a similar thing with the Raspberry Pi a few years ago, but the QEMU support wasn't great back then which made debugging incredibly hard. I think the support has improved a lot since then, so I should probably give it another go.
Mainly playing with old operating systems, I'm a retrocomputing enthusiast.
While years ago I would mainly use Virtual PC (I do miss its great GUI) and VMWare (and rarely bochs), qemu improved a lot and has (somewhat decent) support for so many more architectures. You can run Solaris for SPARC and Mac Os 9, for example.
Well, usually the internet is large and you can find lots of guides :)
I'm an avid follower of virtuallyfun.com and right now the latest post is about installing AIX on QEMU. If you look into the qemu category ( https://virtuallyfun.com/wordpress/category/qemu/ ) you'll find OS X Server, Unixware and more.
The author, neozeed, is very friendly and you can leave him comments or shoot him an email and he usually answers pretty in depth.
As for OS 9, I'm not sure it is as refined and functional as SheepShaver on Windows yet, but I'm on OS X, where SheepShaver takes more effort to set up. There's a modified QEMU for enhanced OS 9 compatibility (I think more G3 emulation?) here, and people managed to install 9.2: https://www.emaculation.com/forum/viewtopic.php?f=34&t=9028
QEMU has just the right mix of performance and hackability that I need for my "printf debugging" workflow :) (I've used Bochs a lot, too, which is more hackable, but far less performant.)
I use it to run aarch64 unit test builds on my x86 system. Pretty cool with binfmt support, allowing me to run aarch64 executables just like any other x86 executable: https://wiki.debian.org/QemuUserEmulation
My employer provides me with an iPad Pro and rather than carry around an additional laptop I run x86 systems (Linux and Windows 10) using QEMU-KVM on my ridiculously overpowered home server and remote desktop to them from the iPad Pro.
For software I use Jump Desktop. On Linux I connect via VNC and Windows I use their custom Fluid protocol. For keyboard and mouse I use an Anker Bluetooth keyboard and mouse is a Swiftpoint GT.
Latency varies a lot and I’ve found hotel WiFi to sometimes be unusable. ATT LTE on the device generally works pretty well. I’m primarily running this setup just to have access to a full browser environment for my tinkering with web scraping so having a super responsive low latency connection isn’t as vital as it would be to do something like gaming. The main drawback is always needing a solid connection which is mostly solved with LTE where I go.
We use it here exclusively at my current place of employment in the OS and BSP team. We create pre-built, bootable ramdisk images inside a QEMU instance which are hashed and saved for fast provisioning, as well as having multiple QEMU instances on targets to prevent cross domain pollution (being mandated by NSA). I work in defense, for context.
Currently using it as part of a testing workflow in making my own linux distribution. It makes booting kernels with initramfs, as well as live images, super easy and painless. QEMU is great software! https://jrl.ninja/joy/1/
Trying to run Linux and Windows guest on macOS host with hvf acceleration. Found that FreeBSD won’t boot perhaps due to lack of support of v86 mode IVT in hvf part.
Also trying to setup gvt-g with dmabuf but corner cases still need fixes, e.g. video playback doesn’t work on overlay sprite of yuv encoding.
I use it as a VirtualBox replacement via libvert - no thanks to Oracle setting it (VB) up as an IP licensing minefield. Some VirtualBox extensions are non-free in commercial environments, and installing them creates liability for your organization.
I have a library that needs to target, among other platforms, big endian MIPS. It does a lot of endianness sensitive stuff but all the CI workers are naturally little endian, so I run the testsuite through qemu-user simulating a MIPS system.
Recently I started playing with classic Forth systems like figforth and pygmy.
If you are using MS-DOS on QEMU there is a open source utility called DOSIDLE.EXE that keep the CPU usage low and stop the annoying CPU fan from spinning.
i use qemu/kvm almost everyday. i also use qemu-img tools quite often. and as other people have mentioned, building inside qemu is quite convenient, especially with tools like packer.
I have not upgraded to the latest VMWare Workstation -- should I take a look at QEMU first or is this something meant for a completely different use case? (I use VMWare Workstation for contracting/consulting and build out a workstation to match the client engagement rather than pollute my base OS with five different versions of Visual Studio; IntilliJ, Eclipse, AND Android Studio; multiple version of SQL Server/PostgreSQL/Mongo/MySQL... and I can dump those VMs off to network storage when I'm done with the engagement and reclaim the storage space on my laptop.)
I always wanted (but never succeeded) in emulating SPARC guests using QEMU. Is there a comprehensive list of systems and hardware emulated? I suspect a lot of people in the HN crowd would be curious about emulating Alphas, even if for only running Genera.
Having a collection of emulated Unix workstations is an upgrade in terms of space saved.
I got Debian for SPARC64 running under QEMU a few months ago. I wanted a big endian system for testing. I'll post the link on steps if I can find it. It was on GitHub IIRC.
I clicked on the story because of the major version number bump, then belatedly realised they had moved to time based version numbering. The major version number gets incremented every year.
So the version number is essentially meaningless now - the year of release (eg, QEMU v2019.0.0) would give you more information. I can't say I'm a fan.
I use Virtualbox on home PC to dev. I ssh into it and use tmux/vim as my IDE. Is there a better setup for performance on windows? Should I bite the bullet and get myself a Windows Pro upgrade for Hyper-V? If my memory serves well you Hyper-V can't have Hyper-V and Virtualbox at the same time.
True. I use vmware and my ubuntu vm is faster than my windows host. This makes hyper-v and docker on windows incompatible, though. I run i3 inside vmware. I would expect virtualbox to be ok too, for this use case.
It all depends on how and if someone will implement a Windows driver for VirGL. Given how complex video drivers are nowadays, it looks like a pretty daunting task.
Intel has support for KVMGT, but AMD/NVIDIA won't support it. This would really be the correct approach to get support from the graphics manufacturers:
You don't really need VMs for gaming, at least if you are ready to ditch a few broken games that don't work in Wine. For the most part they work very well these days.
The official documentation just explains the numerous parameters. It also points to tutorials (wikibook, etc), which unfortunately are very basic and out of sync. Even Arch Linux does not provide a good documentation (lack of clarity, some obsolete syntax, "Networking" section is a mess...). Like often, Debian's wiki is obsolete, and even their QEMU images are 5 years old. I had to combine the following 3 sources and a lot of try and guesses to build my VMs.
QEMU Gentoo : https://wiki.gentoo.org/wiki/QEMU/Options
QEMU official : https://qemu.weilnetz.de/doc/qemu-doc.html
KVM Performance : http://www.linux-kvm.org/page/Tuning_KVM
E.g. here is a minimal QEMU command-line that gives you access to a serial console in the guest:
`qemu-system-x86_64 -display none -no-user-config -nodefaults -m 2048 -device virtio-scsi-pci,id=scsi -device virtio-serial-pci -serial stdio -drive file=/export/cirros.qcow2,format=qcow2,if=virtio`
Simple, yeah? Now here (it's too long to post in this comment) is a real world QEMU command-line (as launched by the libvirt[1]):
htttps://kashyapc.fedorapeople.org/Fedora-28-QEMU-command-line-by-libvirt.txt
Compare and contrast the number of devices that libvirt adds.
FWIW, I strongly recommend to use libvirt to launch your QEMU-based guests. It is far more effective, and more importantly, provides security infrastructure like running the QEMU process as an unprivileged user, protecting the QEMU process (and its associated disk iamges) via the SELinux-based sVirt mechanism, and so forth.
Check out these[2] slides (recording is online, too) on "Security in QEMU" from one of the QEMU maintainers, Stefan Hajnoczi, at last year's KVM Forum in Edinburgh.
[1] http://libvirt.org/
[2] https://vmsplice.net/~stefan/stefanha-kvm-forum-2018.pdf
Yes, there may be many switches in the invocation, but at least they are in one place, the invocation syntax rarely changes and works accross distributions, and they are applied from scratch in each invocation rather than having lots of implicit state hidden in daemons and their configurations / databases.
Your example is nicer on the eyes written in a config file style and leaving out the unnecessary options:
I appreciate that libvirt may not be a suitable option for some. That said, you might want to consider the below points on where it adds value (for production VMs; not talking about test and development setups):
• It's not about having all the QEMU "switches available in one place". The dizzying array of switches make it trivial to let you shoot yourself in the shoulder (e.g. getting fiddly details like PCI device addressing right manually).
• Knowing the "best practice" command-line will get you optimal performance out of your (production) guest—libvirt handles that for you. And if the best practices change, libvirt will automatically handle that for you, as it keeps track of "QEMU command-line best practice evolution".
• Launching the command-line is only the first step. If it's a production guest, at some point you might need to live-migrate the guest, or take a live disk backup, or if your guest has a long disk image backing chain, you want to "merge" the disk images into a single, coalesced image—all of this without the guest going offline. These tasks can be done manually (assuming you launched QEMU the 'right away' upfront) via extremely tedious and error-prone ways using QEMU's JSON-based run-time interface, QMP (QEMU Machine Protocol). But that's too risky and a colossal waste of time.
• Speaking of QMP, the run-time interface, it has even more dizzying array of commands—to be more precise, as of 2015: QMP had about 126 commands + 33 events. And more than 700 named arguments and results. Try keeping track of all of that manually. It reminds me of how once a QEMU sub-maintainer memorably compared (in his 2015 talk, "QEMU interface introspection: From hacks to solutions") the volume of QMP schema to old religious books: "QMP schema is larger than "Gospel of Luke", but smaller than "Genesis". :-)
IOW, I'm yet to see anyone who launches multiple guests (on the moderate scale of hundreds of VMs) use direct QEMU in production. There are odd examples, as I noted earlier, but it's certainly not the norm.
But NixOS has a neat feature where any (?) NixOS configuration can trivially instead be built as a "vm", which will assemble an initrd for the described system along with a script which will run a tailored qemu invocation to launch the system, leaving a qcow2 file of any changes you make to the filesystem in your cwd. Makes the whole thing extremely usable, and is done in a very extensible way.
> -nodefaults
Once you add `-nodefaults`, the list of arguments grows because now you must add a bunch of things that were there by default. If by "minimal command-line" you mean "the simplest list of arguments", then your example is wrong. If by "minimal command-line" you mean "command-line that gives the minimal VM", then you're being misleading, because that's not how people will take it.
Most of the devices added by libvirt would be included by qemu automatically if you didn't say `-nodefaults`. libvirt uses `-nodefaults` and specifies its own defaults, so that it doesn't need to keep track of changes in Qemu's defaults between versions of Qemu (since libvirt cares about the nitty-gritty more than most humans do).
PS: You have too many "t"s in the "htttps://" in one of those URLs.
Partly true (refer below).
> If by "minimal command-line" you mean [...]
I appreciate your corrections, but please don't read too much into the word "minimal". I just quickly pasted it from one of the "subjective scripts" named `min-qemu.sh` lying around on my file system. By the time I realized, it was too late to adjust it.
> Most of the devices added by libvirt would be included by qemu automatically if you didn't say `-nodefaults`
Not quite; I just double-checked with an upstream libvirt dev: when you remove `-nodefaults`, some of the built-in devices get removed, whether those equate to "most" or not depends how many devices you've asked for.
> PS: You have too many "t"s in the "htttps://" in one of those URLs.
Yeah, realized too late that there was a typo :-(
Anyone can run your example "qemu-system-x86_64 -display none -no-user-config -nodefaults -m 2048 -device virtio-scsi-pci,id=scsi -device virtio-serial-pci -serial stdio -drive file=/export/cirros.qcow2,format=qcow2,if=virtio" from the command line/bash, as non-root user, yet libvirt which requires root to install, and is expecting root to configure it, is better? How?
Libvirt is one of those softwares which ads a layer of complexity for the sake of complexity, yet offers nothing of value in return.
Sure its possible to do all this without libvirt, but it comes as one integrated, easy to manage package instead of hundreds of disparate parts.
You might not need (all) these features, but to claim libvirt is useless is extremely parochial.
Its also untrue that it needs root, you can virt-manager for instance without a root libvirtd instance at all, and it will run VMs without any permissions needed.
everything libvirt does can be done without it by talking to qemu directly. and yes, Ive done many of the above "by hand", it was easier to read man qemu and other qemu docs than to swallow libvirt.
Its fine if your use cases are covered by plain QEMU, but libvirt does a hell of a lot more than plain QEMU alone is capable of.
Again all of that is possible via other means, but when other means involve recreating half of libvirt in shell scripts or other code, id rather let somebody else do the work for me.
QEMU is designed to run virtual machines, libvirtd is designed to manage VM server infrastructure.
Libvirtd is much better suited to managing that sort of complexity.
People forget that software is malleable. Bend it to your needs, instead of adding fragile wrapper layers.
The libvirt team definitely does view qemu as malleable, and bends it to their needs.
The problem is that the libvirt authors contributing to qemu look at qemu as a machine-interface (to be consumed by libvirt), not as a human-interface. Many of the more poorly documented or otherwise difficult-for-humans flags were contributed by the libvirt team. And they're useful! They make it so qemu can do things it couldn't do before. But they're also kinda the reason qemu is difficult to use without libvirt.
Isn't that sort of like saying the system startup scripts/subsystem adds complexity for the sake of complexity, and you should just use ifconfig and route manually in some script after startup?
In both cases, that you can fall back to the underlying commands is great. That you should use them instead of the normalized interface provided is debatable.
EDIT: Nevermind, `-enable-kvm` also doesn't need root. I'm not sure what libvirt uses root for.
Instead, one of the more annoying things I have ever done was foolishly installing libvirt and having it do all kinds of whackiness to a previously excellent QEMU set-up. I still cannot figure out where it managed to change all QEMU invocations to be VNC-based.
> `qemu-system-x86_64 -display none -no-user-config -nodefaults -m 2048 -device virtio-scsi-pci,id=scsi -device virtio-serial-pci -serial stdio -drive file=/export/cirros.qcow2,format=qcow2,if=virtio`
That's far more than I typically put in my qemu calls. You can technically be fine with just:
but I always add `-enable-kvm` and use `-monitor stdio` or `-nographic` depending if I want a graphic display from the vm or not. `-nographic` is all I need to give me access to a serial console in the guest (besides configuring the guest kernel with the paramater `console=ttyS0`). I also add `-m` because the default amount of memory is just 128 MiB, which is typically not enough for my use. Sometimes I add network related arguments, but we're going beyond minimal then.Both gnome-boxes and virt-manager use QEMU under the hood though, and they provide a point-and-click experience. I use virt-manager myself and I do not think it's any more difficult than VirtualBox is.
Everytime I tried to migrate to qemu or virt-manager or gnome-boxes it was a real nightmare especially installing SPICE+virtio drivers+ networking. I read in Phoronix the performance of virtualbox is subpar and all KVM/qemu is the way to go.
Please, could the OSS community build a easy to use GUI that SPICE+virtio drivers+ networking is made simple. Any one from qemu, I really appreciate the work but still very challenging for sysadmins.
The exercise was a good (and incremental) learning experience and increased my general networking and sys-admin knowledge, which I've used well as part of my software engineering career. I would have been lost in many professional situations without it. QEMU/KVM are awesome.
EDIT: more and better words
The problems I did have were with VMs I tried to convert from VMware to KVM, and those stemmed from Windows' inability (version 7) to properly handle changing hardware. Perhaps it's better at nowadays.
I would say forget spice + libvirt graphics layer. Use RDP, if on linux remmina work's great. RDP will outperform in my experience SPICE. I've also had constant troubles with the vga/qxl/virtio graphics.
Secondily for virtio: https://docs.fedoraproject.org/en-US/quick-docs/creating-win...
There is an iso at the above link. Grab that and have it added during install. When you get to the select a drive. None will show up until you load the virtiostorage driver.
There are a lot of nuances to first setting up a windows vm. But the performance is great. It has removed the need for a windows install for gaming at all.
https://www.spice-space.org/download.html
It will contain everything you need for a fairly seamless experience on windows, and libvirtd and virt-manager make host side a cakewalk as well.
I suggest QEMU's documentation could mention libvirt and its user-friendly interfaces.
"QEMU aims to fit into a variety of use cases. It can be invoked directly by users wishing to have full control over its behaviour and settings. It also aims to facilitate integration into higher level management layers, by providing a stable command line interface and monitor API. It is commonly invoked indirectly via the libvirt library when using open source applications such as oVirt, OpenStack and virt-manager."
But I get it, not everyone goes in search of a README buried in a Git repository. I myself am not sure if this REAMDE is rendered in an HTML format somewhere.
[1] https://git.qemu.org/?p=qemu.git;a=blob_plain;f=README
actually, having gone through the process of successfully setting up the vm’s make you eminently suitable to contribute documentation knowledge that is sorely lacking in the online resources you have pointed out.
i am sure it will be greatly appreciated and will get improved upon areas that are missing etc.
> [1] Especially VirtualBox. If you use VirtualBox after reading this article you make poor life choices and are an embarrassment to us all.
Well I'm convinced! What a well-presented article!
Appropriate Futurama quote: "Remember when I told you about always ending your stories a sentence earlier?".
Drew was doing well until he ended with this unnecessary condescending and hostile conclusion (and if you think it's funny, note that "the failure mode of smartass is asshole"). It's a trend of his. He's got interesting and informed things to say, but it's hard to really take him seriously because of this.
https://wiki.gnome.org/action/show/Apps/Boxes
I believe his pain point is that with Virtual Box you get a very simple GUI where you can get VM up and running very quickly without having to use Google or read any guides at all.
[1] https://virt-manager.org/
Virt-manager adds a very nice VirtualBox-style GUI front-end to libvirt.
In the end we rolled our own solution that is much more lightweight, uses QEMU directly and provides better visibility, ease of use and even some potentially useful networking features. Here are a few screencasts [1].
This is mainly designed as a module for Flockport and we will preview this soon.
[1] https://www.youtube.com/watch?v=uEnk9m-egeA
I ran Foreman inside a CentOS VM, which talked to my desktop's Libvirtd to provision new VMs, hand out IPs via DHCP, boot via TFTP, then kickstart. It worked pretty well, and helped me to spin up a vcenter foreman controlled dev environment for a few hundred users in just a few days.
My recommendation would be Vagrant + Libvirt.
That's a fact.
And not many people need to look at the code in real life...
In Linux distributions, you basically install `virt-manager` and the package manager will resolve all dependencies to provide you a fully working virtualization desktop software which is well capable to be used on servers.
https://github.com/jeffreywildman/homebrew-virt-manager
https://news.ycombinator.com/item?id=19736609
https://news.ycombinator.com/item?id=19736722
[1]: https://www.youtube.com/watch?v=ATnpEOo3GJA
To get macOS to see my GPUs with full acceleration and audio over HDMI, I use the Clover bootloader to inject a small section into the DSDT, mapping the emulated PCI address to a device that actually looks like a graphics card to macOS. This same DSDT patch works for both NVIDIA and AMD GPUs (the ones I've tried anyway).
The best thing about Hackintosh in a VM is that you no longer need to fear macOS upgrades. Just create a snapshot, try the upgrade, and if it doesn't work out, you can restore, investigate, and try again.
LTT's video is a nice overview, but his claim that "because this is a VM... it will run on any hardware" isn't quite true when talking about GPUs. Unlike the CPU, memory, and some other devices, the GPU is seen (nearly) directly by the guest VM, so guest OS support is in fact required to see the advantages of GPU passthrough.
In summary, it's never going to be super easy, but it's not too difficult either, and the performance (and flexibility) is actually really great.
I was thinking about expanding the setup to include a beefier linux host (more gpus and better hardware) and more seats -- windows gaming for me and osx for my wife. But you mentioned several possible osx specific caveats: amd gpus, clover (?) bootloader w/ dsdt (?), disable_idle_d3, etc. Can you link to some info or READMEs regarding your setup or how you came across these solutions?
- Good starting point for setting up macOS under QEMU/KVM: https://github.com/kholia/OSX-KVM
- DSDT/SSDT patching: https://www.tonymacx86.com/threads/ssdt-gpu-graphics-card-in...
- AMD reset bug: https://www.reddit.com/r/VFIO/comments/5h351m/gpu_stuck_in_d...
- Kernel extension that can help with some GPU issues: https://github.com/acidanthera/WhateverGreen
The DSDT/SSDT patch was probably the trickiest thing. I've posted my particular patch here if you're interested: https://pastebin.com/ngvkVZYN
Also, a few of my other scripts/config: https://pastebin.com/9Nh5rheZ
Previously, I also attempted using a kernel patch that's been floating around (DECLARE_PCI_FIXUP_HEADER ... quirk_no_bus_reset), but it just gives me corrupt graphics when rebooting the VM.
I had success getting macOS running on QEMU with my existing PC, but its performance suffered without a dedicated GPU and CPU pinning, which my new build will be configured to handle
[1] https://wiki.debian.org/QemuUserEmulation
Edit: Tested it, aaaaaand NaN compares are still broken :(
When you launch a KVM-based guest, most often you would see QEMU doing the major user space heavy-lifting by providing a PC-like environment—it handles guest memory management, network cards, emulated storage devices, and so forth. QEMU also has a robust Block Layer; not to mention its versatile, and widely-used, native disk image format, QCOW2.
PS: I am biased, as I work with QEMU on a daily basis :-)
Also I recently discovered some very in-depth blog posts here: https://heiko-sieger.info/iommu-groups-what-you-need-to-cons...
Another whole website about this topic (no joke): https://passthroughpo.st/
The most complete guide is probably the Arch Linux one: https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVM....
You can write a system configuration, generate a QEMU VM and run it. Also all NixOS automatic tests[1] are running in QEMU VMs that are generated from a Nix configuration.
The most beautiful thing is that you can create multiple VMs and run them simultaneously to create a virtual network. For example you can test a client-server application by running the client and the server on two hosts or test some network configuration[2].
[1]: https://nixos.org/nixos/manual/index.html#sec-nixos-tests
[2]: https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nat...
There are a lot of rough edges to the desktop experience, but it generally works. And while that's faint praise, I have no current plans to switch back to Fedora (3 years) or Debian (12 years prior to that).
It's comforting to know that my entire base system configuration is declarative, and it's pleasant to not worry about libraries conflicting or unnecessary packages lingering around.
Immediate, ephemeral access to programs via, e.g., `nix-shell -p pdftk` is extremely convenient: they're only available in that shell session, and they (along with their dependencies) get cleaned up the next time you run `nix-collect-garbage`. Super handy.
https://godarch.com/
edit: nevermind, I didn't realize it was a Linux distro.
I got sick of trying to figure out which thing I needed to click in virt-manager to get it to pass the flags I wanted, so I wrote my own minimal qemu-wrapper that covers all of the functionality from libvirt that I care about (vCPU pinning, binding to NUMA nodes, running as non-privileged user), and integrates better with the other system (systemd) monitoring than libvirt does. https://git.lukeshu.com/systemd-qemu/
----
I'm also involved with the Parabola GNU/Linux-libre distribution. Qemu user-mode emulation is essential to our support of non-x86 platforms (ARM and PPC). It lets us run cross-architecture chroots without a full-blown VM.
This has really freed me up because I'm no longer tied to a single machine. My home dirs are mounted volumes that are also backed up, so I can tear down and rebuild the environment with a single command (which means that my environments are also deterministic). I can connect to them from anywhere on the planet, using any machine as a client (even my phone or a tablet). My main laptop is now a weaksauce 1080p chromebook that acts as a glorified thin client and web browser.
[1] https://github.com/kstenerud/virtual-builders
Setting it up was somewhat difficult. When I did it there weren't many good guides for just exactly my situation, so I had to use many, many different resources to get it running. Based on some of the links others have posted here, there seems to be more out there on it now.
This was my last working startup script. I no longer recall why or even if any of these options were necessary or beneficial. At the time, it was just what worked. (Dual monitor, separate video cards for host and guest, L-CTRL+R-CTRL to pass mouse/keyboard control back and forth.) I haven't used this in quite a while, so I can't say if all of these options are still legit anymore.
[1] - https://www.gson.org/netbsd/anita/
[2] - http://releng.netbsd.org/test-results.html
I did a similar thing with the Raspberry Pi a few years ago, but the QEMU support wasn't great back then which made debugging incredibly hard. I think the support has improved a lot since then, so I should probably give it another go.
[1] https://github.com/SgtCoDFish/bedrock-bootstrap - I'm writing the steps as a kind of tutorial to myself to aid learning, but fair warning that it's all a bit of a mess right now!
While years ago I would mainly use Virtual PC (I do miss its great GUI) and VMWare (and rarely bochs), qemu improved a lot and has (somewhat decent) support for so many more architectures. You can run Solaris for SPARC and Mac Os 9, for example.
I'm an avid follower of virtuallyfun.com and right now the latest post is about installing AIX on QEMU. If you look into the qemu category ( https://virtuallyfun.com/wordpress/category/qemu/ ) you'll find OS X Server, Unixware and more.
The author, neozeed, is very friendly and you can leave him comments or shoot him an email and he usually answers pretty in depth.
As for OS 9, I'm not sure it is as refined and functional as SheepShaver on Windows yet, but I'm on OS X, where SheepShaver takes more effort to set up. There's a modified QEMU for enhanced OS 9 compatibility (I think more G3 emulation?) here, and people managed to install 9.2: https://www.emaculation.com/forum/viewtopic.php?f=34&t=9028
Link?
Or do you mean a copy of the ISO? For that I'd try archive.org or winworldpc
(See https://taoofmac.com/space/blog/2019/04/21/2300 and https://github.com/insightfulsystems/alpine-python)
Will be trying ARM64 soon, as soon as I have a suitable development board.
QEMU has just the right mix of performance and hackability that I need for my "printf debugging" workflow :) (I've used Bochs a lot, too, which is more hackable, but far less performant.)
But support appears to be there as of a week ago, fyi.
What remote desktop app do you use?
What’s your latency like? What are the drawbacks to a setup like this?
Latency varies a lot and I’ve found hotel WiFi to sometimes be unusable. ATT LTE on the device generally works pretty well. I’m primarily running this setup just to have access to a full browser environment for my tinkering with web scraping so having a super responsive low latency connection isn’t as vital as it would be to do something like gaming. The main drawback is always needing a solid connection which is mostly solved with LTE where I go.
Also trying to setup gvt-g with dmabuf but corner cases still need fixes, e.g. video playback doesn’t work on overlay sprite of yuv encoding.
If you are using MS-DOS on QEMU there is a open source utility called DOSIDLE.EXE that keep the CPU usage low and stop the annoying CPU fan from spinning.
Having a collection of emulated Unix workstations is an upgrade in terms of space saved.
https://astr0baby.wordpress.com/2018/09/22/running-solaris-2...
I used a Solaris 7 .iso from archive.org.
https://archive.org/details/solaris_7_1199_sparc
http://www.w6rz.net/netscape.png
That was with a Solaris 8 disk image that I found here:
https://github.com/cclark64/qemu_solaris_sparc
The steps are here for posterity: https://github.com/kholia/mips-hacking
So the version number is essentially meaningless now - the year of release (eg, QEMU v2019.0.0) would give you more information. I can't say I'm a fan.
https://wiki.archlinux.org/index.php/Intel_GVT-g