Opening files under qemu windows guest from the Linux host

I migrated my VirtualBox Windows 7 client to qemu which uses kernel-level virtualisation. The performance improvement is mind blowing. The migration was really easy but I really missed my shell scripts for file integration tasks that made my life so much easier for years.

I had a well working bash script on my Linux machine that allowed my linux applications (like file manager or e-mail client, etc.) to open windows files (like doc, excel, etc.) in the Windows guest machine. VirtualBox has a "vboxmanage" tool to accomplis that. qemu does not have anything similar though. My script has helped me a lot during my everyday life and I simply don't want to give up this seamless integration between my host and guest. I work with documents and excel sheets in Windows but my everyday OS is Linux. In my Arch Linux OS I simply select a .doc or even an .exe file and it will be opened under the guest Windows. Similarly when I get a .ppt slideshow in e-mail as an attachment I simply click on it in my linux mail client and the powerpoint slideshow pops up in my Windows machine.

Migrating from VirtualBox to QEMU:

I sum up the migration process in bulletpoints but the whole process is well documented in the QEMU Arch Wiki page. I am not a qemu or virtualisation expert by any means.

So the process looks like this:

install qemu stuff (see arch wiki)
# pacman -S libvirt qemu spice-guest-tools-windows virtio-win spice spice-gtk
$ qemu-img convert -f vdi -O raw <your_Windows_vdi_file> <destination_image_file>

It looked like this in my case:
$ qemu-img convert -f vdi -O raw VirtualBox\ VMs/Windows/Windows\ Clone.vdi virt/win.img
 

This will take a while...

Try it:
qemu-system-x86_64 -m 3G -enable-kvm virt/win.img
Now your qemu-driven Windows guest should start in a window using the newly created img file.

Insert the virtio driver CD to the guest:
$ qemu-img create -f qcow2 fake.qcow2 1G
$ qemu-system-x86_64 -m 3G -enable-kvm -drive file=virt/win.img,if=ide -drive file=fake.qcow2,if=virtio -cdrom /usr/share/virtio/virtio-win.iso
Now the guest should start the same way as before but an inserted CD rom should show up in the machine.

Install the virtio guest drivers in the Windows guest that you just started. Windows did it automatically for me, probably you have to do some next-next-finish on the CD rom under your guest Windows.

Now insert the Spice Guest Tools CD to the guest similarly:
$ qemu-system-x86_64 -m 3G -drive file=virt/win.img,if=ide,format=raw -drive file=fake.qcow2,if=virtio -cdrom /usr/share/spice-guest-tools/spice-guest-tools.iso

In the Windows guest run the .exe on the cdrom to install the Spice Guest Tools.

Start the thing without gui and connect to it with a Spice client (some cool features enabled):
$qemu-system-x86_64 -nographic -m 3G -drive file=virt/win.img,if=virtio,format=raw -spice port=5900,addr=127.0.0.1,disable-ticketing -device virtio-serial -chardev spicevmc,id=vdagent,debug=0,name=vdagent -device virtserialport,chardev=vdagent,name=com.redhat.spice.0 -vga qxl -machine type=pc,accel=kvm -usb -device usb-tablet -net nic -net user,smb=/

In another terminal connect to the spice server:
$spicy -h 127.0.0.1 -p 5900

Now the guest machine should appear in a spice client window. The screen resolution should appply when you resize this window and the clipboard should be shared between the host and the guest. You also have file access to your Linux host: the host's / folder should be accessible in the guest under \\10.0.2.4\qemu\

If all is ok then install the headless version of qemu instead of the normal one since we use the spice guest to connect to the server:
#pacman -S qemu-headless

Setting up file acceess:
The process is based on two components: 1. on the Linux host we write the path of a file (in windows format) that we want to open under windows TO A "watch" file. In the Windows guest we read the contents of this file every 3 seconds and if we find a filename in it then we open that file and delete the "watch" file. This is not very nice but I am not a programmer and I wanted to use native commands instead of external apps.

1. On the linux host:
This script will acept a filename (i.e. a doc file or similar) as argument. It removes special characters from the filename (using another script but you can use detox or anything similar) then converts the file pah to windows format and writes this path to the "watch" file. (The "watch" file will be processed on the other side in the guest.)

My script is called winopen.sh and looks like this:


#!/bin/bash

# needs 'clean' script

# stdout and errors go here:
exec 2>>"/home/<username>/.xlog"
exec 1>&2

script="$(basename "$0")"
driveletter='\\10.0.2.4\qemu'
watchfile="/home/<username>/virt/watch.txt"

# Start vm if not running already

if (( $(ps -ef | grep qemu-system-x86_64 | wc -l) == 1 ))
then ( qemu-system-x86_64 -m 3G -drive file=/home/<username> /virt/win.img,if=virtio,format=raw -spice port=5900,addr=127.0.0.1,disable-ticketing -device virtio-serial -chardev spicevmc,id=vdagent,debug=0,name=vdagent -device virtserialport,chardev=vdagent,name=com.redhat.spice.0 -vga qxl -machine type=pc,accel=kvm -usb -device usb-tablet -net nic -net user,smb=/ &
    ( sleep 4 ; spicy -f -h 127.0.0.1 -p 5900 ) &
    )
fi

# Get the absolute path of the file on the linux host
for i in "$@"
do
    if echo $i | grep '^/' > /dev/null
    then targetfullpath=$i
    else targetfullpath=$PWD'/'$i
    fi
   
    #Cleanup filename
    if ! cleanpath=$(clean -F "$targetfullpath")
    then
        echo "File cleaning failed"
        exit 1
    fi

    # The same filepath under the windows guest machine
    winfullpath=$(echo "$driveletter""$cleanpath" | sed 's/\//\\/g')

    # Place the filename to the watchfile
    echo "$script"": opening ""$targetfullpath"" (""$winfullpath"") in qemu"
    echo "$winfullpath" >> $watchfile
done



The path cleaning script called 'clean':

cat bin/clean
#!/bin/bash

accent='ÀÁÂÃÄÅÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖŐØÙÚÛÜŰÝàááâãäåçèééêëìíîïðñòóôõőöőøùúûüűý'
nonaccent='AAAAAACEEEEIIIIDNOOOOOOOUUUUUYaaaaaaaceeeeeiiiidnoooooooouuuuuy'
allowed='.ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789'
substitute='_'
fullpathchars='~/'

# Parse command line arguments
shortopts="a:fFhs:"
longopts="allow:,fullpath,filerename,help,substitute:"
options=$(getopt -n $(basename $0) -u -o $shortopts -l $longopts -- "$@") || exit 1
set -- $options
while [ $# -gt 0 ]
do
    case $1 in
    # for options with required arguments, an additional shift is needed
    -a|--allow) allowed+="$2" ; shift ;;
    -f|--fullpath) allowed+="$fullpathchars" ;;
    -F|--filerename) allowed+="$fullpathchars" ; filerename="true" ;;
    -h|--help) help=y ;;
    -s|--substitute) substitute="$2" ; shift ;;
    (--) ;;
    (*) break ;;
    esac
    shift
done

# --help #
if [[ $help == "y" ]]
then
    cat <<EOF
clean: a tool for cleaning a string from special characters. Transforms the accented characters of STRING to their non-accented form and substitutes special characters with a substitute string, "_" by default. Allowed characters are lower- and uppercase letters, numbers and '.' by default, other characters are substituted. More than one non-allowed subsequent characters are substituted with only one substitute string.

Usage: clean [OPTION] STRING
    -a "ALLOW_STR", --allow="ALLOW_STR"  Accepts characters of ALLOWED_STR besides the default allowed characters.
    -f, --fullpath                       Accepts the tilde and slash ("~" and "/") character besides the allowed characters (useful for transforming full pathnames)
    -F --filerename                      Accepts a file name as STRING. It will rename the file to the cleaned string. If the file with the cleaned name already exists, it creates a backup of that before renaming.
    -h, --help                           Print this help

Default values:
---------------
The following accented characters are transformed to non-accented ones:
$accent

The default allowed (non-substituted) characters are:
$allowed

The default substitute character is: $substitute

EOF
exit
fi

string="$*"
#substitute=$(echo "$substitute" | sed 's/[\/\\\?\&]/\\&/g')

result=$(echo "$string" | sed -e "y/$accent/$nonaccent/" -e "s/[^$allowed]\+/$substitute/g")

if [[ $filerename = "true" ]]
then
    if [ ! -f "$string" ]
    then
        echo "Cannot find file or not a regular file: $string"
        exit 1
    fi

    if echo $string | grep '/' > /dev/null
    then pathname=${string%/*}"/"
    fi
   
    filename=${string##*/}
    resultfilename=$(echo ${string##*/} | sed -e "y/$accent/$nonaccent/" -e "s/[^$allowed]\+/$substitute/g")
   
    if [[ "$pathname$filename" != "$pathname$resultfilename" ]]
    then
        if ! mv --backup=numbered "$pathname$filename" "$pathname$resultfilename"
        then exit 1
        fi
    fi
    echo $result
fi


2. On the Windows guest:
 
A bat file contains an endless loop watching the "watch" file every 3 seconds. If the file exists then it reads out its contents (which are filenmames) open them then deletes the "'watch" file. A vbs file does nothing but invokes the bat script to eliminate the "cmd" window that the bat file would pop up. The vbs is started on Windows startup.

Create two files:

winopen.bat:

set watch=\\10.0.2.4\qemu\home\cuh\virt\watch.txt

:loop
if exist %watch% (
  for /f "delims=" %%i in (%watch%) do (
    icacls %%i /grant:r Everyone:F
    start %%i
  )
  del %watch%
)
timeout /t 3
goto loop


winopen.vbs:

Set oShell = CreateObject ("Wscript.Shell")
Dim strArgs
strArgs = "cmd /c C:\Users\cuh\winopen\winopen.bat"
oShell.Run strArgs, 0, false

Create a shortcut of the vbs file and place it in the startup folder to start the script automatically. The startup folder is:

C:\Users\cuh\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\



Now if you invoke winopen.sh under the host with a filename, the file should open under the guest.

E.g.:
winopen.sh <your_doc_file_under_linux.docx>

Powerful networking with Wicd

Wicd is a very clever network manager. It is independent of any desktop environments meaning that it is compatible with any window managers. It has a daemon running in the background and a client with a ststus icon in the systray and a cool GUI to manipulate your network connections. Wicd handles both wired and wireless connections. You can set it up to automatically connect to the wired network when you plug in your ethernet cable and in other circumstances connect to your preferred wifi access points.

Installation is easy, just follow the Wicd Arch Wiki. I put the wicd daemon in my /etc/rc.conf (after dbus) and disabled the original networking daemon:
DAEMONS=(...dbus ... !network ... wicd ...)
I edited my .xinitrc to run the client when X starts:

...
wicd-client &
...
exec ck-launch-session wmfs

(See my actual rc.conf and .xinitrc files on the right panel of this blog.)

Above the basic networking services Wicd gives you scripting possibilities to execute programs on different networking events and different networking environments. This feature is very useful for laptops that are brought from one network to another (home / office / net café, etc.) and the behavior of the machine can be altered according to the network it is connected to. Just some ideas:
- You can automatically mount nfs and/or samba shares when you connect to your office network.
- You can start or stop daemons according to your network connection (i.e. if you use network printers you can set up your cupsd service to start only at places where you use printers).
- You can switch off your wi-fi antenna after disconnecting from a wireless network to gain battery life when your laptop is running on battery power.

Mainly I use my laptop in my office and at home. In the office I have both wired and wireless networks (at my desk I usually plug in the network cable but when I am at a meeting in another room I use wi-fi). I need the same settings for both cases. At home I use wireless internet, I do not have a printer nor do I use nfs file shares. I want to access samba shares anywhere.

You can find the preconnect, postconnect, predisconnect, postdisconnect directories under /etc/wicd/scripts/. Bash scripts placed in these directories are executed each time the appropriate network event (the nemes of the directories are self-explanatories) occurs. I use two of them but their usage depends only on your creativity.

I created a share_mount.sh script that runs after wicd connects to a network and a share_umount.sh that runs before disconnecting. If the network is wired or the wireless ESSID belongs to my office's wifi router then I start the rpcbind and nfs-common daemons to use nfs, the cupsd daemon for printing and I mount my office nfs shares (defined previously in fstab with noauto option). In all networks I start samba and smbnetfs to automatically mount samba shares after network connection. Logs go to the /etc/wicd/<script_name>.log file. I used wicd's sample scripts to build up my own ones:

Postconnect script ( /etc/wicd/scripts/postconnect/share_mount.sh ):

#!/bin/bash

script="$(basename "$0")"
script_name="${script/.sh/}"

echo "Running ${script}" >"/var/log/wicd/${script_name}.log"
exec 2>>"/var/log/wicd/${script_name}.log"
exec 1>&2

connection_type="$1"
echo "Connection type: ${connection_type}"

if [ "${connection_type}" == "wired" ]; then
profile="$3"
echo "Profile: ${profile}"
elif [ "${connection_type}" == "wireless" ]; then
essid="$2"
bssid="$3"
echo "ESSID: ${essid}"
echo "BSSID: ${bssid}"
else
echo "Unknown connection type: ${connection_type}" >&2
exit
fi

case $2 in "wired" | "myoffice_essid")
# I am at office:
rc.d start rpcbind nfs-common cupsd;
mount -a -t nfs;;
esac
# I am anywhere:
rc.d start samba smbnetfs


Predisconnect script ( /etc/wicd/scripts/predisconnect/share_umount.sh ):

Before disconnecting a network this script handles the file unmounting and daemon stopping process in the opposite way I used in my connect script.

#!/bin/bash

script="$(basename "$0")"
script_name="${script/.sh/}"

echo "Running ${script}" >"/var/log/wicd/${script_name}.log"
exec 2>>"/var/log/wicd/${script_name}.log"
exec 1>&2

echo umounting nfs and smb filesystems
#umount /media/Erba_nfs
umount -a -t nfs
rc.d stop cupsd smbnetfs samba nfs-common rpcbind

Wicd's actual (1.7.0) version is not able to handle ad-hoc networking. Scripts can help you again, see the workaround here.