Opening files under qemu windows guest from the Linux host

I migrated my VirtualBox Windows 7 client to qemu which uses kernel-level virtualisation. The performance improvement is mind blowing. The migration was really easy but I really missed my shell scripts for file integration tasks that made my life so much easier for years.

I had a well working bash script on my Linux machine that allowed my linux applications (like file manager or e-mail client, etc.) to open windows files (like doc, excel, etc.) in the Windows guest machine. VirtualBox has a "vboxmanage" tool to accomplis that. qemu does not have anything similar though. My script has helped me a lot during my everyday life and I simply don't want to give up this seamless integration between my host and guest. I work with documents and excel sheets in Windows but my everyday OS is Linux. In my Arch Linux OS I simply select a .doc or even an .exe file and it will be opened under the guest Windows. Similarly when I get a .ppt slideshow in e-mail as an attachment I simply click on it in my linux mail client and the powerpoint slideshow pops up in my Windows machine.

Migrating from VirtualBox to QEMU:

I sum up the migration process in bulletpoints but the whole process is well documented in the QEMU Arch Wiki page. I am not a qemu or virtualisation expert by any means.

So the process looks like this:

install qemu stuff (see arch wiki)
# pacman -S libvirt qemu spice-guest-tools-windows virtio-win spice spice-gtk
$ qemu-img convert -f vdi -O raw <your_Windows_vdi_file> <destination_image_file>

It looked like this in my case:
$ qemu-img convert -f vdi -O raw VirtualBox\ VMs/Windows/Windows\ Clone.vdi virt/win.img
 

This will take a while...

Try it:
qemu-system-x86_64 -m 3G -enable-kvm virt/win.img
Now your qemu-driven Windows guest should start in a window using the newly created img file.

Insert the virtio driver CD to the guest:
$ qemu-img create -f qcow2 fake.qcow2 1G
$ qemu-system-x86_64 -m 3G -enable-kvm -drive file=virt/win.img,if=ide -drive file=fake.qcow2,if=virtio -cdrom /usr/share/virtio/virtio-win.iso
Now the guest should start the same way as before but an inserted CD rom should show up in the machine.

Install the virtio guest drivers in the Windows guest that you just started. Windows did it automatically for me, probably you have to do some next-next-finish on the CD rom under your guest Windows.

Now insert the Spice Guest Tools CD to the guest similarly:
$ qemu-system-x86_64 -m 3G -drive file=virt/win.img,if=ide,format=raw -drive file=fake.qcow2,if=virtio -cdrom /usr/share/spice-guest-tools/spice-guest-tools.iso

In the Windows guest run the .exe on the cdrom to install the Spice Guest Tools.

Start the thing without gui and connect to it with a Spice client (some cool features enabled):
$qemu-system-x86_64 -nographic -m 3G -drive file=virt/win.img,if=virtio,format=raw -spice port=5900,addr=127.0.0.1,disable-ticketing -device virtio-serial -chardev spicevmc,id=vdagent,debug=0,name=vdagent -device virtserialport,chardev=vdagent,name=com.redhat.spice.0 -vga qxl -machine type=pc,accel=kvm -usb -device usb-tablet -net nic -net user,smb=/

In another terminal connect to the spice server:
$spicy -h 127.0.0.1 -p 5900

Now the guest machine should appear in a spice client window. The screen resolution should appply when you resize this window and the clipboard should be shared between the host and the guest. You also have file access to your Linux host: the host's / folder should be accessible in the guest under \\10.0.2.4\qemu\

If all is ok then install the headless version of qemu instead of the normal one since we use the spice guest to connect to the server:
#pacman -S qemu-headless

Setting up file acceess:
The process is based on two components: 1. on the Linux host we write the path of a file (in windows format) that we want to open under windows TO A "watch" file. In the Windows guest we read the contents of this file every 3 seconds and if we find a filename in it then we open that file and delete the "watch" file. This is not very nice but I am not a programmer and I wanted to use native commands instead of external apps.

1. On the linux host:
This script will acept a filename (i.e. a doc file or similar) as argument. It removes special characters from the filename (using another script but you can use detox or anything similar) then converts the file pah to windows format and writes this path to the "watch" file. (The "watch" file will be processed on the other side in the guest.)

My script is called winopen.sh and looks like this:


#!/bin/bash

# needs 'clean' script

# stdout and errors go here:
exec 2>>"/home/<username>/.xlog"
exec 1>&2

script="$(basename "$0")"
driveletter='\\10.0.2.4\qemu'
watchfile="/home/<username>/virt/watch.txt"

# Start vm if not running already

if (( $(ps -ef | grep qemu-system-x86_64 | wc -l) == 1 ))
then ( qemu-system-x86_64 -m 3G -drive file=/home/<username> /virt/win.img,if=virtio,format=raw -spice port=5900,addr=127.0.0.1,disable-ticketing -device virtio-serial -chardev spicevmc,id=vdagent,debug=0,name=vdagent -device virtserialport,chardev=vdagent,name=com.redhat.spice.0 -vga qxl -machine type=pc,accel=kvm -usb -device usb-tablet -net nic -net user,smb=/ &
    ( sleep 4 ; spicy -f -h 127.0.0.1 -p 5900 ) &
    )
fi

# Get the absolute path of the file on the linux host
for i in "$@"
do
    if echo $i | grep '^/' > /dev/null
    then targetfullpath=$i
    else targetfullpath=$PWD'/'$i
    fi
   
    #Cleanup filename
    if ! cleanpath=$(clean -F "$targetfullpath")
    then
        echo "File cleaning failed"
        exit 1
    fi

    # The same filepath under the windows guest machine
    winfullpath=$(echo "$driveletter""$cleanpath" | sed 's/\//\\/g')

    # Place the filename to the watchfile
    echo "$script"": opening ""$targetfullpath"" (""$winfullpath"") in qemu"
    echo "$winfullpath" >> $watchfile
done



The path cleaning script called 'clean':

cat bin/clean
#!/bin/bash

accent='ÀÁÂÃÄÅÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖŐØÙÚÛÜŰÝàááâãäåçèééêëìíîïðñòóôõőöőøùúûüűý'
nonaccent='AAAAAACEEEEIIIIDNOOOOOOOUUUUUYaaaaaaaceeeeeiiiidnoooooooouuuuuy'
allowed='.ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789'
substitute='_'
fullpathchars='~/'

# Parse command line arguments
shortopts="a:fFhs:"
longopts="allow:,fullpath,filerename,help,substitute:"
options=$(getopt -n $(basename $0) -u -o $shortopts -l $longopts -- "$@") || exit 1
set -- $options
while [ $# -gt 0 ]
do
    case $1 in
    # for options with required arguments, an additional shift is needed
    -a|--allow) allowed+="$2" ; shift ;;
    -f|--fullpath) allowed+="$fullpathchars" ;;
    -F|--filerename) allowed+="$fullpathchars" ; filerename="true" ;;
    -h|--help) help=y ;;
    -s|--substitute) substitute="$2" ; shift ;;
    (--) ;;
    (*) break ;;
    esac
    shift
done

# --help #
if [[ $help == "y" ]]
then
    cat <<EOF
clean: a tool for cleaning a string from special characters. Transforms the accented characters of STRING to their non-accented form and substitutes special characters with a substitute string, "_" by default. Allowed characters are lower- and uppercase letters, numbers and '.' by default, other characters are substituted. More than one non-allowed subsequent characters are substituted with only one substitute string.

Usage: clean [OPTION] STRING
    -a "ALLOW_STR", --allow="ALLOW_STR"  Accepts characters of ALLOWED_STR besides the default allowed characters.
    -f, --fullpath                       Accepts the tilde and slash ("~" and "/") character besides the allowed characters (useful for transforming full pathnames)
    -F --filerename                      Accepts a file name as STRING. It will rename the file to the cleaned string. If the file with the cleaned name already exists, it creates a backup of that before renaming.
    -h, --help                           Print this help

Default values:
---------------
The following accented characters are transformed to non-accented ones:
$accent

The default allowed (non-substituted) characters are:
$allowed

The default substitute character is: $substitute

EOF
exit
fi

string="$*"
#substitute=$(echo "$substitute" | sed 's/[\/\\\?\&]/\\&/g')

result=$(echo "$string" | sed -e "y/$accent/$nonaccent/" -e "s/[^$allowed]\+/$substitute/g")

if [[ $filerename = "true" ]]
then
    if [ ! -f "$string" ]
    then
        echo "Cannot find file or not a regular file: $string"
        exit 1
    fi

    if echo $string | grep '/' > /dev/null
    then pathname=${string%/*}"/"
    fi
   
    filename=${string##*/}
    resultfilename=$(echo ${string##*/} | sed -e "y/$accent/$nonaccent/" -e "s/[^$allowed]\+/$substitute/g")
   
    if [[ "$pathname$filename" != "$pathname$resultfilename" ]]
    then
        if ! mv --backup=numbered "$pathname$filename" "$pathname$resultfilename"
        then exit 1
        fi
    fi
    echo $result
fi


2. On the Windows guest:
 
A bat file contains an endless loop watching the "watch" file every 3 seconds. If the file exists then it reads out its contents (which are filenmames) open them then deletes the "'watch" file. A vbs file does nothing but invokes the bat script to eliminate the "cmd" window that the bat file would pop up. The vbs is started on Windows startup.

Create two files:

winopen.bat:

set watch=\\10.0.2.4\qemu\home\cuh\virt\watch.txt

:loop
if exist %watch% (
  for /f "delims=" %%i in (%watch%) do (
    icacls %%i /grant:r Everyone:F
    start %%i
  )
  del %watch%
)
timeout /t 3
goto loop


winopen.vbs:

Set oShell = CreateObject ("Wscript.Shell")
Dim strArgs
strArgs = "cmd /c C:\Users\cuh\winopen\winopen.bat"
oShell.Run strArgs, 0, false

Create a shortcut of the vbs file and place it in the startup folder to start the script automatically. The startup folder is:

C:\Users\cuh\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\



Now if you invoke winopen.sh under the host with a filename, the file should open under the guest.

E.g.:
winopen.sh <your_doc_file_under_linux.docx>

Changing key bindings of Firefox Quantum with xdotools

Firefox does not allow to change its key bindings since the Quantum series. There are extensions for that but none of them work as it should be. E.g. on built-in pages like the homepage or the about:blank page or when the URL bar is focused these extensions fail. There have been bug tickets about this issue for half a year now but sadly Mozilla is not doing anything to make its browser as flexible as it used to be in its pre-Quantum versions.

I have read ideas about changing the contents of the omni.ja file but anything I did it broke Firefox. At the same time this is a very dirty way of getting the job done as we have to change the installed files of Firefox meaning that an update can ruin our changes.

After quite a long research I found an idea to change key bindings at OS level "tricking" Firefox to accept key presses. The process contains of two steps:

1. We create a small shell script that checks if the active window is Firefox and if it is then it sends a key combination to it. Like that we can send a command (like "close tab", undo "close tab", etc. ) to the browser by sending Firefox's defined key combination ("ctrl+w", "ctrl+shift+t", etc. respectively) to the Firefox window. The commands and their key combinations can be checked under the Help menu of Firefox. The script uses xdotools to send the key combination.
2. We assign a key binding in our window manager to run the script (i.e. send the given command to the browser). Like that we can set up our key bindings in OS (or WM) level independently of Firefox.

In practice the two steps:

1. Creating the script:

I have created the script called mozkey.sh that accepts a Firefox command as argument and sends the appropriate key combination to Firefox. You have to install xdotool for this script to work.

/home/$USER/bin/mozkey.sh:

windowname="Firefox Nightly"
#windowname="Mozilla Firefox"

commands=(
"closetab;ctrl+w"
"undoclosetab;ctrl+shift+t"
)

# We check if the active window is a FF one, if so we store its id
if windowid=$(xdotool search --name "$windowname" | grep $(xdotool getactivewindow))
then
    for i in ${commands[@]}
    do
        [[ "$1" == "${i%%;*}" ]] && xdotool key --window "$windowid" "${i#*;}"
    done
fi

The windowname variable contains a pattern that fits the Firefox browser window. You can use xprop and check the WM_NAME attribute or simply read the window name in your window title to determine a correct pattern. I have Firefox Nightly build installed and its name always contain the string "Firefox Nightly".

The script accepts "closetab" or "undoclosetab" as argument but you can edit it to your liking and add more commands to the array with a new line of "<commandname>;<Firefox default binding>".

Its usage is very simple, the script accepts one argument which is the command to be sent to Firefox:
mozkey.sh <command>

It sends the appropriate key combination to Firefox.
mozkey.sh closetab sends ctrl+w, hence closes the actual tab.
mozkey.sh undoclosetab sends ctrl+shift+t, hence reopenes the last closed tab.


The script does nothing if you run it from a terminal because it checks if the active window is Firefox and if it is not (like in this case your active window is your terminal) then does nothing.



2. Defining your bindings:

I have set up my Awesome WM keybindings to run the script like this:

Pressing ctrl+0 executes /home/$USER/bin/mozkey.sh closetab
Pressing ctrl+1 executes /home/$USER/bin/mozkey.sh undoclosetab

My appropriate Awesome WM config lines are:

.config/awesome/rc.lua:

    -- Firefox key hack
    awful.key({ "Control" }, "0", function () awful.spawn("/home/$USER/bin/mozkey.sh closetab") end),
    awful.key({ "Control" }, "1", function () awful.spawn("/home/$USER/bin/mozkey.sh undoclosetab") end),

If you use another window manager set your bindings according to its configuration.

Now, in a Firefox window I press ctrl+0 and it closes the tab. I press ctrl+1 and it reopens the last closed tab.

Possible issues:
 
The thing is new, I haven't tested it a lot. Its caveat can be conflicting bindings. If you want to use a WM key binding which already exists in Firefox for another function that might cause problems since Firefox will get two key combinations at the same time.