Opening files under qemu windows guest from the Linux host

I migrated my VirtualBox Windows 7 client to qemu which uses kernel-level virtualisation. The performance improvement is mind blowing. The migration was really easy but I really missed my shell scripts for file integration tasks that made my life so much easier for years.

I had a well working bash script on my Linux machine that allowed my linux applications (like file manager or e-mail client, etc.) to open windows files (like doc, excel, etc.) in the Windows guest machine. VirtualBox has a "vboxmanage" tool to accomplis that. qemu does not have anything similar though. My script has helped me a lot during my everyday life and I simply don't want to give up this seamless integration between my host and guest. I work with documents and excel sheets in Windows but my everyday OS is Linux. In my Arch Linux OS I simply select a .doc or even an .exe file and it will be opened under the guest Windows. Similarly when I get a .ppt slideshow in e-mail as an attachment I simply click on it in my linux mail client and the powerpoint slideshow pops up in my Windows machine.

Migrating from VirtualBox to QEMU:

I sum up the migration process in bulletpoints but the whole process is well documented in the QEMU Arch Wiki page. I am not a qemu or virtualisation expert by any means.

So the process looks like this:

install qemu stuff (see arch wiki)
# pacman -S libvirt qemu spice-guest-tools-windows virtio-win spice spice-gtk
$ qemu-img convert -f vdi -O raw <your_Windows_vdi_file> <destination_image_file>

It looked like this in my case:
$ qemu-img convert -f vdi -O raw VirtualBox\ VMs/Windows/Windows\ Clone.vdi virt/win.img
 

This will take a while...

Try it:
qemu-system-x86_64 -m 3G -enable-kvm virt/win.img
Now your qemu-driven Windows guest should start in a window using the newly created img file.

Insert the virtio driver CD to the guest:
$ qemu-img create -f qcow2 fake.qcow2 1G
$ qemu-system-x86_64 -m 3G -enable-kvm -drive file=virt/win.img,if=ide -drive file=fake.qcow2,if=virtio -cdrom /usr/share/virtio/virtio-win.iso
Now the guest should start the same way as before but an inserted CD rom should show up in the machine.

Install the virtio guest drivers in the Windows guest that you just started. Windows did it automatically for me, probably you have to do some next-next-finish on the CD rom under your guest Windows.

Now insert the Spice Guest Tools CD to the guest similarly:
$ qemu-system-x86_64 -m 3G -drive file=virt/win.img,if=ide,format=raw -drive file=fake.qcow2,if=virtio -cdrom /usr/share/spice-guest-tools/spice-guest-tools.iso

In the Windows guest run the .exe on the cdrom to install the Spice Guest Tools.

Start the thing without gui and connect to it with a Spice client (some cool features enabled):
$qemu-system-x86_64 -nographic -m 3G -drive file=virt/win.img,if=virtio,format=raw -spice port=5900,addr=127.0.0.1,disable-ticketing -device virtio-serial -chardev spicevmc,id=vdagent,debug=0,name=vdagent -device virtserialport,chardev=vdagent,name=com.redhat.spice.0 -vga qxl -machine type=pc,accel=kvm -usb -device usb-tablet -net nic -net user,smb=/

In another terminal connect to the spice server:
$spicy -h 127.0.0.1 -p 5900

Now the guest machine should appear in a spice client window. The screen resolution should appply when you resize this window and the clipboard should be shared between the host and the guest. You also have file access to your Linux host: the host's / folder should be accessible in the guest under \\10.0.2.4\qemu\

If all is ok then install the headless version of qemu instead of the normal one since we use the spice guest to connect to the server:
#pacman -S qemu-headless

Setting up file acceess:
The process is based on two components: 1. on the Linux host we write the path of a file (in windows format) that we want to open under windows TO A "watch" file. In the Windows guest we read the contents of this file every 3 seconds and if we find a filename in it then we open that file and delete the "watch" file. This is not very nice but I am not a programmer and I wanted to use native commands instead of external apps.

1. On the linux host:
This script will acept a filename (i.e. a doc file or similar) as argument. It removes special characters from the filename (using another script but you can use detox or anything similar) then converts the file pah to windows format and writes this path to the "watch" file. (The "watch" file will be processed on the other side in the guest.)

My script is called winopen.sh and looks like this:


#!/bin/bash

# needs 'clean' script

# stdout and errors go here:
exec 2>>"/home/<username>/.xlog"
exec 1>&2

script="$(basename "$0")"
driveletter='\\10.0.2.4\qemu'
watchfile="/home/<username>/virt/watch.txt"

# Start vm if not running already

if (( $(ps -ef | grep qemu-system-x86_64 | wc -l) == 1 ))
then ( qemu-system-x86_64 -m 3G -drive file=/home/<username> /virt/win.img,if=virtio,format=raw -spice port=5900,addr=127.0.0.1,disable-ticketing -device virtio-serial -chardev spicevmc,id=vdagent,debug=0,name=vdagent -device virtserialport,chardev=vdagent,name=com.redhat.spice.0 -vga qxl -machine type=pc,accel=kvm -usb -device usb-tablet -net nic -net user,smb=/ &
    ( sleep 4 ; spicy -f -h 127.0.0.1 -p 5900 ) &
    )
fi

# Get the absolute path of the file on the linux host
for i in "$@"
do
    if echo $i | grep '^/' > /dev/null
    then targetfullpath=$i
    else targetfullpath=$PWD'/'$i
    fi
   
    #Cleanup filename
    if ! cleanpath=$(clean -F "$targetfullpath")
    then
        echo "File cleaning failed"
        exit 1
    fi

    # The same filepath under the windows guest machine
    winfullpath=$(echo "$driveletter""$cleanpath" | sed 's/\//\\/g')

    # Place the filename to the watchfile
    echo "$script"": opening ""$targetfullpath"" (""$winfullpath"") in qemu"
    echo "$winfullpath" >> $watchfile
done



The path cleaning script called 'clean':

cat bin/clean
#!/bin/bash

accent='ÀÁÂÃÄÅÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖŐØÙÚÛÜŰÝàááâãäåçèééêëìíîïðñòóôõőöőøùúûüűý'
nonaccent='AAAAAACEEEEIIIIDNOOOOOOOUUUUUYaaaaaaaceeeeeiiiidnoooooooouuuuuy'
allowed='.ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789'
substitute='_'
fullpathchars='~/'

# Parse command line arguments
shortopts="a:fFhs:"
longopts="allow:,fullpath,filerename,help,substitute:"
options=$(getopt -n $(basename $0) -u -o $shortopts -l $longopts -- "$@") || exit 1
set -- $options
while [ $# -gt 0 ]
do
    case $1 in
    # for options with required arguments, an additional shift is needed
    -a|--allow) allowed+="$2" ; shift ;;
    -f|--fullpath) allowed+="$fullpathchars" ;;
    -F|--filerename) allowed+="$fullpathchars" ; filerename="true" ;;
    -h|--help) help=y ;;
    -s|--substitute) substitute="$2" ; shift ;;
    (--) ;;
    (*) break ;;
    esac
    shift
done

# --help #
if [[ $help == "y" ]]
then
    cat <<EOF
clean: a tool for cleaning a string from special characters. Transforms the accented characters of STRING to their non-accented form and substitutes special characters with a substitute string, "_" by default. Allowed characters are lower- and uppercase letters, numbers and '.' by default, other characters are substituted. More than one non-allowed subsequent characters are substituted with only one substitute string.

Usage: clean [OPTION] STRING
    -a "ALLOW_STR", --allow="ALLOW_STR"  Accepts characters of ALLOWED_STR besides the default allowed characters.
    -f, --fullpath                       Accepts the tilde and slash ("~" and "/") character besides the allowed characters (useful for transforming full pathnames)
    -F --filerename                      Accepts a file name as STRING. It will rename the file to the cleaned string. If the file with the cleaned name already exists, it creates a backup of that before renaming.
    -h, --help                           Print this help

Default values:
---------------
The following accented characters are transformed to non-accented ones:
$accent

The default allowed (non-substituted) characters are:
$allowed

The default substitute character is: $substitute

EOF
exit
fi

string="$*"
#substitute=$(echo "$substitute" | sed 's/[\/\\\?\&]/\\&/g')

result=$(echo "$string" | sed -e "y/$accent/$nonaccent/" -e "s/[^$allowed]\+/$substitute/g")

if [[ $filerename = "true" ]]
then
    if [ ! -f "$string" ]
    then
        echo "Cannot find file or not a regular file: $string"
        exit 1
    fi

    if echo $string | grep '/' > /dev/null
    then pathname=${string%/*}"/"
    fi
   
    filename=${string##*/}
    resultfilename=$(echo ${string##*/} | sed -e "y/$accent/$nonaccent/" -e "s/[^$allowed]\+/$substitute/g")
   
    if [[ "$pathname$filename" != "$pathname$resultfilename" ]]
    then
        if ! mv --backup=numbered "$pathname$filename" "$pathname$resultfilename"
        then exit 1
        fi
    fi
    echo $result
fi


2. On the Windows guest:
 
A bat file contains an endless loop watching the "watch" file every 3 seconds. If the file exists then it reads out its contents (which are filenmames) open them then deletes the "'watch" file. A vbs file does nothing but invokes the bat script to eliminate the "cmd" window that the bat file would pop up. The vbs is started on Windows startup.

Create two files:

winopen.bat:

set watch=\\10.0.2.4\qemu\home\cuh\virt\watch.txt

:loop
if exist %watch% (
  for /f "delims=" %%i in (%watch%) do (
    icacls %%i /grant:r Everyone:F
    start %%i
  )
  del %watch%
)
timeout /t 3
goto loop


winopen.vbs:

Set oShell = CreateObject ("Wscript.Shell")
Dim strArgs
strArgs = "cmd /c C:\Users\cuh\winopen\winopen.bat"
oShell.Run strArgs, 0, false

Create a shortcut of the vbs file and place it in the startup folder to start the script automatically. The startup folder is:

C:\Users\cuh\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\



Now if you invoke winopen.sh under the host with a filename, the file should open under the guest.

E.g.:
winopen.sh <your_doc_file_under_linux.docx>

Changing key bindings of Firefox Quantum with xdotools

Firefox does not allow to change its key bindings since the Quantum series. There are extensions for that but none of them work as it should be. E.g. on built-in pages like the homepage or the about:blank page or when the URL bar is focused these extensions fail. There have been bug tickets about this issue for half a year now but sadly Mozilla is not doing anything to make its browser as flexible as it used to be in its pre-Quantum versions.

I have read ideas about changing the contents of the omni.ja file but anything I did it broke Firefox. At the same time this is a very dirty way of getting the job done as we have to change the installed files of Firefox meaning that an update can ruin our changes.

After quite a long research I found an idea to change key bindings at OS level "tricking" Firefox to accept key presses. The process contains of two steps:

1. We create a small shell script that checks if the active window is Firefox and if it is then it sends a key combination to it. Like that we can send a command (like "close tab", undo "close tab", etc. ) to the browser by sending Firefox's defined key combination ("ctrl+w", "ctrl+shift+t", etc. respectively) to the Firefox window. The commands and their key combinations can be checked under the Help menu of Firefox. The script uses xdotools to send the key combination.
2. We assign a key binding in our window manager to run the script (i.e. send the given command to the browser). Like that we can set up our key bindings in OS (or WM) level independently of Firefox.

In practice the two steps:

1. Creating the script:

I have created the script called mozkey.sh that accepts a Firefox command as argument and sends the appropriate key combination to Firefox. You have to install xdotool for this script to work.

/home/$USER/bin/mozkey.sh:

windowname="Firefox Nightly"
#windowname="Mozilla Firefox"

commands=(
"closetab;ctrl+w"
"undoclosetab;ctrl+shift+t"
)

# We check if the active window is a FF one, if so we store its id
if windowid=$(xdotool search --name "$windowname" | grep $(xdotool getactivewindow))
then
    for i in ${commands[@]}
    do
        [[ "$1" == "${i%%;*}" ]] && xdotool key --window "$windowid" "${i#*;}"
    done
fi

The windowname variable contains a pattern that fits the Firefox browser window. You can use xprop and check the WM_NAME attribute or simply read the window name in your window title to determine a correct pattern. I have Firefox Nightly build installed and its name always contain the string "Firefox Nightly".

The script accepts "closetab" or "undoclosetab" as argument but you can edit it to your liking and add more commands to the array with a new line of "<commandname>;<Firefox default binding>".

Its usage is very simple, the script accepts one argument which is the command to be sent to Firefox:
mozkey.sh <command>

It sends the appropriate key combination to Firefox.
mozkey.sh closetab sends ctrl+w, hence closes the actual tab.
mozkey.sh undoclosetab sends ctrl+shift+t, hence reopenes the last closed tab.


The script does nothing if you run it from a terminal because it checks if the active window is Firefox and if it is not (like in this case your active window is your terminal) then does nothing.



2. Defining your bindings:

I have set up my Awesome WM keybindings to run the script like this:

Pressing ctrl+0 executes /home/$USER/bin/mozkey.sh closetab
Pressing ctrl+1 executes /home/$USER/bin/mozkey.sh undoclosetab

My appropriate Awesome WM config lines are:

.config/awesome/rc.lua:

    -- Firefox key hack
    awful.key({ "Control" }, "0", function () awful.spawn("/home/$USER/bin/mozkey.sh closetab") end),
    awful.key({ "Control" }, "1", function () awful.spawn("/home/$USER/bin/mozkey.sh undoclosetab") end),

If you use another window manager set your bindings according to its configuration.

Now, in a Firefox window I press ctrl+0 and it closes the tab. I press ctrl+1 and it reopens the last closed tab.

Possible issues:
 
The thing is new, I haven't tested it a lot. Its caveat can be conflicting bindings. If you want to use a WM key binding which already exists in Firefox for another function that might cause problems since Firefox will get two key combinations at the same time.

Tiling wm font rendering

Desktops like KDE or GNOME have their nice panels to set up themes and fonts but in a simple tiling window manager there are no such settings. From time to time I face problems with font rendering on xorg updates. Fonts just start looking ugly from one day to another.

Xorg's font configuration is handled by the extra/fontconfig package as a dependency of the xorg installation.

Configuration can be easily done by creating symlinks of predefined config files under /etc/fonts/conf.avail/ under the /etc/fonts/conf.d/ directory as per the Font Configuration Arch Wiki.

Fontconfig sets up default symlinks in /etc/fonts/conf.d/ which is a good start. But here the problem comes! If you change these symlinks to your liking a fontconfig upgrade can mess them up, hence you have to redo the configuration after such upgrades.

How to override this?

You can specify per-user defined symlinks under the .config/fontconfig/conf.d/ dir. But if there are conflicts then the global one takes precedence (why?!). To override this you do this:

# cd /etc/fonts/conf.d
# cp 50-user.conf 00-user.conf

Then set up your font config symlinks to your taste under your user config directory. I want to change the hinting setting from the globally defined "slight" one (which gives ugly look on my LCD) to "full". I do it with this symlink command:

$ ln -s /etc/fonts/conf.avail/10-hinting-full.conf .config/fontconfig/conf.d/

Set ranger as the default file manager of Firefox

I have just created a short script for using ranger (>=version 1.4.2) as the default file manager to open files from Firefox. This is how you make it:

Create a ~bin/fileopen.sh file with this content:

#!/bin/bash
 
log=$HOME/.xlog
path=${1#file://}
 
if [ -d $path ]
then
     /usr/bin/xterm -e "/usr/bin/ranger $path" &>> $log
else
     /usr/bin/xterm -e "/usr/bin/ranger --selectfile=$path" &>> $log
fi

Of course you make your script executable:

chmod 755 ~bin/fileopen.sh

In Firefox go to Edit > Preferences > Applications
Set the action for the type "file" and whatever other entries to use your fileopen.sh script for opening these types of files. E.g. I have set "file", "GZ file", "Plain text document", "raw CD image", "shell script" and "Zip archive" types to be handled by ranger. Additionnally when Firefox asks how to open such a file, set it up to use fileopen.sh as well.

You are done. Download something and/or open your recent download list in Firefox and an xterm window should pop up with a ranger instance opening the file. If the file is a directory (e.g. when you click "Open Containing Folder" in Firefox) then it will not only choose the directory in the tree but will also enter it.

Awesome WM Reloaded!

I have been playing around with plenty of tiling window managers and until now none of them worked as I wanted. Finally I have succeeded to set up Awesome WM to meet all my needs but I really had to work hard to get to this point. I cannot go further without mentioning the dark sides of Awesome: things are just not working as they are written, sample config files are unfunctional, modules and config scripts are far not compatible with each other, AUR packages are sometimes simply not installable. For example I cannot use bashets because it just does not work on my system although I use the sample config file coming with the latest bashet release. I tried to upgrade my stable awesome to awesome-git from AUR but I could not manage to do it, yaourt complains about a non-existent(!) package as a dependency:
error: target not found: xcb-util-image 
What is this all around? Why is it so difficult?
(UPDATE: xcb-util problem is solved by now, thanks to the package maintainers!).


OK, now the awesome part of awesome and the workarounds for some issues:

Installation:

I installed these packages:

• awesome (3.4.13-1),  community repository 
• vicious (2.1.0-1),  community repository
• shifty-github (20120913-1), AUR
• awesome-scratchpad-git (20120913-1), (optional, if you need a scratchpad), AUR

Configuration tips:

See my rc.lua, it works nicely with the above-mentioned software versions.


Changed "require" format:

From Lua 5.1 the way you have to load modules in your rc.lua has changed:

Older format:
require("vicious")

New format:
vicious = require("vicious")

Vicious, for example will not work with the older format!


Raise clients on tag switching:

Add this line to your rc.lua:

require("awful.autofocus")

If you do not do this, navigating between tags will not focus the clients which is quite annoying in my opinion.


Run or raise:
If you want to have only one instance of certain applications you can place a small code to the end of your rc.lua for defining the run_or_raise function. You can read more on this here. After this you can define your key bindings to run something only once like this:

awful.key({ modkey,}, "f", function () run_or_raise("firefox", { class = "Firefox" }) end),

As a result of this example when you press MOD+f firefox will start or if there is already an existing window with WM_CLASS="Firefox" running that window will gain focus.


Skype - as usual - needs special attention when you deal with window management. Run_or_raise will start a new instance of skype if it is already sitting in your systray. To prevent it you can write a script that examines if skype is already running. My skypestart script looks like this:

 if [ "$(pgrep skype)" ]
  then
     echo skype is already running
  else
     skype
fi

And the appropriate rc.lua line for the binding is:

awful.key({ modkey,}, "s", function () run_or_raise("<your path>/skypestart", { name = "Skype" }) end),


Use your custom theme:

beautiful.init(".config/awesome/themes/<your_theme_dir>/theme.lua")

See my theme.lua.

Custom statusbar script using vicious:
You can set up a shell script to create a status message for your status bar. This script can contain colour codes to have a nicer output. For example I write the battery percentage with red when its value is under 30% instead of the normal orange colour. My bar looks like this:

My status.sh script to generate this output:


span_hi='<span color="#ff8700">'   #colour for normal battery
span_lo='<span color="#aaaaaa">'     #colour for normal battery
span_sep='<span color="#ff8700">'  #colour for normal battery
span_warn='<span color="#ff0000">'

## Set battery message (works on Thinkpad Edge 11 with tp_smapi driver)

batt=`cat /sys/devices/platform/smapi/BAT0/remaining_percent`
if (($batt <= 30));
then battstate=$span_warn$batt'%% </span>'$span_lo`cat /sys/devices/platform/smapi/BAT0/state`'</span>'
else battstate=$span_hi$batt'%% </span>'$span_lo`cat /sys/devices/platform/smapi/BAT0/state`'</span>'
fi

## Create the output

echo ' '$battstate$span_sep' | </span>'$span_lo`cat /proc/cpuinfo | grep "cpu MHz" | awk '{sum+=$4} END { print sum/NR,"MHz"}'`'</span>'$span_sep' | </span>'$span_lo`date "+%a, %Y.%m.%d."`' - </span>'$span_hi`date +"%R"`' </span>'

I initialise this status script from my rc.lua with vicious which upgrades the status bar every 10 seconds:

statwidget = widget({
   type = 'textbox',
   name = 'statwidget'
})

function run_script()
    local filedescriptor = io.popen("/<path>/<to>/<your>/status.sh")
    local value = filedescriptor:read()
    filedescriptor:close()
    return {value}
end
vicious.register(statwidget, run_script, '$1', 10)


And I call the vicious widget wen I construct my top bar later in rc.lua:

    topbar[s].widgets = {
        {
            mytaglist[s],
            mypromptbox[s],
            layout = awful.widget.layout.horizontal.leftright
        },
        mylayoutbox[s],
        s == 1 and mysystray or nil,
        statwidget,
        mytasklist[s],
        layout = awful.widget.layout.horizontal.rightleft
        }

Scratchpad calculator:
It is good to have a calculator at hand in a floating window. The scratchpad is an easy way to pop up a floating window on the actual screen. You can also set the default geometry of the window so it will always show up with the same size and place. With a key binding you can pop up or hide this window.


For this you need to install a calculator, I use qalculate-gtk. You also need  the awesome-scratchpad-git AUR package and some configuration in rc.lua:

-------libraries section------

require("scratch")

------globalkeys section------

 -- Binding for calculator in scratchpad
awful.key({ modkey }, "q", function () scratch.drop("qalculate","top","center",250,300) end),

If you need a calculator you can pop it up any time with Mod+q. When you do not need it simply hide it with the same binding.


Tip: How to take screenshots?
You need scrot, sxiv and xfe installed for this to work. Place this line to your rc.lua globalkeys section:

awful.key({ }, "Print",  function () awful.util.spawn_with_shell("scrot -e 'mv $f ~/Desktop/ 2>/dev/null && xfe ~/Desktop/ & sleep 1 && sxiv ~/Desktop/$f'") end),

Your PrintScreen key will take a screenshot and pop it up in an sxiv image viewer window and also in an xfe file manager window for further use.


Dual screen and VGA hotplugging
One of the major benefits of Awesome is that it can handle a dual head setup quite flawlessly and in an easy-to-configure manner. When you attach or detach displays in your system you can redistribute your clients between your actual monitors by means of a simple keystroke. First you need to install xrandr to fulfil this task. Xinerama might be supported as well, I did not test it with Awesome.

A simple script using xrandr can be handy to make the changes easy. I have a laptop with an LCD panel (LVDS1) and some connectors (VGA1, HDMI1, DP1) where other displays can be attached to the system. My dual_screen script examines if an external display is attached to any of these ports and if it finds one it will make that one a primary display and sets the LVDS1 to be the secondary one placed on the left hand side of the primary display.
If it does not find any attached monitors it will set LVDS to be the one and only, primary display. My .xinitrc and dual-screen scripts to configure dual head setup can be found here.

By running the dual_screen script and restarting Awesome the clients will be arranged according to the rules of your rc.lua.

I placed a key binding to the global keys of my rc.lua to do this on a keystroke:

-- VGA hotplugging and restarting awesome. The dual-screen bash script runs an xrandr screen setup.
    awful.key({ modkey, "Control"}, "r",
        function ()
        awful.util.spawn_with_shell("/dev/shm/scripts/dual-screen")
        awesome.restart()
        end),

And according to my Shifty rules in my rc.lua there are clients to be placed on screen 1 (primary screen) and others on screen 2 (secondary screen). If there is only one screen then Awesome will place all clients on the primary screen.

And the results

 

Two simple yaourt tips

Just a quick post to improve your experience with yaourt. For many it will be straighforward but maybe it will be useful for others.

If yaourt asks too much...

If you fed up with these questions asked all the time by yaourt:
Edit PKGBUILD ? [Y/n] ("A" to abort)
Edit <pkg_name>.install ? [Y/n] ("A" to abort)
Continue building <pkg_name> ? [Y/n]

The simple solution is to create a .yaourtrc under your home directory and configure yaourt to your taste. You can use the global /etc/yaourtrc file as a sample:

cp /etc/yaourtrc ~/.yaourtrc

Edit .yaourtrc to your liking according to http://archlinux.fr/man/yaourtrc.5.html

I simply edited two lines:

BUILD_NOCONFIRM=1

EDITFILES=0

These two will prevent yaourt to ask for editing anything and for continuing the build, it will skip these steps and will build the packages for you automatically. Still it will leave you some manual controls not to install anything unintentionally:

• When you do system upgrade ( yaourt -Syua ), it will tell you what packages are to be installed and will ask "Continue upgrade ? [Y/n]".

• It will always ask the last question for installing a package after building it: "Proceed with installation? [Y/n]". 

Search and install interactively

If you use yaourt without options ( i.e. yaourt <pattern> ) it will search packages just it would do with the -Ss option ( i.e. yaourt -Ss <pattern> ) but it will mark the results with numbers and ask if you want to install one of them. Just type the number of the chosen package and hit Enter to install it. If you hit Enter without a number yaourt will quit.


If you have any other tips for yaourt, please do not hesitate to leave a  comment!

VirtualBox tips and tricks - awesome wm integration

Hiding the menu and statusbar of VirtualBox

I finally managed to set up awesome wm to work just as I want. I only had some minor problems and customisation needs with my windows VirtualBox guest:

- Scale mode simply freezes my guest OS, so I do not use it.
- In seamless, fullscreen or scale modes Virtualbox's host key has to be pressed to leave the guest window. As I use almost every window management opertion by keystrokes, it was a bit annoying not to be able to switch out from my windows guest with a simple keystroke but I had to press the host key first. Fortunately in normal view (with the menu bar and the status bar on the top and bottom of the window) you can switch out directly from the guest without using the host key.
- In normal mode the menu and status bars take a bit of a place needlessly.

I tried some tricks to resize my VB guest in floating mode to "overlflow" my screen but these tricks did not work well. Finally I found the solution here.

You can hide the menu and status bars of the VirtualBox window with this simple command:

vboxmanage setextradata global GUI/Customizations noMenuBar,noStatusBar

Restart Virtualbox and that's it, you are done! :)

Disabling the "Win" key

I configured my window manager to use the "Win" key as the major alt key of keystrokes. Under my virtual windows guest I had a small annoyance, i.e. when using the "win" key for a window management task the windows menu always shown up. If you have the same issue, you can disable the "win" key under windows by simply downloading and executing the appropriate "Fix it" binary provided by Microsoft's support page.

UPDATE: recently I installed a Windows guest machine to a new computer. The method above eliminated the Win key under the guest but my tiling window manager did not work well. The window manager's "Win" key bindings did not work when I was inside the virtualbox guest. To change this behavior:

Open the VirtualBox GUI, go to "File" > "Preferences" menu, then choose the "Input", uncheck the box labeled "Auto Capture Keyboard". There is no need to restart the VM if it's running, so this can be changed "on the fly". Thanks to the Ubuntu folks here.

Hiding the recycle bin

A small cosmetics on the windows desktop: hide your recycle bin.

After all these treatments my win7 guest OS looks likesimple and clean and above all integrates very well ito my window manager.

Also, please take a look at this post if you are interested in a solution for opening windows files in VirtualBox directly from your Linux host.

Opening files under VirtualBox guest from the Linux host

Updated: 08/06/2015

I have been using VirtualBox to run windows software on my linux box for a while. I always wondered how nice it would be to simply click on a windows file (.doc, .xls, .exe, etc.) under linux to open it with the appropriate application (winword, excel, etc.) under the virtual machine instead of saving the file and looking it up under windows. I did not find any solution for this so I looked around and invented my own one. The operation is based on the VBoxManage guestcontrol utility provided by VirtualBox. This command allows you to initiate program execution inside the guest from the host. A small bash script handles this on the host by converting the filename to windows format and opening the file inside the guest Windows machine.

The steps for setting all these up:

Add the / directory to the windows guest

Install VirtualBox guest additions if it is not installed already. I do not want to describe how to install guest additions, it is well documented elsewhere. Briefly under Arch Linux you have to install the virtualbox-additions AUR package, add /usr/lib/virtualbox/additions/VBoxGuestAdditions.iso to your VirtualBox storage and install the guest additions under the guest.

Create a shared folder under VirtualBox settings and add your linux host's root (/) filesystem. Now let's say your linux / direcrory is the f: drive under the windows guest.

As a result a file named /home/<your_username>/document.doc under linux looks like f:\home\<your_username>\document.doc under the guest.

Install Quiet.exe under the windows guest

UPDATE: Under my latest Win7 guest installation cmd.exe works just fine without bringing up a command window and without the need of setting up environment variables. If this is not the case you can use Quiet.exe instead.

To open files remotely you can use cmd.exe or the start command but both opens a console window besides the program that you want to run which I find quite annoying. To hide it you have to use additional software: a hand-made vbs script or a third-party tool like hstart, cmdow or Quiet. I will follow the last one, this simple tool works fine.

Download Quiet from their website and copy the Quiet.exe anywhere you like under your windows guest.

Let's say it will be here:

c:\Users\<your_windows_username>\Quiet.exe

Enable passwordless guest control under the windows guest

If you do not use password in your windows guest (like me), the guest's group policy must be changed under the guest OS. To do so, open the group policy editor on the command line by typing gpedit.msc, open the key Computer Configuration \ Windows Settings \ Security Settings \ Local Policies \ Security Options and change the value of Accounts: Limit local account use of blank passwords to console logon only to Disabled.

Without this step you will get an error when trying use VboxManage:
VBoxManage: error: The specified user was not able to logon on guest.

Install detox for cleaning up filenames

Not only me but VboxManage also hates special characters and spaces in filenames. Detox is a handy tool for swiping such garbages out of filenames. It will replace spaces to underline sign, convert characters with accents for the non-accent version and remove special characters. You can install it from AUR then use it in your scripts (like I did below).

Create the winopen.sh script

UPDATE: since Virtualbox 5.0.0 the execute subcommand of VBoxManage guestcontrol is deprecated. Use run instead with the --exe option (see the script below). 

UPDATE: since Virtualbox 5.0.0 instead of --environment use the --putenv option in your "VBoxManage guestcontrol ... run" command (see the script below). 

UPDATE: since Virtualbox 5.1.x instead of run use start! run gives session problems, commands stuck after termination. run works for me. Oracle's documentation sucks by the way.
 
Create a winopen.sh bash script under the linux host. This will be the command for opening windows files within the guest.

#!/bin/bash

# stdout and errors go here:
exec 2>>"/home/<your_linux_username>/.xlog<or_anything_you_like>"
exec 1>&2

script="$(basename "$0")"
vmname="<the_name_of_your_windows_guest_under_Virtuabox>"
driveletter="f:"  #your ROOT folder's drive letter comes here 

# Start vm if not running already
if [ $(VBoxManage list runningvms | wc -l) = 0 ]; then
   VBoxManage startvm $vmname
   echo "Please wait, starting virtual PC..."
   sleep 5s
fi

until VBoxManage showvminfo $vmname | grep '"Graphics Mode": active/running' > /dev/null
 do
  sleep 0.5s
 done

# Set the absolute full path of the file in the linux host
for i in "$@"
do
 if echo $i | grep '^/' > /dev/null
   then targetfullpath=$i
   else targetfullpath=$PWD'/'$i
 fi

#Cleanup filename, you need detox installed! 
cleanpath=$(detox -vs utf_8 "$targetfullpath" | tail -1 | awk '{print $NF}')

# The same filepath under the windows guest machine
winfullpath=$(echo $driveletter$cleanpath | sed 's/\//\\/g')

# Open the file under windows
echo $script": opening "$targetfullpath" ("$winfullpath") in Virtualbox"
#sleep 1   # uncomment and increase this if you see empty Winword windows
#In newer Windows installations try this:
VBoxManage guestcontrol "$vmname" start --exe "cmd.exe" --username 'cuh' --putenv "USERPROFILE=C:\Users\<your_windows_username>" --putenv "APPDATA=C:\Users\<your_windows_username>\AppData\Roaming" -- "cmd" "/c" $winfullpath &
# For older installations:
#VBoxManage guestcontrol "$vmname" execute --image "cmd.exe" --username "<your_windows_username>" -- "cmd" "/c" $winfullpath & 
# If you want to use quiet.exe or have problems with environment variables, try this:
#VBoxManage guestcontrol "$vmname" execute --image "C:\Users\<your_windows_username>\Quiet.exe" --wait-exit --username "<your_windows_username>" --environment "USERPROFILE=C:\Users\<your_windows_username>" --environment "APPDATA=C:\Users\<your_windows_username>\AppData\Roaming" -- "cmd" "/c" $winfullpath &

done


Make your script executable, i.e.:
chmod 755 winopen.sh

Optionally place winopen.sh to your path, (i.e. /home/<your_username>/bin/winopen.sh and put export PATH=$PATH:~/bin under your .bashrc).

Important notices on environment variables:
Without the --environment "USERPROFILE=C:\Users\<your_windows_username>" attribute in your winopen.sh script your guest OS sometimes does not get this environment variable and cannot open your files. You get errors like this:
"your AutoCorrect file, MSO1038.acl could not be saved." or "Microsoft Excel cannot access the file ... There are several possible reasons: The file name or path does not exist..."

The lack of the --environment "APPDATA=C:\Users\<your_windows_username>\AppData\Roaming" attribute can slow down Word document spell checking.

In general all the environment variables should be the same as in the Windows guest environment but I was just lazy to write all the environment set to the VboxManage command.


Usage

Invoking the winopen.sh script with one or more filenames will start your Virtualbox (if it was not running already), clean up your filename(s) (if there were special characters in them) and open the file(s) under the windows guest. For example:

winopen.sh /home/<your_username>/document.doc

or simply (with relative pathname):

winopen.sh document.doc

You can configure Thunderbird, Firefox, file managers, etc. to use winopen.sh when opening windows files (.doc, .xls, ppt, .exe, etc.), so you simply need to click on an e-mail document attachment or an excel link and it will open automatically under windows.

Any improvements are welcome!

Arch Linux on Thinkpad Edge 11

I use Arch Linux on my Thinkpad Edge 11 notebook (Intel version with Core i3 Duo (i3-380UM)). This is my office device and I spend my everyday life using this tiny little computer. I briefly concluded the first steps and some minor issues during installation here. Now I am collecting all the device-specific settings I made for fine-tuning Arch Linux for this laptop model.

TP smapi

This feature set provides Thinkpad users with detailed battery charge/discharge control, battery status information and hard disk protection.
The package is called tp_smapi and can be installed from AUR. After installation add tp_smapi to your MODULES array of your /etc/rc.conf.
Details can be found in the tp_smapi page of the Arch Wiki.

After installation and loading the tp_smapi module several useful information can be read out and set up from the /sys/devices/platform/smapi/ directory.

The default battery is BAT0, its details can be read from the files under

/sys/devices/platform/smapi/BAT0/

I show some examples how this service can be used:

Battery indicator
I use simple tiling window managers (like awesome and wmfs) and I created a statusbar script to show (among others) the actual battery percentage and status, The bash code for showing a text like "80% charging" is like this:

echo "`cat /sys/devices/platform/smapi/BAT0/remaining_percent`"% "`cat /sys/devices/platform/smapi/BAT0/state`"

See my wmfs-status script here.

Battery notification script
I created a notification script that uses the notify-send command as part of a notification daemon (xfce4-notifyd or similar):

state=`cat /sys/devices/platform/smapi/BAT0/state`
if [ "$state" = 'discharging' ];
  then batt=`cat /sys/devices/platform/smapi/BAT0/remaining_percent`
  if (($batt <= 30));
    then if (($batt <= 10));
           then notify-send -u critical "Battery is low!
`acpi -b | tail -c 24 | head -c 10` remaining" ;
           else notify-send "Battery is critical!
`acpi -b | tail -c 24 | head -c 10` remaining" ;
         fi
  fi
fi


You can run this script  from your .xinitrc like:

 (while true; do sleep 300s; /home/cuh/bin/battwarning.sh; done) &

As a result you will be notified every 5 minutes when your battery goes below 30% and you get a critical warning when your voltage is under 10%.

See my full .xinitrc here.


Battery protection
Some parameters under /sys/devices/platform/smapi/BAT0/ can be set by writing to the files of this directory. For example you can force discharging the battery even if the power supply is plugged in (it can be useful when your battery is over the stop_charge_thresh, see below) by setting the force_discharge parameter to 1 (the default value is 0) by this command:

echo 1 > /sys/devices/platform/smapi/BAT0/force_discharge

or you can set the battery charging thresholds for keeping the current capacity between 30% and 80% for protecting your LiIon battery from low and high percentages (see here) by setting the start_charge_thresh and stop_charge_thresh values. Unfortunately on my Thinkpad Edge 11 there seems to be a bug in this area because I cannot set both threshold limits. Changing one changes the other so that start_charge_thresh is always a higher value than the stop_charge_thresh (who understands...). Anyway, I do not really need to set the lower threshold level because when I am on DC power and my battery discharges I get a notification to plug in the power supply and if I am in a situation that I can plug it in I do it so. I limit the upper threshold to 80% to protect my battery from overcharging. To do so I put this line to my /etc/rc.local to set the threshold at every boot:

echo 80 > /sys/devices/platform/smapi/BAT0/stop_charge_thresh

HDD drop protection - not working on Edge 11
According to the tp_smapi documentation this package contains the hdaps driver as well, which is responsible for HDD protection against physical shocks by means of a built-in accelerometer's signals. After installing tp_smapi I loaded the module:

modprobe hdaps

The module is loaded but dmesg says:

thinkpad_ec: thinkpad_ec_read_row: failed requesting row: (0x01:0x00)->0xfffffff0

At the same time my battery status modules stopped working, I cannot access the contents of the /sys/devices/platform/smapi/BAT0 files.


I read that a temporary workaround might be to switch off my laptop, unplug the power supply, remove the battery and place it back after some minutes. Funny that this can be a workaround, but I faced the same with the malfunctioning Fn buttons, too (see below). This time it did not help, now I cannot even load the hdaps driver.

I tried to overwrite the native kernel hdaps driver by the tp_smapi driver:

mv /lib/modules/3.0-ARCH/kernel/drivers/platform/x86/hdaps.ko.gz /lib/modules/3.0-ARCH/kernel/drivers/platform/x86/hdaps.ko.gz_backup

cp /lib/modules/3.0-ARCH/extra/hdaps.ko.gz /lib/modules/3.0-ARCH/kernel/drivers/platform/x86/hdaps.ko.gz

and this time I got back to the loadable module with the dmesg warning and switched-off battery framework.

I looked around and some people say that the hdaps service is not available on this Thinkpad model, I can confirm this.

cpu_frequtils

See the details at www.thinkwiki.org.CPU scaling and profiling works fine on the Edge 11, the Extra repository contains the cpufrequtils package and the Arch Wiki is pretty helpful as usual. Instead of setting up your frequency scaling manually you can use Laptop Mode Tools, see below. After installing cpufrequtils and setting up LMT you only need to add acpi-cpufreq to your MODULES in /etc/rc.conf and edit the /etc/laptop-mode/conf.d/cpufreq.conf file to your taste.

Laptop Mode Tools

Laptop Mode Tools is a centralised power management solution featuring several power saving services. These services are controlled by only one daemon. In practice this means that you only need to install the laptop-mode-tools package, place laptop-mode to your DAEMONS list of your /etc/rc.conf and tune the power saving options as you like with the appropriate config files: /etc/laptop-mode/laptop-mode.conf and /etc/laptop-mode/conf.d/* . You can set up different CPU governors, different LCD brightness, auto switching off your WiFi antenna when idle, etc. features according to the actual power state (AC / DC) of your laptop. Installation and configuration is well described in the LMT Arch Wiki.

Strange bug of Thinkpad function buttons

Sometimes I have problems with the brightness and volume Fn buttons. They simply do not work. Nothing helps here, neither restarting daemons, nor rebooting the machine. Very strangely the only solution is to switch the laptop off, remove the battery and place it back after 5 secs, and boot up the machine again. I haven't faced this problem for a while, maybe a kernel upgrade (or who knows what) solved it permanently.

Synaptics touchpad

To fine-tune the touchpad you have to install the xf86-input-synaptics package and set up the options under /etc/X11/xorg.conf.d/10-synaptics.conf

With the default settings I had an odd problem with double taps that wanted to drag and drop elements instead of releasing the mouse click. I worked a lot to achieve a functioning, fast and responsive touchpad. My config looks like this:

Section "InputClass"
Identifier "touchpad catchall"
Driver "synaptics"
MatchIsTouchpad "on"
MatchDevicePath "/dev/input/event*"
Option "TapButton1" "1"
Option "TapButton2" "2"
Option "TapButton3" "3"
Option "VertEdgeScroll" "on"
Option "VertTwoFingerScroll" "on"
Option "VertResolution" "100"
Option "HorizResolution" "100"
Option "MaxDoubleTapTime" "100"
EndSection

USB automount

I looked for a solution for simply (preferably automatically) mounting any USB storages (pendrives, external hard drives, etc.) on my Arch Linux notebook. Gnome and other desktop environments do it for you automatically but if you keep things simple with a small window manager (like Awesome or wmfs) you need to set it up by yourself.

There are several solutions for doing so:

1. mounting USB storages manually or with an fstab entry:
You can do it with some basic Linux knowledge, the method is well documented here. It is simple to set up but not flexible enough (i.e. you have to mount the devices manually, it is hard to handle several storages at the same time, fstab can automount only at boot time).

2. using an external application to handle your usb devices: 
pmount,  skvm, udiskie etc. can more or less handle your USB volumes but I  found none of them satisfactory. One of them have dependency problems (i.e. using hal that I did not want to install), others need consolekit or to be run as root, and all of them needs an extra daemon to be run in the background.

3. using udev rules to recognise and automount your volumes:
The udev daemon natively runs in the background, you do not have to install and/or place it in your /etc/rc.conf. You only need to tell it to mount your USB storages when they are plugged in. You can find nice examples for such rules in the udev Arch Wiki.

I use the following udev rule that mounts automatically the hot-plugged device under /media/<label> and works seamlessly with vfat, ntfs and other formats of USB storages. Simply create this file and fill it up with the contents as follows:

/etc/udev/rules.d/10-my-media-automount.rules

# vim:enc=utf-8:nu:ai:si:et:ts=4:sw=4:ft=udevrules:
#
# /etc/udev/rules.d/10-my-media-automount.rules

# start at sdb to ignore the system hard drive
KERNEL!="sd[b-z]*", GOTO="my_media_automount_end"
ACTION=="add", PROGRAM!="/sbin/blkid %N", GOTO="my_media_automount_end"

# import some useful filesystem info as variables
IMPORT{program}="/sbin/blkid -o udev -p %N"

# get the label if present, otherwise assign one based on device/partition
ENV{ID_FS_LABEL}!="", ENV{dir_name}="%E{ID_FS_LABEL}"
ENV{ID_FS_LABEL}=="", ENV{dir_name}="usbhd-%k"

# create the dir in /media and symlink it to /mnt
ACTION=="add", RUN+="/bin/mkdir -p '/media/%E{dir_name}'"

# global mount options
ACTION=="add", ENV{mount_options}="relatime"
# filesystem-specific mount options (777/666 dir/file perms for ntfs/vfat)
ACTION=="add", ENV{ID_FS_TYPE}=="vfat|ntfs", ENV{mount_options}="$env{mount_options},gid=100,dmask=000,fmask=111,utf8"

# automount ntfs filesystems using ntfs-3g driver
ACTION=="add", ENV{ID_FS_TYPE}=="ntfs", RUN+="/bin/mount -t ntfs-3g -o %E{mount_options} /dev/%k '/media/%E{dir_name}'"
# automount all other filesystems
ACTION=="add", ENV{ID_FS_TYPE}!="ntfs", RUN+="/bin/mount -t auto -o %E{mount_options} /dev/%k '/media/%E{dir_name}'"

# clean up after device removal
ACTION=="remove", ENV{dir_name}!="", RUN+="/bin/umount -l '/media/%E{dir_name}'", RUN+="/bin/rmdir '/media/%E{dir_name}'"

# exit
LABEL="my_media_automount_end"

Wicd cannot connect to ad-hoc networks

In my Nokia (5800 Xpress music) phone I have a network sharing application called Joikuspot. Its purpose is to use your phone as a wireless access point offering your mobile data plan via wi-fi to other devices (such as my laptop with Arch Linux). The provided wlan network is in ad-hoc mode though and my Wicd client cannot connect to it. dmesg says: "Unable to find TIM Element in beacon". After some research I found this bug, some says that it is solved in the newest version of wicd but for me upgrading to 1.7.0 did not solve the issue.

Fortunately the ad-hoc network works fine with manual configuration. To bring down the wlan0 interface and restart it in ad-hoc mode without passphrase (as my phone app does not support protected networking) and obtaining an IP address with dhclient (dhcpcd might work as well) simply use this command:

ifconfig wlan0 down && iwconfig wlan0 mode Ad-Hoc && iwconfig wlan0 essid <ESSID_of_your_wlan> key off && ifconfig wlan0 up && dhclient wlan0

As a workaround for wicd I created a preconnect script (more about wicd scripting: here) that runs this command and does not let wicd to destroy my ad-hoc connection (because it wants to do it) until another connection is set manually in the wicd client. I used a loop that does not let the script end while a lock file exists. Choosing another connection in wicd deletes the lock file, the ad-hoc script ends and gives back the control to wicd.

The script lies under /etc/wicd/scripts/preconnect/ad-hoc.sh

#!/bin/bash

script="$(basename "$0")"
script_name="${script/.sh/}"

echo $1 $2 $3 $4 $5 >> "/var/log/wicd/${script_name}.log"

echo "Running ${script}" >"/var/log/wicd/${script_name}.log"
exec 2>>"/var/log/wicd/${script_name}.log"
exec 1>&2

connection_type="$1"
echo "Connection type: ${connection_type}"

if [ "${connection_type}" == "wired" ]; then
        profile="$3"
        echo "Profile: ${profile}"
elif [ "${connection_type}" == "wireless" ]; then
        essid="$2"
        bssid="$3"
        echo "ESSID: ${essid}"
        echo "BSSID: ${bssid}"
else
        echo "Unknown connection type: ${connection_type}" >&2
        exit
fi

rm /var/lock/wicd_adhoc.lock

case $2 in
*JoikuSpot*)
  # When I connect to my phone my ESSID contains JoikuSpot
     touch /var/lock/wicd_adhoc.lock
     ifconfig wlan0 down && iwconfig wlan0 mode Ad-Hoc && iwconfig wlan0 essid $2 key off && ifconfig wlan0 up && dhclient wlan0

     while [ -f /var/lock/wicd_adhoc.lock ]
     do
     sleep 2;
     done

esac


Waiting for the real bugfix in the newer releases of wicd!