Leap motion tracker

This is a collection of notes on the Leap motion tracker and its usage in python 3.6 on linux. It was originally intended to be used in the MR-Keyboard project but wasn't used there due its poor performance in this use case.

We had the following problem with the builtin hand detection/tracking:

• it does not like very reflective (in infrared) surfaces behind the tracked objects
• it does not like other objects very close to the tracked object (like a hand touching a keyboard)
• it does only work reliable in the right orientations (upwards or looking away from the user like on vr goggles)

Installation of the LeapMotion SDK

If you don't want to use motion tracking then you may skip this section and disable motion tracking either in the settings file or with the corresponding CLI flag.

This was written (and tested) on the 22.12.2017. This is intended to run on an Ubuntu/Debian based system but it may work on other systems with some tweaks (mainly because of the different package managers).

To use Leap we need to have the SDK installed and have an interface to it that is compatible to python3.6.

if you already have a compatible lib/interface (this will likely be in the lib folder):

Just install the SDK from https://leapmotion.app.box.com/s/rlwzvt680i5r6zes75ge36ub2yt7ksx3 (or https://developer.leapmotion.com/sdk/v2 if the first one is offline) or use the one in related files (from the hotfix). You're done here, skip the rest of this section.

if you don't have a compatible lib:

Install the Leap SDK with the following steps (based on this Guide):

• Download the official SDK from https://developer.leapmotion.com/sdk/v2 (you may need an account) or use the from the "related_files" folder
• Download the hotfix for the SDK from the the file above from https://leapmotion.app.box.com/s/rlwzvt680i5r6zes75ge36ub2yt7ksx3 or use the one from "related_files"
• unpack both of them in the same folder. You should have two folders whose names start with "LeapDeveloperKit"
• open a console, navigate to the folder where you unpacked the files and run the following lines (use "related_files/Leap.i_remove_….diff" if the url in the wget line is offline). The last line just verifies the functionality of the motion tracker and therefore optional but recommended.

cd LeapDeveloperKit*33747*
sudo dpkg -i Leap*x64.deb
cd ../LeapDeveloperKit*31549*/LeapSDK
mkdir python3.6_project
cp -a lib/x64/libLeap.so lib/Leap.py samples/Sample.py python3.6_project/
sudo apt-get install swig g++ libpython3.6-dev
2to3-3.6 -nw python3.6_project/Sample.py
wget http://tinyurl.com/leap-i-patch -O Leap.i.diff
patch -p0 < Leap.i.diff
swig -c++ -python -o /tmp/LeapPython.cpp -interface LeapPython include/Leap.i
g++ -fPIC -I/usr/include/python3.6m -I./include /tmp/LeapPython.cpp lib/x64/libLeap.so -shared -o python3.6_project/LeapPython.so
cd python3.6-project/
LD_PRELOAD=./libLeap.so python3.6 Sample.py

• copy "Leap.py", "LeapPython.so" and "libLeap.so" from the "python3.6_project" folder to the "lib" folder of the program.

Configuration

Start the Leap Motion control panel with LeapControlPanel