Donnerstag, 5. März 2015

FPV using RaspberryPI

Update 2015/07/06
in the meantime i have figured out that 5GHz WLAN is not allowed between Aircrafts and ground stations in Germany.
In the general admission for 5GHz there is a small sidenote saying that this is prohibited.
I was not aware of this fact before and therefore will not use it on any of my RC Aircrafts.


Since i like flying QuadCopters as well as playing around with RaspberryPi an idea came into my mind soon to build a FPV - First Person View with a RaspberryPI.
Wikipedia: First-person view (radio control) (aka "video piloting"), a method of piloting radio-controlled vehicles using a wireless video camera in the vehicle.

The requirements i had at start of investigation

  • Wireless transmission needs to be on 5GHz to do not interfere with the copters radio control operating on 2.4GHz.
  • Live video view should be possible on any mobile device (that supports 5GHz W-LAN) without need for additional hardware.
  • GPS information (position, height, speed, direction) should be available in the video.
  • Video should be stored on a USB-stick with a reasonable quality.
  • The Raspberry Pi should act as Wireless AP to minimize hardware.
View from the Quad-copter (with the B+ Model and raw GPS parsing)
  
After doing some investigation with a Raspberry Pi Model B+ as well as some trial and error with parsing the raw GPS data...





RCeye one eXtreme as test platform for the Raspberry Pi 2 B






... i came up with the final solution as described below.
With this i'm able to record mpeg video files with ~15Frames/sec at a resolution of 640x480 beside having a live view.


The final result


Warning - to enjoy the live view on a computer / mobile you need 5GHz WLAN (802.11a) available on this device as well.
 

Hardware

..and finally something to lift that off ;-) ..Copter, Balloon,..
Raspberry Pi 2 Model B, Raspberry Pi camera, BEC voltage regulator
TL-WDN3200 WLAN, NL-601US GPS and 64GB USB stick

  

Software

On the Raspberry Pi 2 i use ArchLinuxARM as the preferred Linux OS.
  • Motion - web cam software that allows text overlay.
  • GPSD - a GPS service daemon. Autputs GPS data as raw NMEA or JSON format.
  • jshon - Jshon parses, reads and creates JSON from within the unix shell.
  • selfmade bash script to extract the GPS data from GPSD and pass it to Motion.
  • Hostapd - provides the functionality to act as software WLAN AccesPoint.
  • bc - allows floating point calculation via bash.
  • wget - for updating the text overlay data.
  • optional: lighttpd - lightweight web server to allow the video download from the PI 

    

Software Installation

We start with the preparation of the microSD card for running ArchLinuxARM on the Pi.
This is described on ArchLinuxARM on the tab Installation (you need to have a Linux machine available for this).
After you are done with this plug in the SDcard into the PI, connect the PI with the Ethernet port to a network having internet access and power it up.

Connect a monitor and Keyboard or log in as root via ssh (Initially the Pi has the network name 'alarmpi'. The default root password is 'root').
Remark: mine did not show up on the network the first time since dhcpcd was not enabled by default for eth0. To do this you need to log in and issue the following commands:
systemctl enable dhcpcd@eth0
systemctl start dhcpcd@eth0
The first thing you should always do is to set a new password. This is done with passwd.

I have described in previous blog post on how to install gpsd, wireless AP and Motion. Please use the instructions below to do the basic installation for this pieces of software.
After having Motion, GPSD and the Wireless AP running we do the following steps.
1) reconfigure Hostapd to run on 5GHz.
2) adapt motion to do the video recording as needed.
3) install and run gpsd on the Raspberry Pi.
4) implement the script(s) for the GPS data overlay on the video / live stream.
5) tweak configuration to auto mount the USB-stick.
6) optional: install web service to download the stored video files via web browser from the PI.
  

1) reconfigure Hostapd to run on 5GHz

Edit the hostapd config file

/etc/hostapd/hostapd.conf


Content of hostapd.conf - adjustments bold

#channel=11
channel=40
# some laptops will only connect if the mode is set to b
#hw_mode=b
#hw_mode=g
hw_mode=a
country_code=DE

wmm_enabled=1

#ieee80211n=1
ieee80211d=1
# for 5Ghz automatic channel selection
#ieee80211h=1
# required for full speed
ht_capab=[HT40+][SHORT-GI-20][SHORT-GI-40]


   
Remarks
For having Hostapd finally working on 5GHz i had to install the crda - Central Regulatory Domain Agent for wireless networks package and configure the correct country.
Install crda and all dependencies with

pacman -S crda
  
Enable the correct country (by un-commenting it) in /etc/conf.d/wireless-regdom
  
e.g.
#WIRELESS_REGDOM="CY"
#WIRELESS_REGDOM="CZ"
WIRELESS_REGDOM="DE"
#WIRELESS_REGDOM="DK"
#WIRELESS_REGDOM="DM"
#WIRELESS_REGDOM="DO"

  
Reboot the Raspberry Pi to get WLAN working on 5GHz (assuming that you have enabled WLAN already before while following the instructions on Wireless AP on RaspberryPi / ArchLinux).
 
2) adapt motion to do the video recording as needed
Assuming you have done the installation according RaspberryPI Webcam on Archlinux already before.

changes on /etc/motion/motion.conf

# Image width (pixels). Valid range: Camera dependent, default: 352
#width 1280
width 640

# Image height (pixels). Valid range: Camera dependent, default: 288
#height 720
height 480

# Maximum number of frames to be captured per second.
# Valid range: 2-100. Default: 100 (almost no limit).
framerate 15
low_cpu on

......

# Threshold for number of changed pixels in an image that
# triggers motion detection (default: 1500)
threshold 500
.......
# Picture frames must contain motion at least the specified number of frames
# in a row before they are detected as true motion. At the default of 1, all
# motion is detected. Valid range: 1 to thousands, recommended 1-5
minimum_motion_frames 1
 
.......

# Specifies the number of pre-captured (buffered) pictures from before motion
# was detected that will be output at motion detection.
# Recommended range: 0 to 5 (default: 0)
# Do not use large values! Large values will cause Motion to skip video frames and
# cause unsmooth mpegs. To smooth mpegs use larger values of post_capture instead.
pre_capture 0

# Number of frames to capture after motion is no longer detected (default: 0)
post_capture 0

# Gap is the seconds of no motion detection that triggers the end of an event
# An event is defined as a series of motion images taken within a short timeframe.
# Recommended value is 60 seconds (Default). The value 0 is allowed and disables
# events causing all Motion to be written to one single mpeg file and no pre_capture.
gap 60

# Maximum length in seconds of an mpeg movie
# When value is exceeded a new mpeg file is created. (Default: 0 = infinite)
max_mpeg_time 0

# Always save images even if there was no motion (default: off)
output_all off
# this could be set to "on" but then recording of video starts when the system is powered on. I prefer to record only when i'm flying.
........

# Output 'normal' pictures when motion is detected (default: on)
# Valid values: on, off, first, best, center
# When set to 'first', only the first picture of an event is saved.
# Picture with most motion of an event is saved when set to 'best'.
# Picture with motion nearest center of picture is saved when set to 'center'.
# Can be used as preview shot for the corresponding movie.
#output_normal on
output_normal first
.......

# Bitrate to be used by the ffmpeg encoder (default: 400000)
# This option is ignored if ffmpeg_variable_bitrate is not 0 (disabled)
ffmpeg_bps 400000

# Enables and defines variable bitrate for the ffmpeg encoder.
# ffmpeg_bps is ignored if variable bitrate is enabled.
# Valid values: 0 (default) = fixed bitrate defined by ffmpeg_bps,
# or the range 2 - 31 where 2 means best quality and 31 is worst.
ffmpeg_variable_bitrate 17

# Codec to used by ffmpeg for the video compression.
# Timelapse mpegs are always made in mpeg1 format independent from this option.
# Supported formats are: mpeg1 (ffmpeg-0.4.8 only), mpeg4 (default), and msmpeg4.
# mpeg1 - gives you files with extension .mpg
# mpeg4 or msmpeg4 - gives you files with extension .avi
# msmpeg4 is recommended for use with Windows Media Player because
# it requires no installation of codec on the Windows client.
# swf - gives you a flash film with extension .swf
# flv - gives you a flash video with extension .flv
# ffv1 - FF video codec 1 for Lossless Encoding ( experimental )
# mov - QuickTime ( testing )
ffmpeg_video_codec mpeg4
.........
# Draw characters at twice normal size on images. (default: off)
text_double on

.........

# Target base directory for pictures and films
# Recommended to use absolute path. (Default: current working directory)
target_dir /srv/http
...........

# Maximum framerate for webcam streams (default: 1)
webcam_maxrate 1

# u might use higher frame rates for the livestream but this might impact quality of video recording.
...........

############################################################
# HTTP Based Control
############################################################

# TCP/IP port for the http server to listen on (default: 0 = disabled)
control_port 8080

# Restrict control connections to localhost only (default: on)
control_localhost on

# Output for http server, select off to choose raw text plain (default: on)
control_html_output on


The settings above are the ones i used for testing. But this might not be the optimum.
You should play around with frame rates, resolution and other parameters to get the most out of your configuration.
The HTTP Based Control related settings are needed to enable the text overlay functionality via the scripts further down.

3) install and run gpsd on the Raspberry Pi. 
  

    According gpsd on archlinux.
   
4) implement the script(s) for the GPS data overlay on the video / live stream.
For this i have two scripts
  • gpsparser.sh connects to GPSD via gpspipe and converts the JSON string with jshon into the format we need for the motion video overlay. The output is stored in a temporary file /tmp/gps2motion.txt.
  • gps2motion.sh reads out /tmp/gps2motion.txt and sets the text overlay on the video stream once a second.
The reason for separating this job is video recording performance impact. If the text overlay is updated more frequently i have seen video being delayed in some cases.
 
Create the file /usr/local/bin/gps2motion.sh
 
with following content

#!/bin/sh
#
GPS2MOTION_PID=/var/run/motion/gps2motion.pid
GPSDATATEMP=/tmp/gps2motion.txt
# create own pis file
touch $GPS2MOTION_PID
# Set on Camera Image0
# check if i'm already running
#if [ -e $GPS2MOTION_PID ]; then
#    echo "gps2motion.sh already running"
#else
    while true; do
        if [ -e $GPSDATATEMP ]; then
            GPS2MOTION=`cat $GPSDATATEMP`
            wget -q --delete-after "http://localhost:8080/0/config/set?text_left=$GPS2MOTION"
        fi
        sleep 1
    done
#fi
#remove own pid file
rm -r $GPS2MOTION_PID

# END of FILE
  
Create the file /usr/local/bin/gpsparser.sh
  
with following content
  
#!/bin/sh
#
GPS_CLASS=\"VTG\"
GPS_TAG=\"0x0106\"
GPS_VALID=FALSE
GPSDATATEMP=/tmp/gps2motion.txt
# limiting the digits with bc only works on divisions therefore we need to do a division by 1.
# extraction of data elemts from the JSON data string is done with jshon.
# e.g. (echo $JSON_STRING | jshon -e alt) to extract altitute.
#-------------------------------------------------------------
# set initial blank values on all variables
GPS_LAT=""
GPS_LON=""
GPS_NoS=""
GPS_EoW=""
GPS_ALT=""
GPS_DIR=""
GPS_SPEED=""
GPS_CLIMB=""
#-------------------------------------------------------------
#
# wait x sec and connect to gpsd via gpspipe and parse data
#
sleep 1
gpspipe -w | while read -r JSON_STRING; do
    JSON_CLASS=$(echo $JSON_STRING | jshon -e class)
    JSON_TAG=$(echo $JSON_STRING | jshon -e tag)
        FIELD_UPDATE=FALSE
        # parse the $GPRMC sentence
        if [ $JSON_TAG = $GPS_TAG ]; then
        GPS_MODE=$(echo $JSON_STRING | jshon -e mode)
                # check if GPS is valid
        # based on the mode value set the FIX information
                # extract the time from JSON string
                GPS_TIME=$(echo $JSON_STRING | jshon -e time)
                # strip off the date and reduce time
                GPS_DATE=${GPS_TIME:1:10}
                GPS_TIME=${GPS_TIME:12:8}
        # check if GPS data are valid and extract details if so
                if [ $GPS_MODE > 1 ]; then
                        GPS_STATUS="GPS "$GPS_MODE"D FIX"
                    # extract the latitude from JSON string and limit to 6 digit
                    GPS_LAT=$(echo "scale=6;$(echo $JSON_STRING | jshon -e lat)/1" | bc)
                       if [ $GPS_LAT > 0 ]; then
                            GPS_NoS="N"
                    else
                            GPS_NoS="S"
                    fi
                    # extract the longitude from JSON string and limit to 6 digit
                    GPS_LON=$(echo "scale=6;$(echo $JSON_STRING | jshon -e lon)/1" | bc)
                    if [ $GPS_LAT > 0 ]; then
                            GPS_EoW="E"
                    else
                            GPS_EoW="W"
                    fi
                    # extract the ALTitute from JSON string and limit to 2 digit
                    GPS_ALT=$(echo "scale=2;$(echo $JSON_STRING | jshon -e alt)/1" | bc)
                       # extract the heading from JSON string and limit to 1 digit
                    GPS_DIR=$(echo "scale=1;$(echo $JSON_STRING | jshon -e track)/1" | bc)
                    # extract SPEED from JSON string. Speed is expressed in m/s.
                    # convert the SPEED from m/s into km/h and limit to 1 digit
                    GPS_SPEED=$(echo "scale=1;$(echo $JSON_STRING | jshon -e speed)*3.6/1" | bc)
                    # extract CLIMB from JSON string and
                    # limit CLIMB to 1 digit
                    GPS_CLIMB=$(echo "scale=2;$(echo $JSON_STRING | jshon -e climb)/1" | bc)
                    # extract the errors with precision of one digit after decimal - details at the end of the file with
                    GPS_ERR_LAT=$(echo "scale=2;$(echo $JSON_STRING | jshon -e epy)/1" | bc)
                    GPS_ERR_LON=$(echo "scale=2;$(echo $JSON_STRING | jshon -e epx)/1" | bc)
                    GPS_ERR_ALT=$(echo "scale=2;$(echo $JSON_STRING | jshon -e epv)/1" | bc)
                else
                        GPS_STATUS="NO GPS FIX"
                fi
                DATA_STRING=$GPS_STATUS"\nDate "$GPS_DATE"\nTime "$GPS_TIME"\nLAT    "$GPS_LAT" "$GPS_NoS" err "$GPS_ERR_LAT"m\nLON     "$GPS_LON" "$GPS_EoW" err "$GPS_ERR_LON"m\nALT     "$GPS_ALT"mASL err "$GPS_ERR_ALT"m\nDIR   "$GPS_DIR"deg (true)\nSPEED "$GPS_SPEED"km/h\nCLIMB "$GPS_CLIMB"m/s"
        echo $DATA_STRING > $GPSDATATEMP
     fi
done

# JSON string example below
# {"class":"TPV","tag":"0x0106","device":"/dev/ttyACM0","mode":3,"time":"2015-02-18T05:21:08.000Z","ept":0.005,"lat":53.122665202,"lon":9.235810102,"alt":-7.843,"epx":232.834,"epy":644.583,"epv":39.560,"track":37.2745,"speed":0.046,"climb":-0.027,"eps":0.91,"epc":0.00}
# GPSD JSON description is available on http://www.catb.org/gpsd/gpsd_json.html

#
# END of FILE
 

Make both files executable by adding the executable flag.

chmod +
x /usr/local/bin/gps2motion.sh
chmod +x /usr/local/bin/gpsparser.sh

To start the text overlay automatically at boot we need to create systemd service files and enable them.
Systemd service file for gps2motion - /etc/systemd/system/gps2motion.service

with content

[Unit]
Description=GPSparsing script
After=local-fs.target

[Service]
ExecStart=/usr/local/bin/gps2motion.sh
Type=simple
#StandardOutput=null
StandardError=null

[Install]
WantedBy=multi-user.target
 

    
And systemd service file for gpsparser - /etc/systemd/system/gpsparse.service
 
with content
  
[Unit]
Description=GPSparsing script
After=local-fs.target

[Service]
ExecStart=/usr/local/bin/gpsparser.sh
Type=simple
#StandardOutput=null
StandardError=null

[Install]
WantedBy=multi-user.target

   
Enable both services at boot with 
  
systemctl enable gps2motion.service
systemctl enable gpsparse.service


To make all scripts working correctly we need to install some additional software

pacman -S bc 
pacman -S jshon
pacman -S wget

  
5) tweak configuration to auto mount the USB-stick

For this we add the following line (bold) to /etc/fstab

#
# /etc/fstab: static file system information
#
# <file system>    <dir>    <type>    <options>    <dump>    <pass>
/dev/mmcblk0p1  /boot   vfat    defaults        0       0
/dev/sda1    /srv/http vfat    defaults,noatime,nofail        0    0

#

With this the USB storage device (needs to be formatted as FAT32) is mounted at boot automatically as
/srv/http. All videos are then saved on the USB device.
The flag
nofail allows also the absence of the USB device with the consequence that videos will be stored on the SDcard folder /srv/http.If you need access to the videos in that case you need to install the web service as mentioned in step 6. 
6) optional: install web service to download the stored video files via web browser from the PI

Install the lighttpd web server with pacman -S lighttpd

Change the configuration file /etc/lighttpd/lighttpd according settings below.
  • Server port must be changed to 81 since motion is running already on port 80.
  • mimetype for avi needs to be added.
# This is a minimal example config
# See /usr/share/doc/lighttpd
# and http://redmine.lighttpd.net/projects/lighttpd/wiki/Docs:ConfigurationOptions

server.port        = 81
server.username        = "http"
server.groupname    = "http"
server.document-root    = "/srv/http"
server.errorlog        = "/var/log/lighttpd/error.log"
dir-listing.activate    = "enable"
index-file.names    = ( "index.html" )
mimetype.assign        = (
                ".html" => "text/html",
                ".txt" => "text/plain",
                ".css" => "text/css",
                ".js" => "application/x-javascript",
                ".jpg" => "image/jpeg",
                ".jpeg" => "image/jpeg",
                ".gif" => "image/gif",
                ".png" => "image/png",
                ".avi" => "video/avi",
                "" => "application/octet-stream"

   
Finally enable the start of lighttpd at boot with the following command
  
systemctl enable lighttpd
  
You're done!

Reboot the Raspberry Pi, connect to it's WLAN on 5GHz to enjoy FPV.
  
If you connect via the webbrowser to http://[name of the pi] you should see the live video with the text overlay.

If you connect
via the webbrowser to http://[name of the pi]:81 you should see / be able to download the stored video.
Have fun!

Gerald
- dk7xe -

Picture Gallery










  An finally a video from one of the flights (before i have figured out the 5GHz WLAN restriction).