Tuesday, September 10, 2013

OpenWRT based Video Recording System - Part 2 - Software setup for OpenWRT

Fromatting SD card with EXT2

In order to use the SD card with OpenWRT it must be formatted with the EXT2 filesystem. For flash based storage devices EXT2 is preferred as it reduces the amount of writes performed, as it is not a journaled filesystem. Flash devices have a limited number of write cycles, and while this is limit is very high (millions or more) reducing the number of writes will increase the lifespan of the device. A non journaled filesystem essentially means that it does not track changes to files, which results in less writes performed.

These are the steps to format an SD card using Ubuntu:
  1. Check to see where SD card is mounted.

    root@ubuntu:~# mount
    ...
    /dev/sdb1 on /media/sean/9016-4EF8 type vfat
     
  2. Unmount the SD card.

    root@ubuntu:~# umount /dev/sdb1
     
  3. Run fdisk to remove existing partition and create new partition for EXT2.

    root@ubuntu:~# fdisk /dev/sdb

    Command (m for help): d
    Selected partition 1

    Command (m for help): n

    Partition type:
       p   primary (0 primary, 0 extended, 4 free)
       e   extended
    Select (default p): p
    Partition number (1-4, default 1): 1
    First sector (2048-15407103, default 2048):
    Using default value 2048
    Last sector, +sectors or +size{K,M,G} (2048-15407103, default 15407103):
    Using default value 15407103

    Command (m for help): p

    Disk /dev/sdb: 7888 MB, 7888437248 bytes
    13 heads, 13 sectors/track, 91166 cylinders, total 15407104 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00000000

       Device Boot      Start         End      Blocks   Id  System
    /dev/sdb1            2048    15407103     7702528   83  Linux

    Command (m for help): w
    The partition table has been altered!

    Calling ioctl() to re-read partition table.
    Syncing disks.
     
  4. Finally create the EXT2 file-system on the new partition.

    root@ubuntu:~# mkfs.ext2 /dev/sdb1
    ...
    Allocating group tables: done                          
    Writing inode tables: done                          
    Writing superblocks and filesystem accounting information: done
     
The SD Card is now ready for use in OpenWRT. The next steps are to install OpenWRT on the router, and then transfer the OpenWRT filesystem from the internal flash memory to the SD card. Finally we can install the packages needed to capture video from the USB webcam.

OpenWRT configuration on MR3020

The firmware used on the MR3020 can be obtained from:

MR3020 OpenWRT 12.09 r36088 for sysupgrade
MR3020 OpenWRT 12.09 r36088 for factory upgrade

The first link is the firmware image for a sysupgrade using OpenWRT, and the second link is the firmware image that can be used with the factory firmware update.

The following steps will configure OpenWRT to use the SD card as the overlay. The overlay is where any changes to the file system are stored, as the original file system is read only.
  1. Update opkg repositories.
     
    opkg update
     
  2. Install the kmod-usb-storage, kmod-fs-ext4 and block-mount packages to enabled usb flash drive support.
     
    opkg install kmod-usb-storage kmod-fs-ext4 block-mount

     
  3. Create a directory to mount the USB drive on, and mount it.yes
    mkdir /mnt/sda1
    mount /dev/sda1 /mnt/sda1

     
  4. Copy the contents of the existing overlay to the flash drive. 
         
    tar -C /overlay -cvf - . | tar -C /mnt/sda1 -xf -

     
  5. Edit the file /etc/config/fstab and add the following.

    config mount
            option target   /overlay
            option device   /dev/sda1
            option fstype   ext2
            option options  rw,sync
            option enabled  1
            option enabled_fsck 0
     
  6. Reboot.
  7. Check to see if it has mounted correctly by looking at the free space on the root.
     
    OpenWrt:~# df -h
    Filesystem                Size      Used Available Use% Mounted on
    rootfs                    7.2G      3.3M      6.8G   0% /
    /dev/root                 2.0M      2.0M         0 100% /rom
    tmpfs                    14.3M     72.0K     14.2M   0% /tmp
    tmpfs                   512.0K         0    512.0K   0% /dev
    /dev/sda1                 7.2G      3.3M      6.8G   0% /overlay
    overlayfs:/overlay        7.2G      3.3M      6.8G   0% /
     
Creating a swap file increases the stability of the system as capturing video is memory intensive and may exhaust the limited RAM. Also log files may become large if the system isn't rebooted for long periods, they are stored in a temporary directory which is in RAM. The following steps setup a swap file.
  1. Create the swap file using dd, the count is the number of kilobyes as the blocksize (bs) is a kilobyte (1024 bytes). For a count of 65536 a 64 MB swap will be created. Note this will take some time.

    dd if=/dev/zero of=/swapfile bs=1024 count=65536
     
  2. The file needs to be formatted as a swap file using mkswap.

    mkswap /swapfile
     
  3. The swapfile needs to be mounted by the fstab, so edit /etc/config/fstab and add the following.

    config swap
            option device   /swapfile
            option enabled  1

     
  4. Restart fstab to mount and enable the swap file.

    /etc/init.d/fstab restart
     
  5. Check the swap is working using free. You should see 65532 under the total column for the Swap row.

    # free
                 total         used         free       shared      buffers
    Mem:         29212        27856         1356            0         1216
    -/+ buffers:              26640         2572
    Swap:        65532            0        65532

     
The USB flash drive has now been configured as the overlay and with a swapfile.

Logitech C170 Webcam configuration for OpenWRT


The next stage is installing packages and configuring the Webcam, which is as following.
  1. Install the packages ffmpeg, and kmod-video-uvc, the Logitech C170 webcam uses the usb video class (uvc) drivers and ffmpeg is used to encode videos.
         
    opkg update
    opkg install ffmpeg kmod-video-uvc

     
  2. Plug the C170 in and check that the drivers work by looking at the dev directory, it should contain video0.
  3. "Motion" is a software package that will capture videos or images when motion has been detected from a video source (the webcam in our case). The standard motion package that was compiled for OpenWRT does not have ffmpeg support, so I compiled a custom version. This version can be obtained here. To install it on the MR3020 do the following.
         

    wget http://228899seankelly.googlecode.com/svn/trunk/ar71xx/motion/motion_20120605-224837-2_ar71xx.ipk
    opkg install motion_20120605-224837-2_ar71xx.ipk

     
  4. Motion has a configuration file in /etc/motion.conf, to capture video from the C170 webcam the following options were set.

    The pixel format of the C170 webcam is MJPEG which corresponds to a value of 8 for the v4l2_palette option:

    v4l2_palette 8

    The size of the image to be captured is 320x240 so this is given by width and height. 320x240 lowers the strain of motion detection.

    width 320
    height 240


    Decreasing the frame rate to less than 5 fps will also reduce the processing required. For my setup I found 3 fps was adequate to detect and record motion, I set it using the following.

    framerate 3

    To detect motion a threshold is required, this is the number of pixels that must change in the picture for motion to be present. For the 320x240 image size and a wide angle camera about 500 to 1000 pixels should be adequate. To set the number of pixels the threshold is set as following.

    threshold 1000

    De-speckling of the motion to produce better defined bounds for the motion can be turned off to further reduce the processing required. To turn of the filters that do this comment out the line below using ";"

    ;despeckle_filter EedDl

    In order to filter out some noise the minimum number of consecutive frames with motion can be set. This stops random noise from cause a motion event. The minimum number of frames is set as following, I used 2 frames as at 3 fps this is 0.6 of second of motion is required to start a motion event.

    minimum_motion_frames 2

    In order to create better video captures a number of frames after and before the motion event can be saved as well to create a more fluid video. The pre_capture and post_capture variables specify the number of frames to capture before and after and event. I set both to 2 frames, as following.

    pre_capture 2
    post_capture 2


    The goal is to use ffmpeg to encode videos so we need to disable image output by setting the following.

    output_pictures off

    The default bitrate for ffmpeg is high for lower resolutions so lowering it to between 200000 bps to 300000 bps should be ok, to do this change the following.

    ffmpeg_bps 200000

    To enable playback in a browser, ffmpeg must encode videos in a compatible format such as a flash swf. To use the swf format set the following.

    ffmpeg_video_codec swf

    The last setting is the directory to store the videos in, to give access to the videos from the inbuilt http server they can be stored in the /www folder using the variable as following.

    target_dir /www/cam
This completes the setup for motion to capture video from the C170 webcam. The files will be output to to the cam directory in the uhttp deamons www directory, which gives access to them from the web. 

Friday, September 6, 2013

OpenWRT based Video Recording System - Part 1 - Hardware

An OpenWRT router can be used to capture video with a usb webcam in order to record movements in a room. The internal flash storage for the MR3020 is only 4 Mb which isn't sufficient for storing video. So additional storage is required, which can be provided by a USB flash device. However the MR3020 Router only has a single USB port, so it needs to be expanded with a usb hub. The usb hub in the images below is small enough to fit inside the MR3020, and has a built in micro SD card reader. The SD card reader is essentially a usb SD card reader connected to the 4th USB port of the hub. Removing the casing and de-soldering the usb jacks are the first steps.


The USB port on the MR3020 needs to be removed to connect the USB hub. Short small gauge wires are suitable for connecting the hub. The ground wire needs to have a short amount of wire exposed above the board, the reason for this will become clear later. The USB port can be put back onto the MR3020 board but first the power and data lines must be connected to the pins. This will restore the functionality of the USB port however it will be internal connected to the USB hub so it adds the benefit of an internal SD carder reader.



The USB port can be reattached using only the mounting tabs, and the pins that go into the board can be bent outwards. This allows the wires attached to them to be routed along the board and attached to a free port on the USB hub. The ground pin of the USB port can be directly attached the short length of ground wire that was exposed from earlier. This saves having to run an additional line. The power wire should be run back to the hub in case the hub has the capability to shut off power to the ports.


This completes the required modifications to the interal hardware of the MR3020 router. An SD card can now be used to store videos, but OpenWRT needs packages to be installed and configuration to use the SD card reader and webcam, details will be given in part 2.

The logitech C170 webcam has a very narrow field of view, probably around 45 degrees. This is problematic when trying to monitor a room as some of the room is outside the field of view of the webcam. To increase the viewing angle of the camera lenses designed for smart phones can be used. These come with a metallic ring that a magnet on the lense sticks to in order to hold the lense in place. This can be used on the logitech webcam to hold these types of lenses in place. The two vertical bezels above and below the sensor need to be cut so that the lenses sit flush.


There are two types of lenses available, a wide angle lense and a fish eye lense. The wide angle lense gives about 120 degrees of viewing angle, while the fish eye gives 180 degrees. The downside of these lenses is that they distort the image.



The fish eye lense is to the right, and the wide angle is to the left.

I installed the two cameras in the lounge and kitchen, which you can see below.

Configuring the MR3020 routers to capture video is here.

Monday, February 25, 2013

Sparkfuns' Open Logic Sniffer

Sparkfun sells a Logic sniffer which is a low cost OLS (Open Logic Sniffer) platform that supports the sump logic analyser software. It uses a FPGA connected to a usb to serial adapater, and this is where the problem lies. The drivers required for the usb to serial adapter do not work with Windows 8 as they are not signed. The following procedure will sign the drivers:

  1. Obtain the driver to sign by downloading the ols software, found here. Either version (expert or full) contains the driver.
     
  2. Modifications to the inf file are needed in order to use the usbser.sys driver provided by windows. Extract the archive and find the driver directory. Locate mchpcdc.inf and copy it to another directory (for example C:\drivers). Edit the copied version and make the following changes:
     
    [DestinationDirs]
    FakeModemCopyFileSection=12 //add
    DefaultDestDir=12
    ...
    [DriverInstall.nt]
    include=mdmcpq.inf
    CopyFiles=DriverCopyFiles.nt  //remove
    CopyFiles=FakeModemCopyFileSection //add
    AddReg=DriverInstall.nt.AddReg
    ...
    [DriverCopyFiles.nt]  //remove
    usbser.sys,,,0x20     //remove

    ...
    [DriverInstall.NTamd64]
    include=mdmcpq.inf
    CopyFiles=DriverCopyFiles.NTamd64 //remove
    CopyFiles=FakeModemCopyFileSection //add
    AddReg=DriverInstall.NTamd64.AddReg
     
     
  3. The next steps sign the driver and require the Windows Driver Kit, found here.
     
  4. If you have Visual Studio 2012, open a Developer Command Prompt as an administrator. Otherwise open a command prompt as an administrator and use the following if you are running 64 bit windows:

    set PATH=PATH;C:\Program Files (x86)\Windows Kits\8.0\bin\x64
      

    or if you are running 32 bit windows:

    set PATH=PATH;C:\Program Files\Windows Kits\8.0\bin\x86
     

    This adds the path of the tools needed for the driver signing. The Developer Command Prompt already has these paths setup.
     
  5. The directory of the inf used in the following steps is in C:\drivers. The first step is to change the date of the driver in the inf using the following:

    C:\drivers>stampinf -f mchpcdc.inf -d 09/09/2012 -v 9.0.9999.0
    Stamping .\mchpcdc.inf [Version] section with
    DriverVer=09/09/2012,9.0.9999.0
     

    This sets the date of the driver so that it can be signed. The date needs to be newer than 21/04/2012 as we cannot sign software created before windows 8.
     
  6. Next create a catalog file for the driver:
     
    C:\drivers>inf2cat /driver:. /os:8_X86,8_X64
    ....................
    Signability test complete.
    Errors:
    None
    Warnings:
    None
    Catalog generation complete.
     
  7. A certificate is needed to sign the driver catalog, to create a certificate use:
     
    C:\drivers>Makecert -r -pe -ss PrivateCertStore -n "CN=TestCertforWDK" TestCert.cer
    Succeeded
     

    This creates the certificate to sign the driver catalog, it must now be added to the trusted certificates on the computer. This is what allows the driver to be installed as the certificate to sign the driver is also part of the trusted certificates on the computer. The following commands add the certificate to the root and to the trusted publishers:
     
    C:\drivers>certmgr.exe /add TestCert.cer /s /r localMachine root
    CertMgr Succeeded
    C:\drivers>certmgr.exe /add TestCert.cer /s /r localMachine trustedpublisher
    CertMgr Succeeded
     
  8. The driver catalog can now be signed with the certificate created in the previous step, using the following:
     
    C:\drivers>Signtool sign /v /s PrivateCertStore /n TestCertForWDK /t http://timestamp.verisign.com/scripts/timestamp.dll mchpcdc.cat
    The following certificate was selected:
        Issued to: TestCertforWDK
        Issued by: TestCertforWDK
        Expires:   Sun Jan 01 12:59:59 2040
        SHA1 hash: ...
    Done Adding Additional Store
    Successfully signed and timestamped: mchpcdc.cat
    Number of files successfully Signed: 1
    Number of warnings: 0
    Number of errors: 0

     
  9. The final step in signing the driver is to verify the driver catalog and associated inf, this is done using:
     
    C:\drivers>Signtool sign /v /s PrivateCertStore /n TestCertForWDK /t http://timestamp.verisign.com/scripts/timestamp.dll mchpcdc.cat
    The following certificate was selected:
        Issued to: TestCertforWDK
        Issued by: TestCertforWDK
        Expires:   Sun Jan 01 12:59:59 2040
        SHA1 hash: ...
    Done Adding Additional Store
    Successfully signed and timestamped: mchpcdc.cat
    Number of files successfully Signed: 1
    Number of warnings: 0
    Number of errors: 0
    C:\drivers>Signtool verify /pa /v /c mchpcdc.cat mchpcdc.inf
    Verifying: mchpcdc.inf
    File is signed in catalog: mchpcdc.cat
    Hash of file (sha1): ...
    Signing Certificate Chain:
        Issued to: TestCertforWDK
        Issued by: TestCertforWDK
        Expires:   Sun Jan 01 12:59:59 2040
        SHA1 hash: ...
    The signature is timestamped: Mon Feb 25 09:07:48 2013
    Timestamp Verified by:
        Issued to: Thawte Timestamping CA
        Issued by: Thawte Timestamping CA
        Expires:   Fri Jan 01 12:59:59 2021
        SHA1 hash: ...
            Issued to: Symantec Time Stamping Services CA - G2
            Issued by: Thawte Timestamping CA
            Expires:   Thu Dec 31 12:59:59 2020
            SHA1 hash: ...
                Issued to: Symantec Time Stamping Services Signer - G4
                Issued by: Symantec Time Stamping Services CA - G2
                Expires:   Wed Dec 30 12:59:59 2020
                SHA1 hash: ...
    Successfully verified: mchpcdc.inf
    Number of files successfully Verified: 1
    Number of warnings: 0
    Number of errors: 0
     
  10. The driver is now signed. Right click on the mchpcdc.inf and click install. Next time the Open Logic sniffer is plugged in it should be detected and the correct driver installed.
This guide was based on these links:

http://www.microchip.com/forums/m488342.aspx

http://msdn.microsoft.com/en-nz/library/windows/hardware/ff547089(v=vs.85).aspx

http://stackoverflow.com/questions/10832499/workaround-to-skip-driver-signing-in-64bit-windows