Vision based Object Tracker with Pan & Tilt USB Camera

This project demonstrates how to do a visual object tracking with a camera, mounted on a pan & tilt assembly. A USB camera mounted on a 2-axis (pan and tilt) servo assembly and connected to the host PC is used to track an object. A micro-controller board is used to control the servos of the pan & tilt assembly based on the control signals received from the host PC.

Hardware Development

· Vega Aries v2.0 micro-controller board is used to drive the servos. However, any micro-controller board supported by CASP can be used.

· Pan & Tilt servos are connected to the PWM pins of micro-controller board as shown in below figure. The microcontroller generates required PWM signals to control the servos of the camera pan & tilt assembly.

· A USB camera connected to the host PC is used to capture live images and send them to the PC. A software that is running on the host PC detects the object and outputs suitable control commands to the micro-controller board to control the servos.

Software Development

Two models are developed to achieve desired objective.

Target Model that runs on the target micro-controller consists of

· Blink logic that indicates the system is running.

· Digital input block that is mapped to the on-board push button (BTN0). It is used to manually reset the camera PID controller logic if the tracking is lost.

· Pan & Tilt servo blocks are mapped to the pins PWM0 and PWM1 of the microcontroller.

· Serial block along with its associated blocks are used to communicate the above signals to the host PC

Following are the steps to properly program the target board.

· Connect the target to the host PC via a USB cable.

· Note the serial port number to which the board is connected to the host PC, from the host operating system.

· Run CASP and load project ‘target_model’.

· Open Home->Simulation->Setup Simulation Parameters menu item. Under TargetHW->General tabs set ‘Target Hardware Programmer Port’ parameter to the serial port to which the board is connected.

· Build the model and program the board by clicking on Run button.

Native Model that runs on Host PC consists of

· Camera block that receives live video from the connected USB camera. If multiple cameras are connected, change the camera index parameter of the camera block to select the desired camera.

· Object tracker block that does the object tracking and outputs the centered X and Y co-ordinates of the tracked object.

· Image viewer block that displays the captured video, overlapped with a bounding rectangle of the tracked object.

· Two number of PID controllers and associated blocks to generate required servo angles to rotate the camera so that the tracked object is always at the center of the captured video.

· GPIO blocks (such as ‘reset’ and ‘servo’ blocks) that maps to the target model via the serial communication channel.

Following are the steps to run the native model on the host PC

· With the target board connected open another instance of CASP and load the ‘native_model’ project.

· Click on Home->Simulation->Configure Simulation IO menu item.

· ‘Configure Simulation Hardware’ window will open. Under Native Nodes change the serial port to which the micro-controller is connected by double clicking on the serial port item as indicated in the below figure.

· Click on ‘Connect Device’ button and check the ‘Online Data’ check box. The program should now communicate with the target with cycle time around 4msecs. Target board is now available as end point ‘EP0’ to the native model. Native model can use this end point to connect to respective IOs on the target.

· Click on ‘Save’ button to save the configuration and close the window.

· Run the model by clicking on the Run button. A simulation panel window should open and communicate with the board.

· Screen shot of the output simulation panel running on host PC is shown below.

Adjustments

None

Control Methodology

· Reset button (BTN0) on the target board may be used to reset the tracking in case the object tracking is lost.

References

Please go through our video tutorials, tutorial projects and CASP main documentation for getting started with CASP.