Category Archives: Programming

Software, compilers, API design, user interfaces.

MSL Input and Drawing Programming Interface

Input: 
//True when key/button is down, false when up. 
bool input_check(const int key); 
 
//True when key/button is pressed once, after being pressed, the key must be released and then pressed down again. 
bool input_check_pressed(const int key); 
 
//True when the key/button is released. 
bool input_check_released(const int key); 
 
//Regular Keyboard Bindings 
kb_bang                 '!' 
kb_quote                '\"' 
kb_she                  '#' 
kb_dollar               '$' 
kb_percent              '%' 
kb_ampersand            '&' 
kb_apostrophe           '\'' 
kb_open_parentheses     '(' 
kb_close_parentheses    ')' 
kb_asterisk             '*' 
kb_plus                 '+' 
kb_comma                ',' 
kb_minus                '-' 
kb_period               '.' 
kb_forward_slash        '/' 
kb_0                    '0' 
kb_1                    '1' 
kb_2                    '2' 
kb_3                    '3' 
kb_4                    '4' 
kb_5                    '5' 
kb_6                    '6' 
kb_7                    '7' 
kb_8                    '8' 
kb_9                    '9' 
kb_colon                ':' 
kb_semi_colon           ';' 
kb_less_than            '<' 
kb_equal                '=' 
kb_greater_than         '>' 
kb_question             '?' 
kb_at                   '@' 
kb_A                    'A' 
kb_B                    'B' 
kb_C                    'C' 
kb_D                    'D' 
kb_E                    'E' 
kb_F                    'F' 
kb_G                    'G' 
kb_H                    'H' 
kb_I                    'I' 
kb_J                    'J' 
kb_K                    'K' 
kb_L                    'L' 
kb_M                    'M' 
kb_N                    'N' 
kb_O                    'O' 
kb_P                    'P' 
kb_Q                    'Q' 
kb_R                    'R' 
kb_S                    'S' 
kb_T                    'T' 
kb_U                    'U' 
kb_V                    'V' 
kb_W                    'W' 
kb_X                    'X' 
kb_Y                    'Y' 
kb_Z                    'Z' 
kb_open_square          '[' 
kb_back_slash           '\\' 
kb_close_square         ']' 
kb_carrot               '^' 
kb_under_score          '_' 
kb_accent               '`' 
kb_a                    'a' 
kb_b                    'b' 
kb_c                    'c' 
kb_d                    'd' 
kb_e                    'e' 
kb_f                    'f' 
kb_g                    'g' 
kb_h                    'h' 
kb_i                    'i' 
kb_j                    'j' 
kb_k                    'k' 
kb_l                    'l' 
kb_m                    'm' 
kb_n                    'n' 
kb_o                    'o' 
kb_p                    'p' 
kb_q                    'q' 
kb_r                    'r' 
kb_s                    's' 
kb_t                    't' 
kb_u                    'u' 
kb_v                    'v' 
kb_w                    'w' 
kb_x                    'x' 
kb_y                    'y' 
kb_z                    'z' 
kb_open_bracket         '{' 
kb_pipe                 '|' 
kb_close_bracket        '}' 
kb_tilde                '~' 
 
//Advanced Keyboard Bindings 
kb_backspace 
kb_tab 
kb_enter 
kb_escape 
kb_space 
kb_insert 
kb_numlock 
kb_delete 
 
//Special Keyboard Bindings 
kb_BASE 
kb_f1 
kb_f2 
kb_f3 
kb_f4 
kb_f5 
kb_f6 
kb_f7 
kb_f8 
kb_f9 
kb_f10 
kb_f11 
kb_f12 
kb_left 
kb_up 
kb_right 
kb_down 
kb_page_up 
kb_page_down 
kb_home 
kb_end 
 
//Mouse Bindings 
mb_BASE 
mb_left 
mb_middle 
mb_right 
mb_scroll_up 
mb_scroll_down
 
Drawing: 
//Color Class Declaration 
class color 
{ 
    public: 
        //Friend Classes 
        friend class sprite; 
 
        //Constructor (Default) 
        color(const float red=1,const float green=1,const float blue=1,const float alpha=1); 
 
        //Member Variables 
        float r; 
        float g; 
        float b; 
        float a; 
}; 
 
//Point Direction Function (Returns direction between two points in degrees) 
double point_direction(const double x1,const double y1,const double x2,const double y2); 
 
//Point Distance Function (Returns distance between two points) 
double point_distance(const double x1,const double y1,const double x2,const double y2); 
 
//Basic Shape Drawing Functions 
void draw_point(const double x,const double y,const msl::color& color=msl::color(1,1,1,1)); 
 
void draw_line(const double x1,const double y1,const double x2,const double y2,const msl::color& color=msl::color(1,1,1,1)); 
 
void draw_triangle(const double x1,const double y1,const double x2,const double y2,const double x3,const double y3, 
    const bool fill,const msl::color& color=msl::color(1,1,1,1)); 
 
void draw_rectangle(const double x,const double y,const double width,const double height,const bool fill,const msl::color& color=msl::color(1,1,1,1)); 
 
void draw_rectangle_center(const double x,const double y,const double width,const double height,const bool fill,const msl::color& color=msl::color(1,1,1,1)); 
 
void draw_rectangle_gradient(const double x,const double y,const double width,const double height,const bool fill, 
    const msl::color& color_top_left=msl::color(1,1,1,1), 
    const msl::color& color_top_right=msl::color(1,1,1,1), 
    const msl::color& color_bottom_right=msl::color(1,1,1,1), 
    const msl::color& color_bottom_left=msl::color(1,1,1,1)); 
 
void draw_rectangle_center_gradient(const double x,const double y,const double width,const double height,const bool fill, 
    const msl::color& color_top_left=msl::color(1,1,1,1), 
    const msl::color& color_top_right=msl::color(1,1,1,1), 
    const msl::color& color_bottom_right=msl::color(1,1,1,1), 
    const msl::color& color_bottom_left=msl::color(1,1,1,1)); 
 
void draw_circle(const double x,const double y,const double radius,const msl::color& color=msl::color(1,1,1,1));

OpenCV for multi-color tracking and localization

Localization, or “where is my robot?”, is really important, since you can’t tell the robot where to go unless you know where you’re starting.  It’s also a hard problem to solve reliably, especially indoors where you don’t have GPS.  For the 2013 CPS challenge, we used a Kinect to find our flying UAV, but we’d like to support ground vehicle localization too, and that’s not easy in a depth image.

I’ve done webcam color matching for decades, but I’ve always used my own homebrew image access and processing libraries, which makes it hard to use, port, and modify my stuff–even for me!  This month, I’ve been finally learning a standard, portable video and image analysis library: OpenCV.  It’s even got a simple GUI library built in, so you can add sliders and such.

Here’s the program in action:

The red and green blobs to the right of my head are bright pink and green squares of paper, detected and highlighted in the image.  Note the bad matches on my red chair.

The basic process is to find all the pixels that match a given color, estimate their center of mass, then draw a smaller box around that center of mass for a second pass.  This produces a “trimmed mean” position, which is less sensitive to outliers.

The output of the program is the average (X,Y) coordinates of the center of mass of each color, and the number of pixels that matched.  If not enough pixels match, the target object probably isn’t visible.  The program has sliders so you can adjust the matching tolerances, and you can click to get a new hue, saturation, value (HSV) colorspace target color.

If you have the target color set exactly right, and your webcam is working well, you can get accuracy better than 0.1 pixel!  But if the colors are dancing around, you might get 2-3 pixel variance in the location.  And if you have the colors wrong, or something else is the same color, you can get entirely the wrong location.

Because the apparent color of a reflective object like a sheet of paper depends on the ambient lighting, I need to allow loose color tolerances or else tweak the target colors to get a good match.  We should try using some diffused LEDs or color lasers to produce inherent emissive colors; back in 2008 David Krnavek used an LED in a white ping-pong ball for color tracking, with good results.

Latency seems good, and I get about 20 frames per second even on my low-end backup laptop and random webcam.  However, the OpenCV graphical output is fairly expensive, so I don’t do it every frame.

Download the color tracker code at my github.  The plan is to build up something like this into a reliable web accessible multi-robot tracking system!

Aero Balance Beam: Control Theory Demo

At our workshop this week, we did some hands-on testing of our control theory knowledge with an aerodynamically-driven balance beam–basically just an electrically controlled fan that blows to lift up a weighted board.  The idea was to capture the difficulties of UAV control in a simpler and slightly less dangerous system.  Little did we know what we were signing up for!  (Arduino code and data after the break.)

Continue reading Aero Balance Beam: Control Theory Demo

RobotC Gamepad Library

Controlling an NXT with a gamepad in RobotC isn’t too difficult, but it does require some more advanced programming techniques.  Most programmers end up creating some helper functions and variables that they copy and paste throughout their projects, and here is my “library”.

LINK

Gamepad Layout

Example:

//RobotC Joystick Helper Example Source
//    Created By:        Mike Moss
//    Modified On:       01/09/2013

//Joystick Helper Code
#include "joystick_helper.c"

//Our Program
task main()
{
    //Do Forever
    while(true)
    {
        //Get Gamepad States
        js_update();

        //Move motor A with the X button on joystick 1
        if(js_button(1,js_x)==true)
        {
            motor[motorA]=100;
        }

        //Move motor C with the up arrow on the dpad (0 is up, 2 is right, 4 is down, etc...)
        if(js_dpad(0))
        {
            motor[motorC]=100;
        }

        //Move motor B based on the Y axis on the left thumb stick
        //    of joystick 2 with a deadzone of 15
        motor[motorB]=js_lthumb_y(2,15);
    }
}

Arduino Serial Latency & Bandwidth vs. Message Size

We’ve been using Arduinos for all our projects, and it was time I got around to benchmarking the serial communication performance.  It’s actually not very fast; even at the maximum baud rate of 115200 bits per second, delivered performance is only a little over 10KB/second each way, and it only hits this bandwidth when sending over 100 bytes at a time.

Arduino Uno to PC roundtrip serial communication bandwidth, as a function of message size.

The problem for small messages seems to be a 4 millisecond minimum roundtrip latency.  Messages over about 40 bytes seem to take several such latencies, so there’s a stair step pattern to the latency.  Paul Stoffregen says this is due to the Uno firmware’s 4.1 millisecond transmit timeout.

Arduino Uno to PC roundtrip serial communication latency, measured in milliseconds, for various message sizes.

Evidently, the Teensy (with direct USB to the chip, not a USB-to-serial onboard) gets about 1ms serial latency.  The same page reported the Duemillanove at 16ms minimum (62Hz!).

Overall, this means you’re only going to get a 250Hz control rate if you’re shipping sensor data from an Arduino Uno up to a PC, and then sending actuator signals back down to the Arduino.  But 250Hz is enough for most hardware projects we’ve been thinking about.

The other annoying problem?  After opening the serial port, the Arduino Uno takes 1.7 seconds to boot before it responds to serial commands.  Anything you send from the PC before that time seems to be lost, not buffered.  The fix is probably to have the Uno send the PC one byte at startup, so the PC knows the Uno is ready.