XML Stream format

The MultiTouch CornerStone XML-Stream Data Format description.

The MultiTouch CornerStone tracker can send out data over a TCP/IP XML-socket. The data is sent out for every sample. With single-camera installations a sample maps to one camera frame. Sample information is sent out regardless of whether there are any fingers on the screen or not.

Below is a snippet of typical XML stream.

<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<sample>
</sample>
<sample>
</sample>
<sample>
<finger><id>2</id><age>1&lt/age><loc><x>246.54</x><y>261.02</y></loc></finger>
<hand><id>1</id><age>1</age><palm><x>320.93</x><y>339.45</y></palm><fingerid>2</fingerid></hand>
</sample>
<sample>
<finger><id>2</id><age>2</age><loc><x>246.82</x><y>260.60</y></loc></finger>
<hand><id>1</id><age>2</age><palm><x>331.22</x><y>341.42</y></palm><fingerid>2</fingerid></hand>
</sample>

...

<sample>
<finger><id>2</id><age>193</age><loc><x>298.27</x><y>379.13</y></loc></finger>
<finger><id>4</id><age>1</age><loc><x>394.16</x><y>345.49</y></loc></finger>
<hand><id>1</id><age>193</age><palm><x>441.85</x><y>496.62</y></palm><fingerid>2</fingerid><fingerid>4</fingerid></hand>
</sample>
<sample>
<finger><id>2</id><age>194</age><loc><x>301.10</x><y>377.91</y></loc></finger>
<finger><id>4</id><age>2</age><loc><x>394.10</x><y>345.29</y></loc></finger>
<hand><id>1</id><age>194</age><palm><x>442.73</x><y>495.81</y></palm><fingerid>2</fingerid><fingerid>4</fingerid></hand>
</sample>

The multitouch data is packed inside "sample" elements. Each sample reflects the finger/hand information on the screen at one time. The tracker produces samples at the frame rate of the camera(s), typically about 60 fps. After each sample the server outputs one byte with zero value, because otherwise Flash will not parse the stream.

Inside sample there are two types of data elements:

"Finger" elements contain information about a given finger, while "hand" elements contain information about a hand.

Each finger has the following parameters:

  • id - A unique integer identifier for the finger. The id numbers will be increasing over time. How-ever some ids maybe skipped over due to the internal workings of the engine, a typical id series might be: 1, 2, 3, 5, 6, 9, 10...

  • age - The age of the finger as an integer. This number reflects the number of samples since this finger was first seen.

  • loc - The location of the finger-tip, with x and y values. The coordinate values are typically either screen coordinates or window coordinates.

Each hand has the following parameters:

  • id - A unique idenfier for the hand. The semantics are identical to finger ids.

  • age - The age of the hand as an integer. The semantics are identical to finger age.

  • palm - The estimated location of the palm center, with xy-coordinates.

  • fingerid - One or more finger-ids that link to the fingers that are connected to this hand.