Simi MotionCapture3D Workflow

Specification

You start with defining which objects and which of their movements you would like to capture. Generally, for a human these are the movements of the joints. A so called specification is built for these objects, consisting of the points you want to track on the video image later. All points are listed to let you pick the desired ones and they can also become connected to each other. This makes it possible to create a hierarchical structure, for example a human skeleton. The file format of the subsequent data export should be kept in mind, for example, if you would like to export motion data to the BVH file format for the human body, a special set of points is needed. Suitable templates for this purpose are included in delivery.

Camera setup

Several cameras respectively record the motion of the object simultaneously. The following calculation of three-dimensional data requires at least two cameras. Using more cameras and well chosen positioning improves accuracy of the captured data. At the same time the probability of occlusion of a point in all camera views during its movement decreases.

Types of cameras

You can use any kind of camera, also image and video data from other recording systems if they show the object captured from at least from two perspectives and are synchronized by time. Simi MotionCapture3D offers integrated support for simultaneoused video capturing using DV cameras and using synchronized High Speed FireWire cameras.

Calibration

Each camera is being calibrated once before recording starts. This means, its position and field of view relative to the other cameras is being determined. A so called calibration system is used for this purpose, which is offered by Simi in different configurations. Arbitrary objects in the room or home made calibration system can act as a makeshift. You have to make sure the system has enough points, all of them are visible in all cameras, all relative distances between the points are known, and they encompass the entire space of the motion you want to capture.

Markers

Simi MotionCapture3D works markerbased, therefore points which shall be captured have to distinguish themselves from the background by color. SIMI offers special marker sets for this purpose. You can also use any kind of adhesive labels or other markings on the points you want to capture as substitution. The software is able to distinguish between markers with different colors. Optionally recognition of patterns (pattern matching) is offered. This allows striking objects or points to be tracked automatically on the video image without additional markers.

Realtime

Simi MotionCapture3D processes video data not in realtime. This means that video files are initially created from all cameras which are processed in a second work step. The so-called tracking process is done quite efficiently and is being heavily supported by the automatic marker tracking respectively by pattern matching.

Tracking

The points from which you want to acquire motion data are being marked by a mouse click in the first frame of the video. If the points on the image are highlighted with markers and are always visible, the software tracks movements automatically. In case the points cannot be marked, either pattern matching can be applied or they can be tracked manually. The latter is also possible when some markers are temporarily covered or if the automatic tracking temporarily loses a point.

Post-editing

Now it is possible to edit, correct and smooth the gained 2D data from each camera separately. Data can be exported into a text file on demand and can be edited or even utilized externally. Processed data can be re-imported into Simi MotionCapture3D as well.

Calculation and export

The software calculates three-dimensional movements of joints or markers using the two-dimensional information from several camera views and from the calibration. These movements can be exported into a file afterwards and processed e.g. with any animation application. Simi MotionCapture3D therefore supports diverse file formats. Prior to export you have the chance to edit the 3D data once more. It can be smoothed or processed with optional software modules.