Capture movement ( Mo-cap in short) is the process of recording the movement of an object or person. It is used in military, entertainment, sports, medical applications, and for the validation of computer vision and robotics. In filmmaking and video game development, it refers to the recording of acts of human actors, and uses that information to animate digital character models in 2D or 3D computer animations. When it includes the face and fingers or capturing a subtle expression, it is often referred to as performance capture . In many areas, motion capture is sometimes called motion tracking , but in movie and game making, motion tracking usually refers to match moving .
In a motion-taking session, the motion of one or more actors is taken several times per second. While the initial technique uses images from multiple cameras to calculate the 3D position, it is often the purpose of motion capture to record only the actors' movements, not the visual appearance. animated data is mapped to a 3D model so that the model performs the same action as the actor. This process may contrast with older rotoscoping techniques, as seen in Ralph Bakshi's The Lord of the Rings (1978) and American Pop (1981). Movement of animated characters is accomplished in these films by tracing the live-action actor, capturing the movements and movements of the actor. To clarify, an actor filmed an action, and then the recorded film is projected onto the frame by frame animation table. Animators track live-action recordings into animated cels, capture the lines and movements of frame-by-frame actors, and then they fill in lines that are tracked with animated characters. The completed animated gels are then photographed frame by frame, exactly according to the movements and actions of the live-action recording. The end result is that the animated character replicates exactly the live action of the actor. However, this process takes a lot of time and effort.
Movement of the camera can also be captured so that the virtual camera in the scene will shift, tilt or dolly around the stage which is driven by the camera operator when the actor is performing. At the same time, the motion capture system can capture the camera and props as well as the performance of the actor. This allows the computer's characters, images and sets to have the same perspective as the video images from the camera. The computer processes data and displays actors' movements, providing the desired camera position in terms of objects set. Retroactively get camera motion data from recorded recordings known as mobile games or camera tracking .
Video Motion capture
Benefits
Motion retrieval offers several advantages over traditional 3D computer animation models:
- Low latency, near real time, results can be obtained. In this entertainment app can reduce the cost of keyframe-based animation. The Hand Over technique is an example of this.
- The number of jobs does not vary with the complexity or duration of performance at the same level as when using traditional techniques. This allows many tests to be performed with different styles or deliveries, giving different personalities limited only by the talent of the actor.
- Complex movements and realistic physical interactions such as secondary movement, weight, and power exchange can be easily recreated in a physically accurate way.
- The amount of animated data that can be produced in a given time is huge when compared to traditional animation techniques. This contributes to cost effectiveness and meeting production deadlines.
- The potential for free software and third party solutions reduces costs.
Maps Motion capture
Loss
- Special hardware and special software programs are required to obtain and process data.
- The cost of necessary software, equipment, and personnel can be a barrier for small production.
- The capture system may have special requirements for its operating room, depending on the camera's field of view or magnetic distortion.
- When the problem occurs, it's easier to reshoot the scene than to try to manipulate the data. Only a few systems allow viewing data in real time to decide whether the retrieval needs to be reworked.
- Initial results are limited to what can be done in the capture volume without additional data editing.
- Movements that do not follow the laws of physics can not be captured.
- Traditional animation techniques, such as additional emphasis on anticipation and follow-up, secondary movements or character form manipulations, such as squash and stretch animation techniques, should be added later.
- If the computer model has a different proportion of the fetch subject, the artifact may occur. For example, if a cartoon character has big and big hands, it can cut the body of a character if the human player is not careful with his physical movements.
Apps
Video games often use motion capture to turn on athletes, martial artists, and other in-game characters. This has been done since the arcade game Sega Model 2 Virtua Fighter 2 in 1994. By mid 1995, the use of motion capture in video game development has become commonplace, and developer/publisher Acclaim Entertainment has just left. As far as having a motion-taking studio in his own home built at his headquarters. The 1995 arcade game Namco Soul Edge uses a passive optical system marker for motion capture.
The film uses motion capture for CG effects, in some cases replacing traditional cel animations, and for fully computer-generated creatures, such as Gollum, The Mummy, King Kong, Davy Jones of Pirates of the Caribbean, Na ' vi from the movie Avatar , and Clu from Tron: Legacy . The Great Goblin, three Stone-trolls, many orcs and goblins in the 2012 movie The Hobbit: An Unexpected Journey , and Smaug is made using motion capture.
The Indian-American Movie Sinbad: Beyond the Veil of Mists (2000) is the first long film made primarily with motion capture, although many animator characters also work on films, which have very limited. let go. 2001
Lord of the Rings: The Two Towers is the first feature film to utilize a real-time motion capture system. This method drains the actor Andy Serkis action to the computer produced Gollum/Smeagol skin as it is being done.
Of the three nominations for the 2006 Academy Award for Best Animated Feature, two of the nominees ( Monster House and Happy Feet winners ) used motion capture, and only Disney Ã, Â · Pixar's Cars is animated without motion capture. In the final credits of Pixar Ratatouille , a stamp will label the movie as "100% Pure AnimationÃ, - No Motion Capture!"
Since 2001, motion capture has been widely used to produce films that try to simulate or estimate the appearance of live-action movies, with almost photorealistic digital character models. The Polar Express uses motion capture to allow Tom Hanks to appear as several different digital characters (in which he also votes). The 2007 adaptation of Beowulf's stories of animated digital characters whose appearance is partly based on actors who give their movements and voices. James Cameron is so popular Avatar using this technique to create the Na'vi that inhabit Pandora. The Walt Disney Company has produced Robert Zemeckis A Christmas Carol using this technique. In 2007, Disney acquired Zemeckis' ImageMovers Digital (which produced motion capture films), but later closed it in 2011, following a series of failures.
Television series produced entirely with motion capture animation including Laflaque in Canada, Sprookjesboom and Cafe de Wereld in the Netherlands and Headcases i> in the UK.
Virtual Reality and Augmented Reality providers, such as uSens and Gestigon, allow users to interact with digital content in real time by capturing hand gestures. This can be useful for simulation training, visual perception tests, or virtual walk-through in a 3D environment. Motion capture technology is often used in digital doll systems to drive computer-generated characters in real-time.
Gait analysis is the main application of motion capture in clinical medicine. Techniques allow physicians to evaluate human movements in some biometric factors, often while streaming this information directly into analytical software.
During the filming of James Cameron's Avatar all scenes involving this process are directed in realtime using Autodesk Motion Builder software to render screen images that allow directors and actors to see what their look on the movie makes it easier to direct the movie as viewers will see it. This method allows views and angles to not be possible from previously rendered animations. Cameron was so proud of the result that he even invited Steven Spielberg and George Lucas on set to see the system in action.
In Marvel's critically acclaimed The Avengers, Mark Ruffalo uses motion capture so he can play his character as a Hulk, rather than making it a CGI just like the previous movie, making Ruffalo the first actor to play a human and a Hulk version of Bruce Banner.
The FaceRig software uses facial recognition technology from ULSee.Inc to map the player's facial expressions and body tracking technology from Neuron Perceptions to map body movements to a 3D or 2D character motion screen.
During the Game Developers Conference 2016 in San Francisco Epic Games demonstrate intake of motion across the body directly on the Unreal Engine. The whole scene, from the upcoming game Hellblade about a female soldier named Senua, is given in real-time. Keynote is a collaboration between Unreal Engine , Ninja Theory , 3Lateral , Cubic Motion , > and Xsens .
Methods and systems
Motion tracking or motion capture began as a photogrammetric analysis tool in biomechanical research in the 1970s and 1980s, and expanded into education, training, sports, and recent computer animation for television, cinema and video games as technology matures. Since the 20th century, players have to wear a marker near each joint to identify movements with positions or corners between markers. Acoustic markers, inertia, LEDs, magnetic or reflective, or any combination of these, are traced, at least optimally at least twice the desired frequency movement rate. System resolutions are important both in spatial resolution and temporary resolution because motion blur causes almost the same problem with low resolution. Since the beginning of the 21st century and due to rapid technological developments, new methods have been developed. Most modern systems can extract the silhouette of players from the background. After that all the combined corners are calculated by installing in the mathematical model into the silhouette. For movement you can not see the silhouette change, there is a hybrid system available that can do both (marker and silhouette), but with less marker.
Optical system
Optical systems utilize data taken from the image sensor to triangulate the 3D position of the subject between two or more calibrated cameras to provide overlapping projection. Data acquisition is traditionally implemented using a special marker attached to an actor; However, newer systems can generate accurate data by tracking the surface features that are dynamically identified for each particular subject. Tracking a large number of players or expanding the catch area is accomplished with the addition of more cameras. The system generates data with three degrees of freedom for each marker, and the rotation information must be inferred from the relative orientation of three or more markers; such as a marker of the shoulders, elbows and wrists that provide the angle of the elbow. The newer hybrid system incorporates inertial sensors with optical sensors to reduce occlusion, increase the number of users and increase the ability to track without having to manually clean the data.
Passive tags
Passive optical system uses markers coated with a retroreflective material to reflect the light produced near the camera lens. Camera thresholds can be adjusted so that only bright reflective markers will be sampled, ignoring the skin and fabric.
The mass center of the marker is predicted as a position in the captured two-dimensional image. The grayscale value of each pixel can be used to provide sub-pixel accuracy by searching the Gaussian center.
An object with a marker attached to a known position is used to calibrate the camera and obtain their position and lens distortion of each camera is measured. If two calibrated cameras see a marker, three-dimensional improvements can be obtained. Usually the system will consist of about 2 to 48 cameras. System over three hundred cameras exist to try to reduce marker exchanges. An extra camera is required for full coverage around the fetch subject and some subjects.
Vendors have software constraints to reduce marker exchange problems because all the passive markers look identical. Unlike active marker systems and magnetic systems, passive systems do not require users to use cables or electronic equipment. Instead, hundreds of rubber balls are attached with reflective tape, which needs to be replaced periodically. Markers usually stick directly to the skin (as in biomechanics), or they are glued to a player wearing a full body spandex/lycra suit designed specifically for movement capture. This type of system can capture a large number of markers at image frequencies typically around 120 to 160 fps though by lowering the resolution and tracking the smaller interest areas that they can track as high as 10,000 fps.
Active marker
An active optical system performs triangulation of positions by illuminating one LED at a very fast time or several LEDs with software to identify them in their relative positions, somewhat similar to celestial navigation. Rather than reflecting back externally generated light, the markers themselves are empowered to emit their own light. Because the inverse square law provides a quarter of power at twice the distance, this can increase the distance and volume for retrieval. It also allows a high signal-to-noise ratio, resulting in very low jitter marker and high resolution measurements produced (often down to 0.1 mm in calibrated volume).
The TV series Stargate SG1 produces episodes using an active optical system for VFX that lets actors walk around props that will make motion capture difficult for other non-active optical systems.
ILM uses an active marker in Van Helsing to allow the capture of Dracula's flying bride on a very large set similar to the use of Weta active markers in Rise of the Planet of the Apes . Strength for each marker can be given sequentially in phase with the retrieval system providing a unique identification of each marker for a given fetch frame at a cost to the resulting frame rate. The ability to identify each marker in this way is useful in real time applications. An alternative method for identifying a marker is by doing it in an algorithm that requires processing of extra data.
It is also possible to find a position by using a color LED marker. In this system, each color is set to the specific point of the body.
One of the early active marker systems of the 1980s was a hybrid active passive mocap system with rotating mirrors and glass-colored reflective markers and used covert linear array detectors.
Activated time modulated marker
Active marker systems can be further enhanced by enabling one marker at a time, or tracking multiple markers over time and modulating the width of the amplitude or pulse width to provide the marker ID. The 12 megapixel spatial modulation system features smoother motion than a 4 megapixel optical system with higher spatial and temporal resolution. Directors can see the performance of actors in real time, and watch the results on movement driven by CG characters. The unique marker ID reduces turnaround, by eliminating swapping markers and providing much cleaner data than any other technology. LEDs with onboard processing and radio synchronization allow outdoor motion capture in direct sunlight, while capturing at 120 to 960 frames per second due to high-speed electronic shutter. The computer processing of the modulated ID allows less hand-cleansing or filtered results for lower operational costs. This higher accuracy and resolution requires more processing than passive technology, but additional processing is done in the camera to increase resolution through subpixel or centroid processing, providing high resolution and high speed. This motion capture system is usually $ 20,000 for eight cameras, 12 megapixel spatial resolution 120 hertz system with one actor.
Unsurprisingly semi-passive marker
One can reverse the traditional approach based on high speed cameras. Systems such as Prakash use low-cost multi-LED low-cost projectors. Dedicated multi-LED IR projectors are optically encoded space. Instead of a retro-reflective or light-emitting diode (LED) marker, the system uses a photosensitive marking marker to decode the optical signal. By attaching tags with photo sensors to scene points, tags can calculate not only their own location from any point, but also their own orientation, incident lighting, and reflections.
These tracking tags work in natural lighting conditions and can be embedded in clothing or other objects. The system supports unlimited number of tags in a scene, with each tag uniquely identified to eliminate marker recovery issues. Because the system eliminates high-speed cameras and high-speed image streams as appropriate, it requires much lower data bandwidth. This tag also provides incident lighting data that can be used to match scene lighting when entering synthetic elements. This technique seems ideal for capturing real-time set-up or broadcasting of virtual sets but has not been proven.
Underwater motion capture system
Motion capture technology has been available to researchers and scientists for decades, which has provided new insights into many areas.
Underwater camera
The vital part of the system, the underwater camera, has a waterproof home. This housing has a corrosion-resistant and chlorine-resistant finish that makes it perfect for use in swimming pools and basins. There are two types of cameras. High-speed industrial cameras can also be used as infrared cameras. Underwater infrared cameras are equipped with cyan light strobes, not typical IR light - for minimum falloffs under water and high-speed camera cones with LED lights or with the option of using image processing.
Volume measurements
Underwater cameras are usually capable of measuring 15-20 meters depending on water quality, camera and type of marker used. Not surprisingly, the best range is achieved when the water is clear, and as always, the volume of measurements also depends on the number of cameras. Various underwater markers are available for various situations.
Customized
Different pools require different mounting and equipment. Therefore, all underwater movement capture systems are uniquely tailored for each special pool installment. For cameras placed in the center of the pond, a specially designed tripod, using suction cups, is provided.
No Marker
Appearing techniques and research in computer vision leads to the rapid development of an unmarked approach to motion capture. Unmarked systems such as those developed at Stanford University, the University of Maryland, MIT, and the Max Planck Institute, do not require subjects to use specialized equipment for tracking. The computer algorithms are specifically designed to allow the system to analyze multiple optical input streams and identify human shapes, breaking them into constituent parts for tracking. Entertainment ESC, a subsidiary of Warner Brothers Pictures created specifically to enable virtual cinematography, including digital photorealistics similar to The Matrix Reloaded and The Matrix Revolutions, uses a technique called Universal Capture that utilizes 7 camera settings and optical flow tracking of all pixels in over all 2-D aircraft from the camera for movement, movement and facial expressions that lead to photorealistic results.
Traditional system
Traditional track tracking without traditional traces is used to track various objects, including airplanes, launch vehicles, missiles, and satellites. Many such optical motion tracking applications take place outdoors, requiring different lens and camera configurations. High resolution images of the target being tracked can thus provide more information than just motion data. Images obtained from NASA's remote tracking system at the fatal launch of the Challenger shuttle provide important evidence about the cause of the accident. An optical tracking system is also used to identify spacecraft and known space debris despite the fact that it has a weakness over the radar in which the object must reflect or emit sufficient light.
Optical tracking systems typically consist of three subsystems: optical imaging systems, mechanical tracking platforms and tracking computers.
The optical imaging system is responsible for converting the light from the target area into digital images that can be processed by a tracking computer. Depending on the design of optical tracking systems, optical imaging systems can vary from as simple as standard digital cameras to specialized such as astronomical telescopes on mountain tops. The optical imaging system specification sets the upper limit of the effective range of the tracking system.
The mechanical tracking platform holds the optical imaging system and is responsible for manipulating the optical imaging system in such a way that it always points to the target being tracked. The dynamics of the mechanical tracking platform combined with the optical imaging system determines the ability of the tracking system to keep locks on targets that change speed quickly.
The tracking computer is responsible for capturing images from optical imaging systems, analyzing images for extracting target positions and controlling the mechanical tracking platform to follow targets. There are some challenges. First the tracking computer must be able to capture images at relatively high frame rates. This posts the requirements on the bandwidth of the capturing hardware images. The second challenge is that the image processing software must be able to extract the target image from its background and calculate its position. Some textbook image processing algorithms are designed for this task. This problem can be simplified if the tracking system can expect certain characteristics that are common to all targets to be tracked. The next issue under the line is to control the tracking platform to follow the target. It is a matter of control system design that is typical rather than a challenge, which involves modeling system dynamics and designing a controller to control it. But this will be a challenge if the tracking platform that the system has to work on is not designed for real time.
The software that runs the system is also customized for the appropriate hardware components. One example of such software is OpticTracker, which controls computerized telescopes to track moving objects at great distances, such as planes and satellites. Another option is SimiShape software, which can also be used in combination with hybrid markers.
Markerless 3D System
Traditional motion capture systems can be complex, costly and/or operationally intensive. The modern 3D markerless system has several advantages: 1) low capital cost 2) easy operation: no marker, only one 3D camera is needed & amp; holistic assessment in 3 minutes) portable and compact - no longer require special motion laboratory facilities.
Non-optical system
System inertia
Inertial motion capture technology is based on miniature inertial sensors, biomechanical models and fusion sensor algorithms. Inertial sensor movement data (inertial guidance systems) are often transmitted wirelessly to computers, where movement is recorded or viewed. Most inertial systems use an inertial measurement unit (IMU) containing a combination of gyroscopes, magnetometers, and accelerometers, to measure rotation rates. This rotation is translated to the framework in the software. Just like optical markers, the more IMU sensors the more natural the data. No external cameras, emitters or markers are required for relative motion, although they are required to provide the absolute position of the user if desired. The inertial motion capture system captures the full six-degree body movement of human freedom in real-time and can provide limited direction information if they include magnetic bearing sensors, although this resolution is much lower and susceptible to electromagnetic noise. The benefits of using Inertial systems include: capturing in various environments including narrow spaces, no solutions, portability, and large catchment areas. Losses include positional accuracy and lower position shifts over time. The system is similar to a Wii controller but more sensitive and has higher resolution and updates. They can accurately measure the direction to the ground in a certain degree. The popularity of the inertial system is increasing among independent game developers, mainly due to the quick and easy setting that produces fast pipes. A variety of settings are now available from different manufacturers and base prices range from $ 1,000 to $ 80,000 USD.
Mechanical movement
The mechanical motion capture system directly tracks the angle of the joints of the body and is often referred to as the exoskeleton motion capture system, due to the way the sensor is attached to the body. A player attaches a skeletal-like structure to his body and as they move it is the articulated part of the mechanic, measuring the relative motion of the player. Mechanical motion photography systems are real-time, relatively inexpensive, free-occlusive, and wireless (not installed) systems that have unlimited capture volume. Typically, they are rigid structures of straight, jointed metal or plastic rods linked together with potentiometers that articulate on the joints of the body. It tends to be in the range of $ 25,000 to $ 75,000 plus an absolute external positioning system. Some settings provide limited-style feedback or haptic input.
Magnetic system
The magnetic system calculates position and orientation by the relative magnetic flux of three orthogonal coils on both the transmitter and each receiver. The relative voltage or current intensity of the three coils allows the system to calculate ranges and orientations by carefully mapping the tracking volume. The sensor output is 6DOF, which provides useful results obtained by two-thirds the number of markers required in the optical system; one in the upper arm and one in the forearm for the position and angle of the elbow. The signs are not clogged by nonmetallic objects but are vulnerable to magnetic and electrical interference from metal objects in the environment, such as rebar (steel rein in concrete) or wires, affecting magnetic fields, and electrical sources such as monitors, lamps, cables and computers. The sensor response is not linear, especially towards the edge of the catchment area. Cables from sensors tend to block extreme performance movements. With a magnetic system, it is possible to monitor the results of real-time motion capture sessions. The capture volume for magnetic systems is dramatically smaller than for optical systems. With a magnetic system, there is a difference between "AC" and "DC" systems: one using square pulses, the other using a sine wave.
Related techniques
Face motion capture
Most hardware capture traditional hardware provides multiple types of low-resolution shots using 32 to 300 markers with active or passive marking systems. All of these solutions are limited by the time required to apply the marker, calibrate the position and process the data. Ultimately this technology also limits their resolution and quality of raw yield.
The high-accuracy motion capture movement, also known as performance capture , is the next generation of loyalty and is used to record more complex movements in the human face to capture higher levels of emotion. The current face capture organizes itself in several different camps, including traditional motion capture data, mixed-shaped mixed solutions, capturing the true topology of the actor's face, and proprietary systems.
The two main techniques are stationary systems with various cameras that capture facial expressions from different angles and use software such as a stereo mesh solver from OpenCV to create a 3D surface mesh, or use a light array as well to calculate the surface normals of the variance in brightness as a light source, camera or both modified. These techniques tend to be limited only in feature resolution by camera resolution, clear object size and number of cameras. If the user's face is 50 percent of the camera's work area and the camera has a megapixel resolution, then the sub-millimeter face movement can be detected by comparing the frame. The latest work focuses on increasing frame rates and performing optical flow to allow movement to be re-targeted to other computer-generated faces, rather than just creating 3D Mesh from their actors and expressions.
RF Positioning
RF frequency (radio frequency) determination systems are becoming more active because higher frequency RF devices allow higher precision than older RF technologies such as traditional radar. The speed of light is 30 cm per nanosecond (a millionth of a second), so a 10 gigahertz RF signal (billion per second) allows an accuracy of about 3 cm. By measuring amplitudes up to quarter wavelength, it is possible to improve the resolution to about 8 mm. To achieve an optical system resolution, a frequency of 50 gigahertz or higher is required, which is almost dependent on each other and is easily blocked as an optical system. Multipath and signal reradiation are likely to cause additional problems, but this technology would be ideal for tracking larger volumes with reasonable accuracy, since the required resolution at 100 meters distance may not be the same height. Many RF scientists believe that radio frequencies will never produce the accuracy required for motion capture.
Non-traditional system
An alternative approach developed where actors are given unlimited walking areas through the use of spinning balls, similar to hamster balls, containing internal sensors that record angular motion, eliminating the need for external cameras and other equipment. Although this technology could potentially result in a much lower cost for motion capture, the base ball can only record one continuous direction. Additional sensors worn on the person needed to record more.
Another alternative is to use a 6DOF (Degrees of freedom) motion platform with an integrated omni-directional treadmill with high-resolution optical motion capture to achieve the same effect. The captured person can walk in an unlimited area, negotiating different uneven terrains. Applications include medical rehabilitation for balance training, bio-mechanical research and virtual reality.
See also
- Animated data base
- Gesture Introduction
- Finger tracking
- Inverted kinematics (different ways to create realistic CGI effects)
- Kinect (created by Microsoft Corporation)
- List of motion and motion file formats
- Acting motion capture
References
External links
- The attraction for capturing motion, the introduction of the history of motion capture technology
Source of the article : Wikipedia
0 Comments