March 6, 2024
When many of us think of motion capture, we’re immediately transported to visions of latex-clad performers tangled in more wires than the back of an IT room server circa. 2005. This technology has and still continues to serve a purpose but lacks the opportunities and flexibility provided by a markerless setup.
So what is markerless motion capture?
Markerless motion capture is a way to capture motion without the need for suits, markers, specialized hardware or any other wearables. In our case, it uses artificial intelligence (AI) to capture high-quality motion data from video. AI is used to identify points on the human body and extract its motion from the video footage captured on the digital camera device. This data can then be processed by developers, 3D artists and content creators to create animated characters – also known as ‘character animation’. No suits, wearables or markers are required.
When using markerless motion capture technology, the 3D motion data is generated from 2D video footage, and this is used by creators with animation, CG, 3D and VFX skills to bring animations to life in software of their preference. This is often in the form of Unreal Engine, Autodesk’s Maya, Blender or Houdini.
On the other hand, more traditional methods of motion capture, such as optical and inertial systems, include hardware. This doesn’t have to be in the form of a suit, it can include wearables such as gloves and head-mounted cameras for facial recognition.
These methods aren’t markerless, as they require a form of hardware to capture motion. While markerless motion capture requires cameras to capture the 2D video, no further hardware is required. This is the differentiator.
Optical motion capture involves a system of specialized cameras strategically placed in a designated space. These cameras track the movement of visible reflective markers attached to a person or an object – often humored as the reflective balls attached to a performer's body. As the subject moves, the cameras capture the precise positions of these markers in three-dimensional space.
Inertial motion capture, on the other hand, utilizes sensors that are attached to different parts of a person's body to measure changes in motion. The collected data allows a computer to calculate the body's movement. Inertial motion capture is usually found in the form of suits.
Markerless motion capture uses neither specialized cameras, markers or suits to capture data. In our case, it uses a cocktail of computer vision, physics, biomechanics, and AI to identify the points of movement on the body.
Optical and inertial methods of motion and performance capture are not cheap and nor are they easy to wear or especially flexible when it comes to movement. The irony, right?
With markerless motion capture, no suits, cables or wires are present. Unless of course, you want to wear your Sunday best on-set (no judgment).
Markerless motion capture can deliver high quality motion data, effortlessly adapting to project needs and minimizing post-capture cleanup. It can facilitate the creation of authentic human motion, eliminating the need for restrictive suits and enhancing the realism of performances. Accessibility is a key advantage, with markerless motion capture offering a user-friendly solution for converting 2D videos into dynamic 3D motion data, accessible even on everyday devices like smartphones and action cameras. Moreover, this approach accelerates animation creation by bypassing time-consuming traditional methods such as keyframing. Crucially, the absence of suits, markers, or specialized cameras broadens the scope of applications, allowing motion capture in diverse environments like outdoor spaces, underwater, underground, in the air, and expansive locations such as sports fields.
Just about anywhere with light and good visibility! All you need is a smartphone or a camera, and depending on the types of moves you make, a few square meters of space to calibrate and capture human motion. Like photography and videography, you need light to capture your vision and this is no different for motion capture.
We’ve seen some diverse uses of the technology from capturing a jetpack in flight to simulated robotic ballet!
Believe it or not, markerless mocap extends beyond the world of multimedia entertainment. Healthcare and rehabilitation programmes, especially within physiotherapy, can utilize mocap for movement analysis and patient diagnostics.
Likewise, sports and athletic applications include performance analysis and injury prevention.
Surveillance and security businesses can also benefit from markerless motion tracking, aiding with simulations, monitoring and training. The automotive industry can utilize markerless motion tracking to analyze and enhance safety mechanisms in vehicles, as well as monitor the movement of drivers.
Here’s a breakdown of a few industries where markerless motion capture can be adopted.
Film and TV productions can incorporate motion capture into their pipelines to upscale animations into more lifelike characters.
With action sequences and stunts, motion capture plays a pivotal role in delivering dynamic and authentic movements for characters engaged in intense physical activities, including fights, chases, and acrobatics. The technology's ability to capture nuanced body motions contributes significantly to the realism and impact of these sequences, enhancing the overall cinematic experience. Markerless motion capture adds more freedom for talent and stunt doubles, as it allows for more lifelike movement due to the lack of hardware required.
Additionally, markerless motion capture serves as a key tool in ensuring consistency across sequels or franchises. By maintaining uniformity in the movements of characters appearing in different installments, filmmakers can create a consistent narrative and visual experience for audiences familiar with recurring characters.
Markerless motion capture plays a pivotal role in enhancing the realism and interactivity of video games. The technology is extensively used in character animation, enabling developers to achieve lifelike movements and expressions that are challenging to replicate through manual animation alone.
In addition to character animation, motion capture is employed for various purposes in gaming, including interactive gameplay, sports simulations, and cinematic storytelling. It allows players' real-time movements to influence in-game actions, making the gaming experience more engaging. Sports games leverage motion capture to replicate the movements of real athletes, while fitness and dance games use it to track and assess players' physical movements.
When it comes to markerless motion capture, the benefit here is consistent. No hardware. Optimized natural performances.
Markerless motion capture technology is extensively employed in sports and athletics, across a variety of applications. A major use is for biomechanical analysis, where it aids in scrutinizing an athlete's movements, identifying areas for improvement in technique, and offering targeted feedback for improvement. This precise measurement of joint angles and range of motion helps coaches design personalized training programs, enhancing an athlete's overall performance. Motion capture also contributes significantly to injury prevention by assessing biomechanical stress during movements, identifying potential risks, and guiding rehabilitation programs.
In addition to performance analysis and injury prevention, motion capture technology is integral to virtual training and simulation. Athletes can engage in virtual reality training environments, refining their skills in simulated scenarios. This technology is also pivotal in creating realistic animations for video games, commercials, and sports broadcasts, replicating the movements of real athletes in virtual settings.
Mocap also plays a role in equipment design and scientific research, contributing to the development of sports gear and enhancing understanding of biomechanics in sports.
The use of markerless motion capture in the fashion industry is becoming widespread and is a hot topic.
A popular use is in the creation of virtual fashion shows, allowing designers to present their collections through digital models in simulated environments. This is particularly beneficial when physical events are challenging (the ‘C’ word is banned!). Additionally, motion capture aids designers in studying fabric movement on the body, contributing to the design process and ensuring garments are both stylish and functional.
Extending on this, mocap can be leveraged in digital marketing to produce realistic animations showcasing how clothing moves and fits. This technology extends to virtual fitting rooms, allowing customers to visualize how garments look and behave on a digital representation of themselves, enhancing the online shopping experience. This technology branches into augmented and extended reality.
Markerless motion capture is pivotal in these types of applications, as suits and hardware are simply not practical for replicating garments and the motion of them on a human body.
Relating to fashion, markerless motion capture is progressively being applied to retail and e-commerce applications. In addition to virtual fitting rooms, mocap can contribute to interactive shopping experiences by allowing customers to navigate virtual stores and interact with products. In e-commerce, the technology enhances product presentation by capturing the movements of models or showcasing products in dynamic ways, making online shopping more interactive and engaging.
There’s also the scope for retailers to use markerless mocap for behavior analysis within physical stores, gaining insights into customer navigation patterns for optimizing store layouts and product placements. It’s an upscaled and digital equivalent to in-store visual merchandising.
The technology also serves security purposes by monitoring and analyzing customer movements to detect suspicious behavior or potential theft, enhancing overall store security.
Markerless motion capture plays a pivotal role here, as suits and markers are simply not viable in situations where members of the public are physically present in-store.
Clinical and healthcare applications for motion capture are becoming extensive, and the sectors have been adopting this method of performance capture for decades already. However, with advancements in markerless methods, practitioners can assist patients more adaptively, as they are free to move without the restrictions of hardware.
In rehabilitation and physical therapy, markerless mocap can assist in analyzing and tracking patients' movements, enabling the creation of personalized treatment plans. Orthopedics benefits from motion capture through gait analysis and assessment of joint range of motion, aiding in the diagnosis and treatment of walking abnormalities and orthopedic conditions. In neurology, the technology is instrumental in assessing motor functions and facilitating stroke rehabilitation by monitoring patients' movements over time. All without the requirement for patients to don suits or footwear – particularly helpful for those with mobility challenges.
Markerless motion capture also finds applications in prosthetics and orthotics, contributing to the customization of devices and footwear based on precise biomechanical data. Virtual reality therapy, incorporating motion capture, creates immersive environments for therapeutic exercises and motor learning. Likewise, surgical planning benefits from motion capture by providing detailed preoperative insights into a patient's anatomy and movement patterns. Additionally, mocap aids in posture analysis, pain management, fall prevention, and contributes to valuable research on human biomechanics for improved healthcare practices and interventions.
Markerless mocap can be applied in security and surveillance to enhance threat detection and monitoring. For example, security systems can utilize motion capture to detect and track unauthorized movement in restricted areas, aiding in intruder detection. It can also be employed in crowd monitoring, enabling the identification of potential threats in crowded public spaces.
Furthermore, automated alert systems can be triggered when motion capture detects unusual activities, ensuring a rapid response to potential security breaches.
Additionally, markerless motion capture is utilized in perimeter security to monitor and detect any breach of defined boundaries. Integration with other security systems such as CCTV cameras and access control enhances the overall security infrastructure.
Markerless motion capture plays a pivotal role in security and surveillance because it is discreet. We’d be very surprised if an intruder actively slips into a mocap suit to make their presence known!
The automotive industry has made significant progress in the development of safety features and vehicle optimisation thanks to the use of motion capture. It can be instrumental in assessing and optimizing vehicle design by analyzing the movements of drivers and passengers, informing the design of motor interiors for enhanced comfort and safety.
Crash test simulations benefit from mocap, as they track the movements of test dummies, providing insights into collision impacts on passengers and aiding in the design of safer vehicle structures.
Additionally, motion capture contributes to understanding driver behavior, refining advanced driver assistance systems, and designing virtual prototyping and simulation environments for efficient product development.
In the manufacturing phase, mocap ensures quality control by overseeing correct component assembly, and maintaining high standards throughout production. It also extends to testing autonomous vehicles, simulating and evaluating their behavior in diverse driving scenarios.
Markerless motion capture advances all of these applications because it allows human and driver movement to be reenacted more authentically – only further enhancing vehicle and safety performance.
The sky is the limit. Markerless mocap takes the restriction of hardware away from applications, so opportunities are endless.
The advancement within software development will continue to grow and expand across industries outside of entertainment and healthcare, penetrating our everyday lives into mass applications – be that security, retail or fitness.
Markerless motion capture will only continue to advance, as 2D to 3D capture improves in quality and precision. Likewise, with artificial intelligence enhancing VFX and wider CG pipelines through real-time rendering, the delivery of shots and CG assets is faster.
As a result, the frequency of content delivered increases and this requires greater demand for more efficient motion capture solutions. Hello, markerless mocap.
Are you ready to get moving with markerless motion capture? You can try the Move One iOS app for free HERE.
