Category: Movies

100,000 frames – quick draw: how computers help animators create

Film projector with film strip on a coloured rainbow background, from Pixabay

Ben Stephenson of the University of Calgary gives us a guide to the basics of animation.

Film projector with film strip on a coloured rainbow background, from Pixabay
Film projector and film strip image by Gerd Altmann from Pixabay

Animation isn’t a new field – artists have been creating animations for over a hundred years. While the technology used to create those animations has changed immensely during that time, modern computer generated imagery continues to employ some of the same techniques that were used to create the first animations.

The hard work of hand drawing

During the early days of animation, moving images were created by rapidly showing a sequence of still images. Each still image, referred to as a frame, was hand drawn by an artist. By making small changes in each new frame, characters were created that appeared to be walking, jumping and talking, or doing anything else that the artist could imagine.

In order for the animation to appear smooth, the frames need to be displayed quickly – typically at around 24 frames each second. This means that one minute of animation required artists to draw over 1400 frames. That means that the first feature-length animated film, a 70-minute Argentinean film called The Apostle, required over 100,000 frames to create.

Creating a 90-minute movie, the typical feature length for most animated films, took almost 130,000 hand drawn frames. Despite these daunting numbers, many feature length animated movies have been created using hand-drawn images.

Drawing with data

Today, many animations are created with the assistance of computers. Rather than simply drawing thousands of images of one character using a computer drawing program, artists can create one mathematical model to represent that character, from which all of his or her appearances in individual frames are generated. Artists manipulate the model, changing things like the position of the character’s limbs (so that the character can be made to walk, run or jump) and aspects of the character’s face (so that it can talk and express emotions). Furthermore, since the models only exist as data on a computer they aren’t confined by the physical realities that people are. As such, artists also have the flexibility to do physically impossible things such as shrinking, bending or stretching parts of a character. Remember Elastigirl, the stretchy mum in The Incredibles? All made of maths.

Once all of the mathematical models have been positioned correctly, the computer is used to generate an image of the models from a specific angle. Just like the hand-drawn frames of the past, this computer- generated image becomes one frame in the movie. Then the mathematical models representing the characters are modified slightly, and another frame is generated. This process is repeated to generate all of the frames for the movie.

The more things change

You might have noticed that, despite the use of computers, the process of generating and displaying the animation remains remarkably similar to the process used to create the first animations over 100 years ago. The animation still consists of a collection of still images. The illusion of smooth movement is still achieved by rapidly displaying a sequence of frames, where each frame in the sequence differs only slightly from the previous one.

The key difference is simply that now the images may be generated by a computer, saving artists from hand drawing over 100,000 copies of the same character. Hand-drawn animation is still alive in the films of Studio Ghibli and Disney’s recent The Princess and the Frog, but we wonder if the animators of hand-drawn features might be tempted to look over at their fellow artists who use computers and shake an envious fist. A cramped fist, too, probably.

This article was originally published on the CS4FN website and also appears on page 3 of issue 11 of the CS4FN magazine “Computer animation proudly presents…” which you can download as a free PDF along with all of our other free material at our CS4FN downloads site.

Related Magazine …

This blog is funded through EPSRC grant EP/W033615/1.

Hidden Figures: NASA’s brilliant calculators #BlackHistoryMonth

Full Moon and silhouetted tree tops

by Paul Curzon, Queen Mary University of London

Full Moon with a blue filter
Full Moon image by PIRO from Pixabay

NASA Langley was the birthplace of the U.S. space program where astronauts like Neil Armstrong learned to land on the moon. Everyone knows the names of astronauts, but behind the scenes a group of African-American women were vital to the space program: Katherine Johnson, Mary Jackson and Dorothy Vaughan. Before electronic computers were invented ‘computers’ were just people who did calculations and that’s where they started out, as part of a segregated team of mathematicians. Dorothy Vaughan became the first African-American woman to supervise staff there and helped make the transition from human to electronic computers by teaching herself and her staff how to program in the early programming language, FORTRAN.

FORTRAN code on a punched card, from Wikipedia.

The women switched from being the computers to programming them. These hidden women helped put the first American, John Glenn, in orbit, and over many years worked on calculations like the trajectories of spacecraft and their launch windows (the small period of time when a rocket must be launched if it is to get to its target). These complex calculations had to be correct. If they got them wrong, the mistakes could ruin a mission, putting the lives of the astronauts at risk. Get them right, as they did, and the result was a giant leap for humankind.

See the film ‘Hidden Figures’ for more of their story (trailer below).

This story was originally published on the CS4FN website and was also published in issue 23, The Women Are (Still) Here, on p21 (see ‘Related magazine’ below).

See more in ‘Celebrating Diversity in Computing‘

We have free posters to download and some information about the different people who’ve helped make modern computing what it is today.

Screenshot showing the vibrant blue posters on the left and the muted sepia-toned posters on the right

Or click here: Celebrating diversity in computing

Related Magazine …

This blog is funded through EPSRC grant EP/W033615/1.

Full metal jacket: the fashion of Iron Man

by Peter W McOwan and Paul Curzon, Queen Mary University of London

Spoiler Alert

Black and White drawing of Iron Man
Image by more photos boosty.to/victoria_art_music from Pixabay

Industrialist Tony Stark always dresses for the occasion, even when that particular occasion happens to be a fight with the powers of evil. His clothes are driven by computer science: the ultimate in wearable computing.

In the Iron Man comic and movie franchise Anthony Edward Stark, Tony to his friends, becomes his crime fighting alter ego by donning his high tech suit. The character was created by Marvel comic legend Stan Lee and first hit the pages in 1963. The back story tells how industrial armaments engineer and international playboy Stark is kidnapped and forced to work to develop new forms of weapons, but instead manages to escape by building a flying armoured suit.

Though the escape is successful Stark suffers a major heart injury during the kidnap ordeal, becoming dependant on technology to keep him alive. The experience forces him to reconsider his life, and the crime avenging Iron Man is born. Lee’s ‘businessman superhero’ has proved extremely popular and in recent years the Iron Man movies, starring Robert Downey Jr, have been box office hits. But as Tony himself would be the first to admit, there is more than a little computer science supporting Iron Man’s superhero standing.

Suits you

The Iron Man suit is an example of a powered exoskeleton. The technology surrounding the wearer amplifies the movement of the body, a little like a wearable robot. This area of research is often called ‘human performance augmentation’ and there are a number of organisations interested in it, including universities and, unsurprisingly, defence companies like Stark Industries. Their researchers are building real exoskeletons which have powers uncannily like those of the Iron Man suit.

To make the exoskeleton work the technology needs to be able to accurately read the exact movements of the wearer, then have the robot components duplicate them almost instantly. Creating this fluid mechanical shadow means the exoskeleton needs to contain massive computing power, able to read the forces being applied and convert them into signals to control the robot servo motors without any delay. Slow computing would cause mechanical drag for the wearer, who would feel like they were wading through treacle. Not a good idea when you’re trying to save the world.

Pump it up

Humans move by using their muscles in what are called antagonistic pairs. There are always two muscles on either side of the joint that pull the limb in different directions. For example, in your upper arm there are the muscles called the biceps and the triceps. Contracting the biceps muscle bends your elbow up, and contracting your triceps straightens your elbow back. It’s a clever way to control biological movement using just a single type of shortening muscle tissue rather than needing one kind that shortens and another that lengthens.

In an exoskeleton, the robot actuators (the things that do the moving) take the place of the muscles, and we can build these to move however we want, but as the robot’s movements need to shadow the person’s movements inside, the computer needs to understand how humans move. As the human bends their elbow to lift up an object, sensors in the exoskeleton measure the forces applied, and the onboard computer calculates how to move the exoskeleton to minimise the resulting strain on the person’s hand. In strength amplifying exoskeletons the actuators are high pressure hydraulic pistons, meaning that the human operators can lift considerable weight. The hydraulics support the load, the humans movements provide the control.

I knew you were going to do that

It is important that the human user doesn’t need to expend any effort in moving the exoskeleton; people get tired very easily if they have to counteract even a small but continual force. To allow this to happen the computer system must ensure that all the sensors read zero force whenever possible. That way the robot does the work and the human is just moving inside the frame. The sensors can take thousands of readings per second from all over the exoskeleton: arms, legs, back and so on.

This information is used to predict what the user is trying to do. For example, when you are lifting a weight the computer begins by calculating where all the various exoskeleton ‘muscles’ need to be to mirror your movements. Then the robot arm is instructed to grab the weight before the user exerts any significant force, so you get no strain but a lot of gain.

Flight suit?

Exoskeleton systems exist already. Soldiers can march further with heavy packs by having an exoskeleton provide some extra mechanical support that mimics their movements. There are also medical applications that help paralysed patients walk again. Sadly, current exoskeletons still don’t have the ability to let you run faster or do other complex activities like fly.

Flying is another area where the real trick is in the computer programming. Iron Man’s suit is covered in smart ‘control surfaces’ that move under computer control to allow him to manoeuvre at speed. Tony Stark controls his suit through a heads-up display and voice control in his helmet, technology that at least we do have today. Could we have fully functional Iron Man suits in the future? It’s probably just a matter of time, technology and computer science (and visionary multi-millionaire industrialists too).

More on …

Related Magazines …

This blog is funded through EPSRC grant EP/W033615/1.

A Wookie for three minutes please – how Foley artists can manipulate natural and synthesised sounds for film, TV and radio

by Jane Waite and Paul Curzon, Queen Mary University of London.
This story was originally published on CS4FN and in an issue of the magazine (see below).

Theatre producers, radio directors and film-makers have been trying to create realistic versions of natural sounds for years. Special effects teams break frozen celery stalks to mimic breaking bones, smack coconut shells on hard packed sand to hear horses gallop, rustle cellophane for crackling fire. Famously, in the first Star Wars movie the Wookie sounds are each made up of up to six animal clips combined, including a walrus! Sometimes the special effect people even record the real thing and play it at the right time! (Not a good idea for the breaking bones though!) The person using props to create sounds for radio and film is called a Foley artist, named after the work of Jack Donovan Foley in the 1920’s. Now the Foley artist is drawing on digital technology to get the job done.

Black and white photo of a walrus being offered a fish, with one already in its mouth
“Are you sure that’s a microphone?” Walrus photo by Kabomani-Tapir from Pixabay

Designing sounds

Sound designers have a hard job finding the right sounds. So how about creating sound automatically using algorithms? Synthetic sound! Research into sound creation is a hot topic, not just for special effects but also to help understand how people hear and for use in many other sound based systems. We can create simple sounds fairly easily using musical instruments and synthesisers, but creating sounds from nature, animal sounds and speech is much more complicated.

The approaches used to recognize sounds can be the basis of generating sounds too. You can either try and hand craft a set of rules that describe what makes the sound sound the way it does, or you can write algorithms that work it out for themselves.

Paying patterns attention

One method, developed as a way to automatically generate synthetic sound, is based on looking for patterns in the sounds. Computer scientists often create mathematical models to better understand things, as well as to recognize and generate computer versions of them. The idea is to look at (or here listen to) lots of examples of the thing being studied. As patterns become obvious they also start to identify elements that don’t have much impact. Those features are ignored so the focus stays on the most important parts. In doing this they build up a general model, or view, that describes all possible examples. This skill of ignoring unimportant detail is called abstraction, and if you create a general view, a model of something, this is called generalisation: both important parts of computational thinking. The result is a hand-crafted model for generating that sound.

That’s pretty difficult to do though, so instead computer scientists write algorithms to do it for them. Now, rather than a person trying to work out what is, or is not important, training algorithms work it out using statistical rules. The more data they see, the stronger the pattern that emerges, which is why these approaches are often referred to as ‘Big Data’. They rely on number crunching vast data sets. The learnt pattern is then matched against new data, looking for examples, or as the basis of creating new examples that match the pattern.

The rain in train(ing)

Number crunching based on Big Data isn’t the only way though, sometimes general patterns can be identified from knowledge of the thing being investigated. For example, rain isn’t one sound but is made up of lots of rain drops all doing a similar thing. Natural sounds often have that kind of property. So knowledge of a phenomenon can be used to create a basic model to build a generator around. This is an approach Richard Turner, now at Cambridge University, has pioneered, analysing the statistical properties of natural sounds. By creating a basic model and then gradually tweaking it to match the sound-quality of lots of different natural sounds, his algorithms can learn what natural sounds are like in general. Then, given a specific natural ‘training’ sound, it can generate synthetic versions of that sound by choosing settings that match its features. You could give it a recorded sample of real rain, for example. Then his sound processing algorithms apply a bunch of maths that pull out the important features of that particular sound based on the statistical models. With the critical features identified, and plugged in to his general model, a new sound of any length can then be generated that still matches the statistical pattern of, and so sounds like, the original. Using the model you can create lots of different versions of rain, that all still sound like rain, lots of different campfires, lots of different streams, and so-on.

For now, the celery stalks are still in use, as are the walrus clippings, but it may not be long before film studios completely replace their Foley bag of tricks with computerised solutions like Richard’s. One wookie for 3 minutes and a dawn chorus for 5 please.

 

Become a Foley Artist with Sonic Pi

You can have a go at being a Foley artist yourself. Sonic Pi is a free live-coding synth for music creation that is both powerful enough for professional musicians, but intended to get beginners into live coding: combining programming with composing to make live music.

It was designed for use with a Raspberry Pi computer, which is a cheap way to get started, though works with other computers too. Its also a great, fun way to start to learn to program.

Play with anything, and everything, you find around the house, junk or otherwise. See what sounds it makes. Record it, and then see what it makes you think of out of context. Build up your own library of sounds, labelling them with things they sound like. Take clips of films, mute the sound and create your own soundscape for them. Store the sound clips and then manipulate them in Sonic Pi, and see if you can use them as the basis of different sounds.

Listen to the example sound clips made with Sonic Pi on their website, then start adapting them to create your own sounds, your own music. What is the most ‘natural sound’ you can find or create using Sonic Pi?

 

 

This article was also originally published in issue 21 of the CS4FN magazine ‘Computing Sounds Wild’ on p16. You can download a PDF copy of Issue 21, as well as all of our previous published material, free, at the CS4FN downloads site.

Computing Sounds Wild explores the work of scientists and engineers who are using computers to understand, identify and recreate wild sounds, especially those of birds. We see how sophisticated algorithms that allow machines to learn, can help recognize birds even when they can’t be seen, so helping conservation efforts. We see how computer models help biologists understand animal behaviour, and we look at how electronic and computer generated sounds, having changed music, are now set to change the soundscapes of films. Making electronic sounds is also a great, fun way to become a computer scientist and learn to program.

Front cover of CS4FN Issue 21 – Computing sounds wild

 

 

Return of the killer robot? Evil scientist?! Helpless woman?!?

by Paul Curzon, Queen Mary University of London

Digital blond copyright www.istock.com 439194

In an early issue of the cs4fn magazine we looked at how robots, female scientists and women generally were portrayed in 20th century science fiction movies. It wasn’t great. Robots were killers, scientists evil. Computer scientist’s were introverted and thickheaded. Women were either sexbots or helpless love interest to be rescued by the hunky male star. 1995’s film Hackers was about as good as it got. At last a woman had expert computing skills. It’s hardly surprising some girls are led to believe computing isn’t for them with a century-long conspiracy aiming to convince them their role in life is to be helpless.

As our area on women in computing shows the truth is far more interesting. Women have always played a big part in the development of modern technology. So have things improved in films? There are more films with strong action-heroine stars now, though few films pass the Bechdel test: do two women ever talk together about anything other than a man? So can we at least find any 21st century films with realistic main character roles for women as computer experts? Here goes…

1999-2003: Matrix Trilogy

Hero Neo discovers reality isn’t what it seems. It is all a virtual reality. Trinity is there to be his romantic interest – she’s been told by the Oracle that she will fall in love with the “One” (that’s him). It’s not looking good. In film 2 Neo has to save her. Oh dear. At least she is supposed to be a super-hacker famous for cracking an uncrackable database. Oh well.

2009: The Girl With the Dragon Tattoo

This is the story of super-hacker Lisbeth Salander. Both emotionally and sexually abused as a child she looks after herself, and that includes teaching herself to be an expert with computers. She uses her immense skills to get what she wants. She is cool and clever and absolutely not willing to let the men treat her as a victim. Wonderful.

2014: Captain America: The Winter Soldier

This film is all about a male hunk, so it’s not looking good, but then early on we see Agent Natasha Romanoff, (also known as superheroine the Black Widow). She is the brains to Captain America’s brawn and from the start she is clearly the expert with computers. While Captain America beats people up, her mission is to collect data. Let’s hope she gets her own film series!

2015: Star Wars: Episode VII – the Force Awakens

Rey is a scavenger with engineering skills. She is very smart, and can look after herself without expecting men to save her. She’s not a hacker! Instead, she creates and mends things. She repurposes parts she finds on wrecked spaceships to sell to survive. She learnt her engineering skills tinkering in old ships and fixes the Millennium Falcon’s electro-mechanical problems. She is even the main character of the whole film!

 

There are plenty of moronic films, made by men who can’t portray women in remotely realistic ways, but at least things are a bit better than they were last century. The women are already here in the real world. They are slowly getting there in the movies. Let’s just hope the trend speeds up, and we have more female leads who create things, like the real female computer scientists.

Email your reviews of female characters in science fiction films (good or bad) to cs4fn@eecs.qmul.ac.uk

More on …

Cyber Security at the Movies: Guardians of the Galaxy (Fail Secure security)

by Paul Curzon, Queen Mary University of London

[Spoiler Alert]

Guardians of the Galaxy Poster

If you are so power hungry you can’t stand the idea of any opposition; if you want to make a grab for total power, so decide to crush everyone in your way, then you might want to think about the security of your power supply first. Luckily, all would-be dictators who crush everyone who gets in their way as they march towards total domination of the galaxy, tend to be very naive about cyber-security.

Take Ronan the Accuser in the original Guardian of the Galaxy film. He’s a villain with a religious streak, whose belief that strength is virtue and weakness is sin leads to his totally corrupted morality. To cut to the guts of the story he manages to get the “Infinity Stone” that gives unimaginable power to its owner. With it he can destroy anyone who gets in his way so sets out to do so.

Luckily for the Galaxy, good-guy Peter Quill, or Star-Lord as he wants to be known, and his fellow Guardians have a plan. More to the point they have Gamora. She is an assassin originally sent to kill Quill, but who changes sides early on. She is an insider who knows how Ronan’s security system works, and it has a flaw: its big, heavy security doors into his control room.

Security Lesson 1. It should still be secure even when the other side know everything about how it works. If your security relies on no one knowing, its almost certainly bad security!

Once inside his ship, to get to Ronan the Guardians will need to get through those big heavy security doors. Now once upon a time big, heavy doors were locked and barred with big, heavy bolts. Even in Roman times you needed a battering ram to get in to a besieged city if they had shut the doors before you got there. Nowadays, how ever big and heavy the door, you may just need some cyber skills to get in if the person designing it didn’t think it through.

Electromagnetic locks are used all over the place and they give some big advantages, such as the fact that they mean you can program who is and isn’t allowed entry. Want to keep someone out – you can just cancel their keycard in the system. They are held locked by electromagnets: magnets that are switched on and off using an electric current. That means computers can control them. As the designer of an electromagnetic lock you have a choice, though. You can make them either “fail safe” or “fail secure”. With a fail safe lock, when the power goes, the doors automatically unlock. With fail secure, instead they lock. Its just a matter of whether the magnet is holding the door open or closed. Which you choose when designing the lock depends on your priorities.

Fail safe is a good idea, for example, if you want people to be able to escape in an emergency. If a fire cuts the electricity you want everyone to still be able to get out, not be locked in with no chance of escape. Fail secure on the other hand is good if you don’t want thieves to be able to get in just by cutting the power. The magnets hold the bolts open, so when the power goes, the spring shut.

Security Lesson 2. If you want the important things to stay secure, you need a fail secure system.

This is Ronan’s problem. Zamora knows that if you cut the power supply then the doors preventing attackers getting to him just open! He needed a fail secure door, but instead had a fail safe one installed. On such small things are galaxies won and lost! All Zamora has to do is cut the power and they can get to him. This of course leads to the next flaw in his security system. It wouldn’t have mattered if the power supply was on the secure side of that door, but it wasn’t. Ronan locks himself in and Zamora can cut the power from the outside … Dhurr!

There is one last thing that could have saved Ronan. It needed an uninterruptible power supply.

Security Lesson 3. If your system is reliant on the power supply, whether a door, your data, your control system or your life-support system, then it should keep going even if the power is switched off.

After all, what if the space ships cleaners (you never see them but they must be there somewhere!) unplug the door lock by mistake just because they need somewhere to plug in the hoover.

The solution is simple: use an “uninterruptible power supply”. They are just very fast electricity storage systems that immediately and automatically take over if the main power cuts out. The biggest on Earth keeps the power going for a whole city in Alaska (you do not want to lose the power running your heating mid-winter if you live in Alaska!). Had Ronan’s doors had a similar system, the doors wouldn’t have just opened as the power would not have been cut off.It’s always the small details that matter in cyber security (and in successfully destroying your enemies and so ruling the universe). As with all computational thinking, you have to think about everything in advance. If you don’t look after your power supply, then you may well lose all your power over the galaxy too (and your life)!

More on …