Ada Lovelace was close friends with John Crosse, and knew his father Andrew: the ‘real Frankenstein’. Andrew Crosse apparently created insect life from electricity, stone and water…
Andrew Crosse was a ‘gentleman scientist’ doing science for his own amusement including work improving giant versions of the first batteries called ‘voltaic piles’. He was given the nickname ‘the thunder and lightning man’ because of the way he used the batteries to do giant discharges of electricity with bangs as loud as canons.
He hit the headlines when he appeared to create life from electricity, Frankenstein-like. This was an unexpected result of his experiments using electricity to make crystals. He was passing a current through water containing dissolved limestone over a period of weeks. In one experiment, about a month in, a perfect insect appeared apparently from no-where, and soon after starting to move. More and more insects then appeared over time. He mentioned it to friends, which led to a story in a local paper. It was then picked up nationally. Some of the stories said he had created the insects, and this led to outrage and death threats over his apparent blasphemy of trying to take the position of God.
(Does this start to sound like a modern social networking storm, trolls and all?) In fact he appears to have believed, and others agreed, that the mineral samples he was using must have been contaminated with tiny insect eggs, that just naturally hatched. Scientific results are only accepted if they can be replicated. Others, who took care to avoid contamination couldn’t get the same result. The secret of creating life had not been found.
While Mary Shelley, who wrote Frankenstein, did know Crosse, sadly perhaps, for the story’s sake, he can’t have been the inspiration for Frankenstein as has been suggested, given she wrote it decades earlier!
Paul Curzon, Queen Mary University of London (from the archicve)
Shortly after Ada Lovelace was born, so long before she made predictions about future “creative machines”, Mary Shelley, a friend of her father (Lord Byron), was writing a novel. In her book, Frankenstein, inanimate flesh is brought to life. Perhaps Shelley foresaw what is actually to come, what computer scientists might one day create: artificial life.
Life it may not be, but engineers are now doing pretty well in creating humanoid machines that can do their own thing. Could a machine ever be considered alive? The 21st century is undoubtedly going to be the age of the robot. Maybe it’s time to start thinking about the consequences in case they gain a sense of self.
Frankenstein was obsessed with creating life. In Mary Shelley’s story, he succeeded, though his creation was treated as a “Monster” struggling to cope with the gift of life it was given. Many science fiction books and films have toyed with these themes: the film Blade Runner, for example, explored similar ideas about how intelligent life is created; androids that believe they are human, and the consequences for the creatures concerned.
Is creating intelligent life fiction? Not totally. Several groups of computer scientists are exploring what it means to create non-biological life, and how it might be done. Some are looking at robot life, working at the level of insect life-forms, for example. Others are looking at creating intelligent life within cyberspace.
For 70 years or more scientists have tried to create artificial intelligences. They have had a great deal of success in specific areas such as computer vision and chess playing programs. They are not really intelligent in the way humans are, though they are edging closer. However none of these programs really cuts it as creating “life”. Life is something more than intelligence.
A small band of computer scientists have been trying a different approach that they believe will ultimately lead to the creation of new life forms: life forms that could one day even claim to be conscious (and who would we be to disagree with them if they think they are?) These scientists believe life can’t be engineered in a piecemeal way, but that the whole being has to be created as a coherent whole. Their approach is to build the basic building blocks and let life emerge from them.
A sodarace in action by CS4FN
The outline of the idea could be seen in the game Sodarace, where you could build your own creatures that move around a virtual world, and even let them evolve. One approach to building creatures, such as a spider, would be to try and work out mathematical equations about how each leg moves and program those equations. The alternative artificial life way as used in Sodarace is to instead program up the laws of physics such as gravity and friction and how masses, springs and muscles behave according to those laws. Then you just put these basic bits together in a way that corresponds to a spider. With this approach you don’t have to work out in advance every eventuality (what if it comes to a wall? Or a cliff? Or bumpy ground?) and write code to deal with it. Instead natural behaviour emerges.
The artificial life community believe, not just life-like movement, but life-like intelligence can emerge in a similar way. Rather than programming the behaviour of muscles you program the behaviour of neurones and then build brains out of them. That it turns out has been the key to the machine learning programs that are storming the world of Artificial Intelligence, turning it into an everyday tool. However, if aiming for artificial life, you would keep going and combine it with the basic biochemistry of an immune system, do a similar thing with a reproductive system, and so on.
Want to know more? A wonderful early book is Steve Grand’s: “Creation”, on how he created what at the time was claimed to be “the nearest thing to artificial life yet”… It started life as the game “Creatures”.
Then have a go at creating artificial life yourself (but be nice to it).
Paul Curzon and Peter W McOwan, Queen Mary University of London
Charles Babbage is famous for his amazing technical skills in designing a computer, but also infamous for his apparent spiky and obsessive personality.
He certainly seems to have had poor social skills in that he often immensely irritated the people who funded his work. Part of the reason he never managed to complete a working version of his computer is that his funders pulled the plug on him. If only he had had better people skills to complement his technical skills and creativity, perhaps we would have had computers a century earlier! However, perhaps we should be less harsh. He wasn’t a total social misfit: his salons (Victorian high society parties) were extremely popular, and attended by what would now be considered celebrity A-listers. They often centred around demonstrations of science and engineering wonders, so presumably he could be the life and soul of the party… as long as he had a technological wonder to talk about. The young Ada Lovelace attended one such salon and was enthralled by his machines. Encouraged by her mathematically trained mother, Lady Byron, she studied maths and in 1840 collaborated with Babbage on a description of his Analytical Engine.
More to the point, if you consider the context of Babbage’s life, he suffered extremes of grief. In one year alone, 1827, he buried three of his children as well as his wife. Of his eight children only three survived beyond the age of ten. That was the brutal reality of the pre-antibiotic world.
In this context perhaps it is better to think about his work and achievements, as a response to adversity. That he achieved so much is a triumph of ambition over terrible loss.
Adrian Johnstone, Royal Holloway, University of LondonandPaul Curzon, Queen Mary University of London
Babbage’s wheel from a 3D model by Adrian Johnstone.
Charles Babbage famously designed the first computer: a steam powered contraption that was never built. At its core was something very simple and elegant: a cunning contraption that allowed his machines to store numbers and do arithmetic, all made of Victorian tech: metal wheels and levers.
Babbage had worked as a human computer, for example computing acturial tables that insurance companies use to help calculate risk or payments as well as for the Nautical Almanac used to navigate at sea. He realised that they were full of mistakes and that if a machine could do the computation mistakes could be eliminated. He set about designing and building machines to do that.
Babbage’s Analytical Engine, had it been built, would have been the first working general-purpose computer. The size of a factory, and powered by a steam engine, it was the ultimate cunning contraption. A giant whirring, clanking, puffing mechanical brain. Babbage’s first attempt at mechanical computation, though, was a simpler machine, the Difference Engine. It could only do a fixed, if very useful, kind of calculation. It computed what are called polynomials: patterns of additions and multiplications. It used a complicated adding mechanism. Later, whilst working on designs for his ambitious Analytical Engine he thought of a much simpler adder (described below). His second Difference Engine used this new adder and so needed about 16,000 fewer parts! The Science Museum built it in the twentieth century.
Representing Numbers
First he had to devise a way to represent numbers. Unlike modern computers which use binary, so only two digits, Babbage stuck to decimal. He was going to do all his calculations using our normal ten digits, 0 – 9. But how? His solution was to use metal cog-like wheels. His wheels had 40 teeth corresponding to the digits 0 to 9 repeated four times. To get the idea of how they worked, imagine a wheel with only 10 teeth, each with a digit 0-9 in order, next to a tooth. The wheel lays flat and one digit faces you. That digit is the number that the wheel represents. Turn it one place to the left and it represents one digit higher. Turn it a place to the right and it represents one digit lower.
That is fine for numbers between 0 and 9. For larger numbers, just do as we do: use the decimal place system where the value of a digit changes with its position. That first wheel is in the ones row so stands for 0-9. Put a wheel above it in a 10s row and it stands for 10 times the value shown. If a 5 is facing you on that wheel it stands for 50. You can now represent numbers 0 – 99. Put more wheels on top and you can represent hundreds, thousands, and so on.
It’s a neat way of representing numbers using the system we do (though our numbers run right to left not bottom to top). It makes it not only easy for a machine to manipulate the numbers by turning the wheels, but also easy for a human operator to read the numbers. His Difference Engine used several stacks of them, but Babbage envisaged a gigantic room-sized data store of column after column of these number wheels as the memory of his analytical engine, each column storing one potentially very large number.
A machine that can count
We now have a way to represent numbers but it isn’t yet enough to allow a machine to manipulate them automatically. As it stands it can’t even count properly. We have seen only how to count on one wheel, so only up to 9: every time we turn a crank the 1s wheel turns one notch and so the number represented moves on one. However, we need the other wheels to turn too, but only when the wheel below turns from a 9 to a 0 (so should really become 10). We need a mechanism to carry up to the wheel above. Babbage did this by adding a ridge (a ‘nib’) on the wheel that triggered the carry. When the wheel got to 0, the nib caught against a mechanism above and pushed it, before allowing it to spring back. That nudged the wheel above along one place as required. The 1s wheel was controlled by the crank. The 10s wheel was turned by the 1s wheel moving to 0. The 100s wheel was turned by the 10s wheel moving to 0, and so on. Babbage had a machine that could count!
A machine that can add
The next problem is how to add numbers stored on wheels. Imagine two wheels, interlocked by their teeth, When one is turned it turns the other the same amount. However if you lift one of the wheels they no longer interlock and move independently.
To do addition, the first wheel is used to hold the number to be added. The second holds the total so far: the answer. That answer wheel starts off set to 0. Now, with the wheels interlocked, turn the first wheel one position at a time counting up to the first number of the addition. It turns the answer wheel exactly the same number of positions transferring the number on to it.
Oops. When cogs interlock, the second wheel turns in the opposite direction to the first! Our machine is subtracting! To make it add, you need a connecting wheel between them. The middle wheel then turns backwards, turning the answer wheel forwards as required. With three wheels like this, any number on the first wheel is added to the answer wheel.
To add a second number, just lift the first wheel to disconnect it, spin it back to zero, drop it back in place and turn it to the second number. The answer wheel then holds the sum of both numbers. If you want to add more numbers, just keep doing this, loading one number at a time onto the first wheel. Each time the total passes 10, the carry mechanism passes the 10 onto the higher wheel and the full decimal total is stored up the stack of wheels.
We have a machine for doing addition!
A machine that can multiply
Babbage’s machines could multiply as well. How do you do multiplication on wheels? Well, multiplication is just repeated addition. If you want to work out 5 x 3, then it can be calculated as 5 + 5 + 5. So multiplication can simply be done using the adder, adding the same number over and over again. A counter keeps track of how many times to add it. There are faster ways to multiply though. For the Analytical Engine, Babbage designed an efficient table-based multiplier that he was justifiably very proud of.
Putting it together
Put this together and you have both a number store (a computer memory) and temporary storage areas (registers). You can transfer numbers from one place to another in the machine, and you have the basics of an arithmetic unit that can do calculations. That is about as far as Babbage managed to build. However, he also envisioned programs on punch cards that determined what instruction to execute, mechanisms that allowed instructions to make decisions, and to repeat instructions…everything needed for a general-purpose computer.
Sadly, only parts of his Analytical Engine were ever built, the Victorians did not start the digital age, and we had to wait nearly a century for the first working computers.
Paul Curzon, Queen Mary University of Londonand Adrian Johnstone, Royal Holloway, University of London
We have explained how core rope memory was used as the computer memory storing the Apollo guidance computer program that got us to the moon. A team from the University of Washington came up with a fun craft activity to make your own core memory. It may not fly you to the moon, but is a neat way to store information in a bracelet. Find their activity pages here [EXTERNAL].
What it involves is threading 8 beads onto a string, with a gap between them to form a storage space for bytes of data. Each byte is 8 binary bits (Eight pieces of information, each a 1 or a 0). Each bead represents the position of one bit in your core rope memory. You then take other threads and weave them through the beads. Each thread will store another byte of actual data. Pass the thread through a bead when you want that bead to read 1, or over, when you want that bead to read 0.
Each thread weaving past or through 8 beads can then encode the information for one letter. By adding lots of threads you can store a word or even a sentence on each core rope memory string (perhaps your name, or some secret message).
Using a binary encoding for each letter (so capital letter A would be the 8 bits 01000001 if you’re following this conversion from binary to letters table) you put that letter’s thread through or over each of the 8 beads to ‘spell’ out the letter in binary.
My name is Jo so a core rope memory encoding my name would have only three threads (one to hold the 8 beads and two to spell my name). The second thread would go over, through, over, over, through, over, through, over to spell the capital letter J (01001010). The second thread would go over, through, through, over, through, through, through, through to spell lowercase o (01101111).
Let’s hope you have a slightly longer name so can have more fun time creating your own personalised core rope memory!
Weaving, in the form of the Jacquard loom, with its swappable punch cards controlling the loom’s patterns inspired Charles Babbage. He intended to use the same kind of punch card to store programs in his Analytical Engine, which had it been built would have been the first computer. However, weaving had a much more direct use in computing history. Weaving helped get us to the Moon.
In the 1960s, NASA’s Apollo moon mission needed really dependable computers. It was vital that the programs wouldn’t be corrupted in space. The problem was solved using core rope memory.
Core rope memory was made of small ‘eyelets’ or beads of a metal called ferrite that can be magnetised and copper wire which was woven through some of the eyelets but not others. The ring-shaped magnets were known as magnetic cores. An electrical current passing through the wires made the whole thing work.
Representing binary
Both data and programs in computers are stored as binary: 1s and 0s. Those 1s and 0s can be represented by physical things in the world in lots of different ways. NASA used weaving. A wire that passed through an eyelet would be read as a binary 1 when the current was on but if it passed around the eyelet then it would be read as 0. This meant that a computer program, made up of sequences of 1s and 0s, could be permanently stored by the pattern that was woven. This gave read-only memory. Related techniques were used to create memory that the computer could change too, as the guidance computer needed both.
The memory was woven for NASA by women who were skilled textile workers. They worked in pairs using a special hollow needle to thread the copper wire through one magnetic core and then the other person would thread it back through a different one.
The program was first developed on a computer (the sort that took up a whole room back then) and then translated into instructions for a machine which told the weavers the correct positions for the wire threads. It was very difficult to undo a mistake so a great deal of care was taken to get things right the first time, especially as it could take up to two months to complete one block of memory. Some of the rope weavers were overseen by Margaret Hamilton, one of the women who developed the software used on board the spacecraft, and who went on to lead the Apollo software team.
The world’s first portable computer?
Several of these pre-programmed core rope memory units were combined and installed in the guidance computers of the Apollo mission spacecraft that had to fly astronauts safely to the Moon and back. NASA needed on-board guidance systems to control the spacecraft independently of Mission Control back on Earth. They needed something that didn’t take up too much room or weigh too much, that could survive the shaking and juddering of take-off and background radiation: core rope memory fitted the bill perfectly.
It packed a lot of information (well, not by modern standards! The guidance computer contained only around 70 kilobytes of memory) into a small space and was very robust as it could only break if a wire came loose or one of the ferrite eyelets was damaged (which didn’t happen). To make sure though, the guidance computer’s electronics were sealed from the atmosphere for extra protection. They survived and worked well, guiding the Landing Modules safely onto the Moon.
One small step for man perhaps, but the Moon landings were certainly a giant leap for computing.
Jo Brodie and Paul Curzon, Queen Mary University of London
Charles Babbage invented wonderful computing machines. But he was not very good at explaining things. That’s where Ada Lovelace came in. She is famous for writing a paper in 1843 explaining how Charles Babbage’s Analytical Engine worked – including a big table of formulas which is often described as “the first computer program”.
Charles Babbage invented his mechanical computers to save everyone from the hard work of doing big mathematical calculations by hand. He only managed to build a few tiny working models of his first machine, his difference engine. It was finally built to Babbage’s designs in the 1990s and you can see it in the London Science Museum. It has 8,000 mechanical parts, and is the size of small car, but when the operator turns the big handle on the side it works perfectly, and prints out correct answers.
Babbage invented, but never built, a more ambitious machine, his Analytical Engine. In modern language, this was a general purpose computer, so it could have calculated anything a modern computer can – just a lot more slowly. It was entirely mechanical, but it had all the elements we recognize today – like memory, CPU, and loops.
Lovelace’s paper explains all the geeky details of how numbers are moved from memory to the CPU and back, and the way the machine would be programmed using punched cards.
But she doesn’t stop there – in quaint Victorian language she tells us about the challenges familiar to every programmer today! She understands how complicated programming is:
“There are frequently several distinct sets of effects going on simultaneously; all in a manner independent of each other, and yet to a greater or less degree exercising a mutual influence.”
the difficulty of getting things right:
“To adjust each to every other, and indeed even to perceive and trace them out with perfect correctness and success, entails difficulties whose nature partakes to a certain extent of those involved in every question where conditions are very numerous and inter-complicated.”
and the challenge of making things go faster:
“One essential object is to choose that arrangement which shall tend to reduce to a minimum the time necessary for completing the calculation.”
She explains how computing is about patterns:
“it weaves algebraical patterns just as the Jacquard-loom weaves flowers and leaves”.
and inventing new ideas
“We might even invent laws … in an arbitrary manner, and set the engine to work upon them, and thus deduce numerical results which we might not otherwise have thought of obtaining”.
and being creative. If we knew the laws for composing music:
“the engine might compose elaborate and scientific pieces of music of any degree of complexity or extent.”
Alan Turing famously asked if a machine can think – Ada Lovelace got there first:
“The Analytical Engine has no pretensions whatever to originate anything. It can do whatever we know how to order it to perform.”
Wow, pretty amazing, for someone born 200 years ago.
Ursula Martin, University of Oxford(From the archive)
Charles Dickens is famous for his novels highlighting Victorian social injustice. Despite what people say, art and science really do mix, and Dickens certainly knew some computer science. In his classic novel about the French Revolution, A Tale of Two Cities, one of his characters relies on some computer science based knitting.
Dickens actually moved in the same social circles as Charles Babbage, the Victorian inventor of the first computer (which he designed but unfortunately never managed to build) and Ada Lovelace the mathematician who worked with him on those first computers. They went to the same dinner parties and Dickens will have seen Babbage demonstrate his prototype machines. An engineer in Dickens novel, Little Dorrit, is even believed to be partly based on Babbage. Dickens was probably the last non-family member to visit Ada before she died. She asked him to read to her, choosing a passage from his book Dombey and Son in which the son, Paul Dombey, dies. Like Ada, Paul Dombey had suffered from illness all his life.
So Charles Dickens had lots of opportunity to learn about algorithms! His novel ‘A Tale of Two Cities’ is all about the French Revolution, but lurking in the shadows is some computer science. One of the characters, a revolutionary called Madame Defarge takes the responsibility of keeping a register of all those people who are to be executed once the revolution comes to pass: the aristocrats and “enemies of the people”. Of course in the actual French Revolution lots of aristocrats were guillotined precisely for being enemies of the new state.
Now Madame Defarge could have just tried to memorize the names on her ‘register’ as she supposedly has a great memory, but the revolutionaries wanted a physical record. That raises the problem, though, of how to keep it secret, and that is where the computer science comes in. Madame Defarge knits all the time and so she decides to store the names in her knitting.
“Knitted, in her own stitches and her own symbols, it will always be as plain to her as the sun. Confide in Madame Defarge. It would be easier for the weakest poltroon that lives, to erase himself from existence, than to erase one letter of his name or crimes from the knitted register of Madame Defarge.”
Computer scientists call this Steganography: hiding information or messages in plain sight, so that no one suspects they are there at all. Modern forms of steganography include hiding messages in the digital representation of pictures and in the silences of a Skype conversation.
Madame Defarge didn’t of course just knit French words in the pattern like a victorian scarf version of a T-shirt message. It wouldn’t have been very secret if anyone looking at the resulting scarf could read the names. So how to do it? In fact, knitting has been used as a form of steganography for real. One way was for a person to take a ball of wool and mark messages down it in Morse Code dots and dashes. The wool was then knitted into a jumper or scarf. The message is hidden! To read it you unpick it all and read the morse code back off the wool.
The names were “Knitted, in her own stitches and her own symbols”
That wouldn’t have worked for Madame Defarge though. She wanted to add the names to the register in plain view of the person as they watched and without them knowing what she was doing. She therefore needed the knitting patterns themselves to hold the code. It was possible because she was both a fast knitter and sat knitting constantly so it raised no suspicion. The names were therefore, as Dickens writes “Knitted, in her own stitches and her own symbols”
She used a ‘cipher’ and that brings in another area of computer science: encryption. A cipher is just an algorithm – a set of rules to follow – that converts symbols in one alphabet (letters) into different symbols. In Madame Defarge’s case the new symbols were not written but knitted sequences of stitches. Only if you know the algorithm, and a secret ‘key’ that was used in the encryption, can you convert the knitted sequences back into the original message.
In fact both steganography and encryption date back thousands of years (computer science predates computers!), though Charles Dickens may have been the first to use knitting to do it in a novel. The Ancient Greeks used steganography. In the most famous case a message was written on a slave’s shaved head. They then let the hair grow back. The Romans knew about cryptographic algorithms too and one of the most famous ciphers is called the Caesar cipher as Julius Caesar used it when writing letters…even in Roman times people were worried about the spies reading their equivalent of emails.
Dickens didn’t actually describe the code that Madame Defarge was using so we can only guess…but why not see that as an opportunity and (if you can knit) why not invent a way yourself. If you can’t knit then learn to knit first and then invent one! Somehow you need a series of stitches to represent each letter of the alphabet. In doing so you are doing algorithmic thinking with knitting. You are knitting your way to being a computer scientist.
Paul Curzon, Queen Mary University of London (From the archive)
Avengers: Age of Ultron is the latest film about robots or artificial intelligences (AI) trying to take over the world. AI is becoming ever present in our lives, at least in the form of software tools that demonstrate elements of human-like intelligence. AI in our mobile phones apply and adapt their rules to learn to serve us better, for example. But fears of AI’s potential negative impact on humanity remain as seen in its projection into characters like Ultron, a super-intelligence accidentally created by the Avengers.
But what relation do the evil AIs of the movies have to scientific reality? Could an AI take over the world? How would it do it? And why would it want to? AI movie villains need to consider the whodunit staples of motive and opportunity.
Motive? What motive?
Let’s look at the motive. Few would say Intelligence in itself unswervingly leads to a desire to rule the world. In movies AI are often driven by self preservation, a realisation that fearful humans might shut them down. But would we give our AI tools cause to feel threatened? They provide benefits for us and there also seems little reason in creating a sense of self-awareness in a system that searches the web for the nearest Italian restaurant, for example.
Another popular motive for AIs’ evilness is their zealous application of logic. In Ultron’s case the goal of protecting the earth can only be accomplished by wiping out humanity. This destruction by logic is reminiscent of the notion that a computer would select a stopped clock over one that is two seconds slow, as the stopped clock is right twice a day whereas the slow one is never right. Ultron’s plot motivation, based on brittle logic combined with indifference to life, seems at odds with todays AI systems that reason mathematically with uncertainty and are built to work safely with users.
Opportunity Knocks
When we consider an AI’s opportunity to rule the world we are on somewhat firmer ground. The famous Turning Test of machine intelligence was set up to measure a particular skill – the ability to conduct a believable conversation. The premise being that if you can’t tell the difference between AI and human skill, the AI has passed the test and should be considered as intelligent as humans.
So what would a Turing Test for the ‘skill’ of world domination look like? To explore that we need to compare the antisocial AI behaviours with the attributes expected of human world domination. World dominators need to control important parts of our lives, say our access to money or our ability to buy a house. AI does that already – lending decisions are frequently made by an AI sifting through mountains of information to decide your credit worthiness. AIs now trade on the stock market too.
An overlord would give orders and expect them to be followed. Anyone who has stood helplessly at a shop’s self-service till as it makes repeated bagging related demands of them already knows what it feels like to be bossed about by AIs.
Kill Bill?
Finally, no megalomaniac Hollywood robot would be complete without at least some desire to kill us. Today military robots can identify targets without human intervention. It is currently a human controller that gives permission to attack but it’s not a stretch to say that the potential to auto kill exists in these AIs, but we would need to change the computer code to allow it.
These examples arguably show AI in control in limited but significant parts of life on earth, but to truly dominate the world, movie style, these individual AIs would need to start working together to create a synchronised AI army – that bossy self-service till talking to your health monitor and denying selling you beer, then both ganging up with a credit scoring system to only raise your credit limit if you both buy a pair of trainers with a built in GPS tracker and only eat the kale from your smart fridge but after the shoe data shows you completed the required five mile run.
It’s a worrying picture but fortunately I think it’s an unlikely one. Engineers worldwide are developing the Internet of things, networks connecting all manner of devices together to create new services. These are pieces of a jigsaw that would need to join together and form a big picture for total world domination. It’s an unlikely situation – too much has too fall into place and work together. It’s a lot like the infamous plot-hole in Independence Day – where an Apple Mac and an alien spaceship’s software inexplicably have cross-platform compatibility. [See video below for a possible answer!]
Our earthly AI systems are written in a range of computer languages, hold different data in different ways and use different and non-compatible rule sets and learning techniques. Unless we design them to be compatible there is no reason why adding two safely designed AI systems, developed by separate companies for separate services would spontaneously blend to share capabilities and form some greater common goal without human intervention.
So could AIs, and the robot bodies containing them, pass the test and take over the world? Only if we humans let them, and help them a lot. Why would we?
Theatre producers, radio directors and film-makers have been trying to create realistic versions of natural sounds for years. Special effects teams break frozen celery stalks to mimic breaking bones, smack coconut shells on hard packed sand to hear horses gallop, rustle cellophane for crackling fire. Famously, in the first Star Wars movie the Wookie sounds are each made up of up to six animal clips combined, including a walrus! Sometimes the special effect people even record the real thing and play it at the right time! (Not a good idea for the breaking bones though!) The person using props to create sounds for radio and film is called a Foley artist, named after the work of Jack Donovan Foley in the 1920’s. Now the Foley artist is drawing on digital technology to get the job done.
Designing sounds
Sound designers have a hard job finding the right sounds. So how about creating sound automatically using algorithms? Synthetic sound! Research into sound creation is a hot topic, not just for special effects but also to help understand how people hear and for use in many other sound based systems. We can create simple sounds fairly easily using musical instruments and synthesisers, but creating sounds from nature, animal sounds and speech is much more complicated.
The approaches used to recognize sounds can be the basis of generating sounds too. You can either try and hand craft a set of rules that describe what makes the sound sound the way it does, or you can write algorithms that work it out for themselves.
Paying patterns attention
One method, developed as a way to automatically generate synthetic sound, is based on looking for patterns in the sounds. Computer scientists often create mathematical models to better understand things, as well as to recognize and generate computer versions of them. The idea is to look at (or here listen to) lots of examples of the thing being studied. As patterns become obvious they also start to identify elements that don’t have much impact. Those features are ignored so the focus stays on the most important parts. In doing this they build up a general model, or view, that describes all possible examples. This skill of ignoring unimportant detail is called abstraction, and if you create a general view, a model of something, this is called generalisation: both important parts of computational thinking. The result is a hand-crafted model for generating that sound.
That’s pretty difficult to do though, so instead computer scientists write algorithms to do it for them. Now, rather than a person trying to work out what is, or is not important, training algorithms work it out using statistical rules. The more data they see, the stronger the pattern that emerges, which is why these approaches are often referred to as ‘Big Data’. They rely on number crunching vast data sets. The learnt pattern is then matched against new data, looking for examples, or as the basis of creating new examples that match the pattern.
The rain in train(ing)
Number crunching based on Big Data isn’t the only way though, sometimes general patterns can be identified from knowledge of the thing being investigated. For example, rain isn’t one sound but is made up of lots of rain drops all doing a similar thing. Natural sounds often have that kind of property. So knowledge of a phenomenon can be used to create a basic model to build a generator around. This is an approach Richard Turner, now at Cambridge University, has pioneered, analysing the statistical properties of natural sounds. By creating a basic model and then gradually tweaking it to match the sound-quality of lots of different natural sounds, his algorithms can learn what natural sounds are like in general. Then, given a specific natural ‘training’ sound, it can generate synthetic versions of that sound by choosing settings that match its features. You could give it a recorded sample of real rain, for example. Then his sound processing algorithms apply a bunch of maths that pull out the important features of that particular sound based on the statistical models. With the critical features identified, and plugged in to his general model, a new sound of any length can then be generated that still matches the statistical pattern of, and so sounds like, the original. Using the model you can create lots of different versions of rain, that all still sound like rain, lots of different campfires, lots of different streams, and so-on.
For now, the celery stalks are still in use, as are the walrus clippings, but it may not be long before film studios completely replace their Foley bag of tricks with computerised solutions like Richard’s. One wookie for 3 minutes and a dawn chorus for 5 please.
Become a Foley Artist with Sonic Pi
You can have a go at being a Foley artist yourself. Sonic Pi is a free live-coding synth for music creation that is both powerful enough for professional musicians, but intended to get beginners into live coding: combining programming with composing to make live music.
It was designed for use with a Raspberry Pi computer, which is a cheap way to get started, though works with other computers too. Its also a great, fun way to start to learn to program.
Play with anything, and everything, you find around the house, junk or otherwise. See what sounds it makes. Record it, and then see what it makes you think of out of context. Build up your own library of sounds, labelling them with things they sound like. Take clips of films, mute the sound and create your own soundscape for them. Store the sound clips and then manipulate them in Sonic Pi, and see if you can use them as the basis of different sounds.
Listen to the example sound clips made with Sonic Pi on their website, then start adapting them to create your own sounds, your own music. What is the most ‘natural sound’ you can find or create using Sonic Pi?
Jane Waite and Paul Curzon, Queen Mary University of London.
explores the work of scientists and engineers who are using computers to understand, identify and recreate wild sounds, especially those of birds. We see how sophisticated algorithms that allow machines to learn, can help recognize birds even when they can’t be seen, so helping conservation efforts. We see how computer models help biologists understand animal behaviour, and we look at how electronic and computer generated sounds, having changed music, are now set to change the soundscapes of films. Making electronic sounds is also a great, fun way to become a computer scientist and learn to program.
Subscribe to be notified whenever we publish a new post to the CS4FN blog.
This blog is funded by EPSRC on research agreement EP/W033615/1.
cs4fn 