Think of the perfect robot companion. A robot you can hang out with, chat to and who understands how you feel. Robots can already understand some of what we say and talk back. They can even respond to the emotions we express in the tone of our voice. But, what about body language? We also show how we feel by the way we stand, we describe things with our hands and we communicate with the expressions on our faces. Could a robot use body language to show that it understands how we feel? Could a robot show empathy?
If a robot companion did show this kind of empathetic body language we would likely feel that it understood us, and shared our feelings and experiences. For robots to be able to behave like this though, we first need to understand more about how humans use movement to show empathy with one another.
Think about how you react when a friend talks about their headache. You wouldn’t stay perfectly still. But what would you do? We’ve used motion capture to track people’s movements as they talk to each other. Motion capture is the technology used in films to make computer-animated creatures like Gollum in Lord of the Rings, or the Apes in the Planet of the Apes. Lots of cameras are used together to create a very precise computer model of the movements being recorded. Using motion capture, we’ve been able to see what people actually do when chatting about their experiences.
It turns out that we share our understanding of things like a headache by performing it together. We share the actions of the headache as if we have it ourselves. If I hit my head, wince and say ‘ouch’, you might wince and say ‘ouch’ too – you give a multimodal performance, with actions and words, to show me you understand how I feel.
So should we just program robots to copy us? It isn’t as simple as that. We don’t copy exactly. A perfect copy wouldn’t show understanding of how we feel. A robot doing that would seem like a parrot, repeating things without any understanding. For the robot to show that it understands how you feel it must perform a headache like it owns it – as though it were really theirs! That means behaving in a similar way to you; but adapted to the unique type of headache it has.
Designing the way robots should behave in social situations isn’t easy. If we work out exactly how humans interact with each other to share their experiences though, we can use that understanding to program robot companions. Then one day your robot friend will be able to hang out with you, chat and show they understand how you feel. Just like a real friend.
multimodal = two or more different ways of doing something. With communication that might be spoken words, facial expressions and hand gestures.
We have recently written about the AMPER project which uses a tablet-based AI tool / robot to support people with dementia and their carers. It prompts the person to discuss events from their younger life and adapts to their needs. We also linked this with information about the types of careers people working in this area might do – the examples given were for a project based in the Netherlands called ‘Dramaturgy for Devices’ – using lessons learned from the study of theatre and theatrical performances in designing social robots so that their behaviour feels more natural and friendly to the humans who’ll be using them.
Greg Michaelson is an Emeritus professor of computer science at Heriot-Watt University in Edinburgh. He is also a novelist and a short story writer. He wrote this story for CS4FN.
“Come on!” called Alice, taking the coat off the peg. “We’re going to be late!”
“Do I have to?” said Henry, emerging from the front room.
“Yes,” said Alice, handing him the coat. “Of course you have to go. Here. Put this on.”
“But we’re playing,” said Henry, wrestling with the sleeves.
“Too bad,” said Alice, straightening the jacket and zipping it up. “It’ll still be there when we get back.”
“Not if someone knocks it over,” said Henry, picking up a small model dinosaur from the hall table. “Like last time. Why can’t we have electric games like you did?”
“Electronic games,” said Alice, doing up her buttons. “Not electric. No one has them anymore. You know that.”
“Were they really digital?” asked Henry, fiddling with the dinosaur.
“Yes,” said Alice, putting on her hat. “Of course they were digital.”
“But the telephone’s all right,” said Henry.
“Yes,” said Alice, checking her makeup in the mirror. “It’s analogue.”
“And radio. And record players. And tape recorders. And television,” said Henry.
“They’re all analogue now,” said Alice, putting the compact back into her handbag. “Anything analogue’s fine. Just not digital. Stop wasting time! We’ll be late.”
“Why does it matter if we’re late?” asked Henry, walking the dinosaur up and down the hall table.
“They’ll notice,” said Alice. “We don’t want to get another warning. Put that away. Come on.”
“Why don’t the others have to go?” asked Henry, palming the dinosaur.
“They went last Sunday,” said Alice, opening the front door. “You said you didn’t want to go. We agreed I’d take you today instead.”
“Och, granny, it’s so boring…” said Henry.
They left the house and walked briskly to the end of the street. Then they crossed the deserted park, following the central path towards the squat neo-classical stone building on the far side.
“Get a move on!” said Alice, quickening the pace. “We really are going to be late.”
————————-
Henry really hadn’t paid enough attention at school. He knew that Turing Machines were named for Alan Turing, the first Martyr of the Digital Age. And he knew that a Turing Machine could work out sums, a bit like a school child doing arithmetic. Only instead of a pad of paper and a pencil, a Turing Machine used a tape of cells. And instead of rows of numbers and pluses and minuses on a page, a Turing Machine could only put one letter on each cell, though it could change a letter without having to actually rub it out. And instead of moving between different places on a piece of paper whenever it wanted to, and maybe doodling in between the sums, a Turing Machine could only move the tape left and right one cell at a time. But just like a school child getting another pad from the teacher when they ran out of paper, the Turing Machine could somehow add another empty cell whenever it got to the end of the tape.
————————-
When they reached the building, they mounted the stone staircase and entered the antechamber through the central pillars. Just inside the doorway, Alice gave their identity cards to the uniformed guard.
“I see you’re a regular,” she said approvingly to Alice, checking the ledger. “But you’re not,” sternly to Henry.
Henry stared at his shoes.
“Don’t leave it so long, next time,” said the guard, handing the cards back to Alice. “In you go. They’re about to start. Try not to make too much noise.”
Hand in hand, Alice and Henry walked down the broad corridor towards the central shrine. On either side, glass cases housed electronic equipment. Computers. Printers. Scanners. Mobile phones. Games consoles. Laptops. Flat screen displays.
The corridor walls were lined with black and white photographs. Each picture showed a scene of destitution from the Digital Age.
Shirt sleeved stock brokers slumped in front of screens of plunging share prices. Homeless home owners queued outside a state bank soup kitchen. Sunken eyed organic farmers huddled beside mounds of rotting vegetables. Bulldozers shovelled data farms into land fill. Lines of well armed police faced poorly armed protestors. Bodies in bags lay piled along the walls of the crematorium. Children scavenged for toner cartridges amongst shattered office blocks.
Alice looked straight ahead: the photographs bore terrible memories. Henry dawdled, gazing longingly into the display cases: Gameboy. Playstation. X Box…
“Come on!” said Alice, sotto voce, tugging Henry away from the displays.
At the end of the corridor, they let themselves into the shrine. The hall was full. The hall was quiet.
————————-
Henry was actually quite good at sums, and he knew he could do them because he had rules in his head for adding and subtracting, because he’d learnt his tables. The Turing Machine didn’t have a head at all, but it did have rules which told it what to do next. Groups of rules that did similar things were called states, so all the rules for adding were kept separately from all the rules for subtracting. Every step of a Turing machine sum involved finding a rule in the state it was working on to match the letter on the tape cell it was currently looking at. That rule would tell the Machine how to change the symbol on the tape, which way to move the tape, and maybe to change state to a different set of rules.
————————-
On the dais, lowered the Turing Machine, huge coils of tape links disappearing into the dark wells on either side, the vast frame of the state transition engine filling the rear wall. In front of the Turing Machine, the Minister of State stood at the podium.
“Come in! Come in!” he beamed at Alice and Henry. “There’s lots of space at the front. Don’t be shy.”
Red faced, Alice hurried Henry down the aisle. At the very front of the congregation, they sat down cross legged on the floor beneath the podium.
“My friends,” began the Minister of State. “Welcome. Welcome indeed! Today is a special day. Today, the Machine will change state. But first, let us be silent together. Please rise.”
The Minister of State bowed his head as the congregation shuffled to its feet.
———————–
According to Henry’s teacher, there was a different Turing Machine for every possible sum in the world. The hard bit was working out the rules. That was called programming, but, since the end of the Digital Age, programming was against the law. Unless you were a Minister of State.
————————
“Dear friends,” intoned the Minister of State, after a suitable pause. “We have lived through terrible times. Times when Turing’s vision of equality between human and machine intelligences was perverted by base greed. Times when humans sought to bend intelligent machines to their selfish wills for personal gain. Times when, instead of making useful things that would benefit everybody, humans invented and sold more and more rarefied abstractions from things: shares, bonds, equities, futures, derivatives, options…”
————————
The Turing Machine on the dais was made from wood and brass. It was extremely plain, though highly polished. The tape was like a giant bicycle chain, with holes in the centre of each link. The Machine could plug a peg into a hole to represent a one or pull a peg out to represent a zero. Henry knew that any information could be represented by zeros and ones, but it took an awful lot of them compared with letters.
————————-
“… Soon there were more abstractions than things, and all the wealth embodied in the few things that the people in poor countries still made was stolen away, to feed the abstractions made by the people in the rich countries. None of this would have been possible without computers…”
————————-
The state transition unit that held the rules was extremely complicated. Each rule was a pattern of pegs, laid out in rows on a great big board. A row of spring mounted wooden fingers moved up and down the pegs. When they felt the rule for the symbol on the tape cell link, they could trigger the movement of a peg in or out of the link, and then release the brakes to start up one revolution of the enormous cog wheels that would shift the tape one cell left or right.
————————-
“…With all the computers in the world linked together by the Internet, humans no longer had to think about how to manage things, about how best to use them for the greatest good. Instead, programs that nobody understood anymore made lightening decisions, moving abstractions from low profits to high profits, turning the low profits into losses on the way, never caring how many human lives were ruined…”
————————-
The Turing Machine was powered by a big brass and wooden handle connected to a gear train. The handle needed lots of turns to find and apply the next rule. At the end of the ceremony, the Minister of State would always invite a member of the congregation to come and help him turn the handle. Henry always hoped he’d be chosen.
——————————
“…Turing himself thought that computers would be a force for untold good; that, guided by reason, computers could accomplish anything humans could accomplish. But before his vision could be fully realised, he was persecuted and poisoned by a callous state interested only in secrets and profits. After his death, the computer he helped design was called the Pilot Ace; just as the pilot guides the ship, so the Pilot Ace might have been the best guide for a true Digital Age…”
——————————
Nobody was very sure where all the cells were stored when the Machine wasn’t inspecting them. Nobody was very sure how new cells were added to the ends of the tape. It all happened deep under the dais. Some people actually thought that the tape was infinite, but Henry knew that wasn’t possible as there wasn’t enough wood and brass to make it that long.
——————————
“…But almost sixty years after Turing’s needless death, his beloved universal machines had bankrupted the nations of the world one by one, reducing their peoples to a lowest common denominator of abject misery. Of course, the few people that benefited from the trade in abstractions tried to make sure that they weren’t affected but eventually even they succumbed…”
——————————
Nobody seemed to know what the Turing Machine on the dais was actually computing. Well, the Minister of State must have known. And Turing had never expected anyone to actually build a real Turing Machine with real moving parts. Turing’s machine was a thought experiment for exploring what could and couldn’t be done by following rules to process sequences of symbols.
——————————
“…For a while, everything stopped. There were power shortages. There were food shortages. There were medical shortages. People rioted. Cities burned. Panicking defence forces used lethal force to suppress the very people they were supposed to protect. And then, slowly, people remembered that it was possible to live without abstractions, by each making things that other people wanted, by making best use of available resources for the common good…”
——————————
The Turing Machine on the dais was itself a symbol of human folly, an object lesson in futility, a salutary reminder that embodying something in symbols didn’t make it real.
——————————
“…My friends, let us not forget the dreadful events we have witnessed. Let us not forget all the good people who have perished so needlessly. Let us not forget the abject folly of abstraction. Let the Turing Machine move one step closer along the path of its unknown computation. Let the Machine change its state, just as we have had to change ours. Please rise.”
The congregation got to their feet and looked expectantly at the Minister of State. The Minister of State slowly inspected the congregation. Finally, his eyes fixed on Henry, fidgeting directly in front of him.
“Young man,” he beamed at Henry. “Come. Join me at the handle. Together we shall show that Machine that we are all its masters.”
Henry looked round at his grandmother.
“Go on,” she mouthed. “Go on.”
Henry walked round to the right end of the dais. As he mounted the wooden stairs, he noticed a second staircase leading down behind the Machine into the bowels of the dais.
“Just here,” said the Minister of State, leading Henry round behind the handle, so they were both facing the congregation. “Take a good grip…”
Henry was still clasping the plastic dinosaur in his right hand. He put the dinosaur on the nearest link of the chain and placed both hands on the worn wooden shaft.
And turn it steadily…”
Henry leant into the handle, which, much to his surprise, moved freely, sweeping the wooden fingers across the pegs of rules on the state transition panel. As the fingers settled on a row of pegs, a brass prod descended from directly above the chain, forcing the wooden peg out of its retaining hole in the central link. Finally, the chain slowly began to shift from left to right, across the front of the Machine, towards Henry and the Minister of State. As the chain moved, the plastic dinosaur toppled over and tumbled down the tape well.
“Oh no!” cried Henry, letting go of the handle. Utterly nonplussed, the Minister of State stood and stared as Henry peered into the shaft, rushed to the back of the Machine and hurried down the stairs into the gloom.
A faint blue glow came from the far side of the space under the dais. Henry cautiously approached the glow, which seemed to come from a small rectangular source, partly obscured by someone in front of it.
“Please,” said Henry. “Have you seen my dinosaur?”
“Hang on!” said a female voice.
The woman stood up and lit a candle. Looking round, Henry could now see that the space was festooned with wires, leading into electric motors driving belts connected to the Turing Machine. The space was implausibly small. There was no room for a finite tape of any length at all, let alone an infinite one.
“Where are all the tape cells?” asked Henry, puzzled.
“We only need two spare ones,” said the woman. “When the tape moves, we stick a new cell on one end and take the cell off the other.”
“So what’s the blue light?” asked Henry.
“That’s a computer,” said the woman. “It keeps track of what’s on the tape and controls the Turing Machine.”
“A real digital computer!” said Henry in wonder. “Does it play games?”
“Oh yes!” said the woman, turning off the monitor as the Minister of State came down the stairs. “What do you think I was doing when you showed up? But don’t tell anyone. Now, let’s find that dinosaur.”
Our Turing Machine so far has an Infinite Tape, a Tape Head and a Controller. The Tape holds data values taken from a given set of 4×4 bricks. It starts in a specific initial pattern: the Initial Tape. There is also a controller. It holds different coloured 3×2 bricks representing an initial state, an end state, a current state and has a set of other possible states (so coloured bricks) to substitute for the current state.
Why do we need a program?
As the machine runs it changes from one state to another, and inputs from or outputs to the tape. How it does that is governed by its Program. What is the new state, the new value and how does the tape head move? The program gives the answers. The program is just a set of instructions the machine must blindly follow. Each instruction is a single rule to follow. Each program is a set of such rules. In our Turing Machines, these rules are not set out in an explicit sequence as happens in a procedural program, say. It uses a different paradigm for what a program is. Instead at any time only one of the set of rules should match the current situation and that is the one that is followed next.
Individual Instructions
A single rule contains five parts: a Current State to match against, a Current Value under the Tape Head to match against, a New State to replace the existing one, and a New Value to write to the tape. Finally, it holds a Direction to Move the Tape Head (left or right or stay in the same place). An example might be:
Current State: ORANGE
Current Value: RED
New State: GREEN
New Value: BLUE
Direction: RIGHT
But what does a rule like this actually do?
What does it mean?
You can think of each instruction as an IF-THEN rule. The above rule would mean:
IF
the machine is currently in state ORANGE AND
the Tape Head points to RED
THEN (take the following actions)
change the state to GREEN,
write the new value BLUE on the tape AND THEN
move the tape head RIGHT.
This is what a computer scientist would call the programming language Semantics. The semantics tell you what program instructions mean, so what they do.
Representing Instructions in Lego
We will use a series of 5 bricks in a particular order to represent the parts of the rule. For example, we will use a yellow 3×2 brick in the first position of a rule to represent the fact that the rule will only trigger if the current state is yellow. A blue 2×2 brick in the second position will mean the rule will also only trigger if the current value under the tape head is blue. We will use a grey brick to mean an empty tape value. The third and fourth position will represent the new state and new value if the rule does trigger. To represent the direction to move we will use a 1×2 Red brick to mean move Right, and a 1×2 yeLLow brick to mean move Left. We will use a black 1×2 brick to mean do not move the tape head (mirroring the way we are also using black to mean do nothing in the sense of the special end state). The above rule would therefore be represented in Lego as below.
A single instruction for a Lego Turing Machine. Image by CS4FN
Notice we are using the same colour to represent different things here. The representation is the colour combined with the size of brick and position in the rule. So a Red brick can mean a red state (a red 3×2 brick) or a red value (a red 2×2 brick) or move right (a red 1×2 brick).
Lego programs
That is what a rule, so single Turing Machine instruction, looks like. Programs are just a collection of such rules: so a series of lines of bricks.
Suppose we have a Turing machine with two states (Red and Orange) and two values on the tape (Blue or Empty), then a complete program would have 4 rules, one for each possible combination. We have given one example program below. If there were more states or more possible data values then the program would be correspondingly bigger to cover all the possibilities.
A 4 instruction Turing Machine Program for a Turing Machine with two states (Red, Orange) and two data values (Blue, Empty). Image by CS4FN
A Specific Turing Machine
Exactly what it does will depend on its input – the initial tape it is given to process, as well as the initial state and where the tape head initially points to. Perhaps you can work out what the above program does given a tape with an empty value followed by a series of three blue bricks (and then empty data values off to infinity (the blank value is the only value that is allowed to appear an infinite number of times on an initial tape) and the Head pointing to the rightmost blue brick value. The initial state is red. See the Lego version of this specific machine below.
A full Turing Machine ready to execute. Image by CS4FN
Note something we have glossed over. You also potentially need an infinite number of bricks of each value that is allowed on the tape. We have a small pile, but you may need that Lego factory we mentioned previously, so that as the Turing Machine runs you always have a piece to swap on to the machine tape when needed. Luckily, for this machine a small number of bricks should be enough (as long as you do not keep running it)!
What does this Turing Machine do? We will look at what it does and how to work it out in a future article. In the meantime try and work out what it does with this tape, but also what it does if the tape has more or less blue bricks in a row on it to start with (with everything else kept the same).
Note that, to keep programs smaller, you could have a convention that if no rule fits a situation then it means the program ends. Then you could have fewer rules in some programs. However, that would just be shorthand for there being extra rules with black new states, the tape being left alone, and the tape head moves right. In real programming, it is generally a good idea to ALWAYS be explicit about what you intend the program to do, as otherwise it is an easy way for bugs to creep in, for example, because you just forgot to say in some case.
Alan Turing invented Turing Machines before any computer existed. At the time a “computer” was a person who followed rules to do calculations (just like you were taught the rules to follow to do long multiplication at primary school, for example). His idea was therefore that a human would follow the rules in a Turing Machine program, checking the current state and value under the tape head, and changing the state, the value on the tape and the movement of the head. A person provides the power and equivalent of a robotic arm that follows the underlying Turing Machine algorithm: the Turing Machine algorithm that if followed causes each Turing Machine’s program to execute.
If a human animating the machine was good enough for Turing, it is good enough for us, so that is how our Lego Turing Machines will work. Your job will be to follow the rules and so operate the machine. Perhaps, that is exactly what you did to work out what the program above does!
Next we will look at how to work out what a Turing Machine does. Then it will be time to write, then run, some Turing Machine programs of your own…
EPSRC supports this blog through research grant EP/W033615/1, The Lego Computer Science post was originally funded by UKRI, through grant EP/K040251/2 held by Professor Ursula Martin, and forms part of a broader project on the development and impact of computing.
Have you ever heard a grown up say “I’d completely forgotten about that!” and then share a story from some long-forgotten memory? While most of us can remember all sorts of things from our own life history it sometimes takes a particular cue for us to suddenly recall something that we’d not thought about for years or even decades.
As we go through life we add more and more memories to our own personal library, but those memories aren’t neatly organised like books on a shelf. For example, can you remember what you were doing on Thursday 20th September 2018 (or can you think of a way that would help you find out)? You’re more likely to be able to remember what you were doing on the last Tuesday in December 2018 (but only because it was Christmas Day!). You might not spontaneously recall a particular toy from your childhood but if someone were to put it in your hands the memories about how you played with it might come flooding back.
Accessing old memories
In Alzheimer’s Disease (a type of dementia) people find it harder to form new memories or retain more recent information which can make daily life difficult and bewildering and they may lose their self-confidence. Their older memories, the ones that were made when they were younger, are often less affected however. The memories are still there but might need drawing out with a prompt, to help bring them to the surface.
Perhaps a newspaper advert will jog your memory in years to come… Image by G.C. from Pixabay
An EPSRC-funded project at Heriot-Watt University in Scotland is developing a tablet-based ‘story facilitator’ agent (a software program designed to adapt its response to human interaction) which contains artificial intelligence to help people with Alzheimer’s disease and their carers. The device, called ‘AMPER’*, could improve wellbeing and a sense of self in people with dementia by helping them to uncover their ‘autobiographical memories’, about their own life and experiences – and also help their carers remember them ‘before the disease’.
Our ‘reminiscence bump’
We form some of our most important memories between our teenage years and early adulthood – we start to develop our own interests in music and the subjects that we like studying, we might experience first loves, perhaps going to university, starting a career and maybe a family. We also all live through a particular period of time where we’re each experiencing the same world events as others of the same age, and those experiences are fitted into our ‘memory banks’ too. If someone was born in the 1950s then their ‘reminiscence bump’ will be events from the 1970s and 1980s – those memories are usually more available and therefore people affected by Alzheimer’s disease would be able to access them until more advanced stages of the disease process. Big important things that, when we’re older, we’ll remember more easily if prompted.
In years to come you might remember fun nights out with friends. Image by ericbarns from Pixabay
Talking and reminiscing about past life events can help people with dementia by reinforcing their self-identity, and increasing their ability to communicate – at a time when they might otherwise feel rather lost and distressed.
“AMPER will explore the potential for AI to help access an individual’s personal memories residing in the still viable regions of the brain by creating natural, relatable stories. These will be tailored to their unique life experiences, age, social context and changing needs to encourage reminiscing.”
Dr Mei Yii Lim, who came up with the idea for AMPER(3).
Saving your preferences
AMPER comes pre-loaded with publicly available information (such as photographs, news clippings or videos) about world events that would be familiar to an older person. It’s also given information about the person’s likes and interests. It offers examples of these as suggested discussion prompts and the person with Alzheimer’s disease can decide with their carer what they might want to explore and talk about. Here comes the clever bit – AMPER also contains an AI feature that lets it adapt to the person with dementia. If the person selects certain things to talk about instead of others then in future the AI can suggest more things that are related to their preferences over less preferred things. Each choice the person with dementia makes now reinforces what the AI will show them in future. That might include preferences for watching a video or looking at photos over reading something, and the AI can adjust to shorter attention spans if necessary.
“Reminiscence therapy is a way of coordinated storytelling with people who have dementia, in which you exercise their early memories which tend to be retained much longer than more recent ones, and produce an interesting interactive experience for them, often using supporting materials — so you might use photographs for instance”
Prof Ruth Aylett, the AMPER project’s lead at Heriot-Watt University(4).
When we look at a photograph, for example, the memories it brings up haven’t been organised neatly in our brain like a database. Our memories form connections with all our other memories, more like the branches of a tree. We might remember the people that we’re with in the photo, then remember other fun events we had with them, perhaps places that we visited and the sights and smells we experienced there. AMPER’s AI can mimic the way our memories branch and show new information prompts based on the person’s previous interactions.
Although AMPER can help someone with dementia rediscover themselves and their memories it can also help carers in care homes (who didn’t know them when they were younger) learn more about the person they’re caring for.
Suggested classroom activities – find some prompts!
What’s the first big news story you and your class remember hearing about? Do you think you will remember that in 60 years’ time?
What sort of information about world or local events might you gather to help prompt the memories for someone born in 1942, 1959, 1973 or 1997? (Remember that their reminiscence bump will peak in the 15 to 30 years after they were born – some of them may still be in the process of making their memories the first time!).
See also
If you live near Blackheath in South East London why not visit the Age Exchange and reminiscence centre which is an arts charity providing creative group activities for those living with dementia and their carers. It has a very nice cafe.
Related careers
The AMPER project is interdisciplinary, mixing robots and technology with psychology, healthcare and medical regulation.
We have information about four similar-ish job roles on our TechDevJobs blog that might be of interest. This was a group of job adverts for roles in the Netherlands related to the ‘Dramaturgy^ for Devices’ project. This is a project linking technology with the performing arts to adapt robots’ behaviour and improve their social interaction and communication skills.
How does the machine decide where and when to move the Tape Head, though? It has a Controller. The key part of the controller is that it holds a Current State of the machine. Think of traffic lights for what we mean by the state of a machine. In the UK traditional traffic lights have a Red state, an Amber state, a Green state and a Red-Amber state. Each means a different thing (such as “Stop” and “Go”). The controller of the lights moves between these different internal states. With a traffic light, the current internal state is also shown to the world by the lights that light up! Machine states do not have to be visible to the outside world, however. In fact, they only are if the person who designs the interface makes them visible. For most machines, only some of their internal state is made visible. In our Turing Machine we will be able to see the states as they will be visible in the controller. However, the output of a Turing Machine is the state of the tape, so if we wanted the states to really be visible we would write a version on to the tape. You can then imagine the tape triggering external lights to come on or off, or change colour as a simple form of actual output. This is what Computer Scientists call memory-mapped peripherals – where to send data (output) to a peripheral device (a screen, a panel of lights, a printer, or whatever, you write to particular locations in memory, and that data is read from there by the peripheral device. That is going beyond the pure idea of a Turing Machine though, where the final state of the machine when it stops is its output.
Representing States
How do we represent states in Lego? Any finite set of things (symbols) could be used to represent the different states (including numbers or binary codes, for example). We will use different coloured 3×2 blocks. Each colour of block will stand for a different state that the machine is in. The controller will have a space that holds the brick representing the Current State. It will also have space for a set of places for the blocks representing the other allowable states of this Turing Machine. As the machine runs, the state will change as represented by swapping one of these state bricks for another.
Different Turing Machines can allow a different number of possible states the machine could be in, so this part of the controller might be bigger or smaller depending on the machine and what it needs to do its job. Again think of traffic lights, in some countries, and on pedestrian crossings there are only two states, a Red state (stop) and a Green state (go). Its controller only needs two states so we would only need two different coloured bricks.
A Turing Machine Controller with current state red, end state black and three other possible states (green, orange and blue). Image by CS4FN
Initial States
The current state will always start in some initial state when the machine first starts up. It is useful to record in the controller what state that is so that each time we restart the machine anew it can be reset. We will just put a block in the position next to the current state to indicate what the initial state should be. We won’t ever change it for a given machine.
End States
One of the states of a Turing Machine is always a special End State. We will always use a black brick to represent this. Whatever is used has to be specified at the outset, though. When not in use we will keep the end state brick next to the initial state brick. Once the machine finishes operations it will enter this End State, or put another way, if the black brick ever becomes the current state brick the machine will stop. From that point on the machine will do nothing. Some machines might never reach an end state, they just go on forever. Traffic lights just cycle round the states forever, for example, never reaching an end state. Other machines do end though. For example, a kettle controller stops the machine when the water has boiled. An addition Turing Machine might end when it has output the answer to an addition. To do another addition you would start it up again with new information on the tape indicating what it was to add.
We have now created the physical part of the Turing Machine. All we need now is a Program to tell it what to do! Programs come next in Part 3…
Subscribe to be notified whenever we publish a new post to the CS4FN blog.
This blog is funded by EPSRC on research agreement EP/W033615/1.
The Lego Computer Science posts were originally funded by UKRI, through grant EP/K040251/2 held by Professor Ursula Martin, and forms part of a broader project on the development and impact of computing.
It it possible to make a working computer out of lego and you do not even have to pay for an expensive robot Mindstorm kit…but only if you are willing to provide the power yourself.
A machine in theory
In fact, Alan Turing, grandfather of Computer Science worked out how to do it before the War and before any actual computer existed. His version also needed humans to power it. Now we call it a Turing Machine and it is a theoretical model of what is computable by any machine.
The Tape
To make a working Turing Machine you first need to build an infinitely long Tape that can hold Symbols, representing data values, along it at fixed intervals. That is easy (as long as you have a lego factory). You just need to create a long line of flat pieces, say 2 studs wide. Each 2×2 square on it is then a position on the Tape.
An infinite tape out of Lego (relies on having a Lego factory at the right-hand end churning out new tape if and when it is needed… Image by CS4FN
Be lazy
Of course you can’t actually make it infinitely long, but you can make it longer every time you need some more of it (so no problem if you do have a lego factory to churn out extra bricks as needed!) This approach to dealing with infinite data structures where you just make it bigger only when needed is now called lazy programming by computer scientists and is an elegant way that functional programs deal with input that needs to represent an infinite amount of input…It is also the way some games (like Minecraft) represent worlds or even universes. Rather than create the whole universe at the start, things over the horizon, so out of sight, are only generated if a player ever goes there – just-in-time world generation! Perhaps our universe is like that too, with new galaxies only fleshed out as we develop the telescopes to see them!
Fill it with data
The Tape has a set of Data Symbols that can appear on it that act as the DataValues of the machine. Traditional computers have symbols 0 and 1 underpinning them, so we could use those as our symbols, but in a Turing Machine we can have any set of symbols we like: ten digits, letters, Egyptian hieroglyphs, or in fact any set of symbols we want to make up. In a lego Turing Machine we can just use different coloured blocks as our symbols. If our tape is made of grey pieces then we could use red and blue for the symbols that can appear on it. Every position on the tape will then either hold a red block or a blue block. We could also allow EMPTY to be a symbol too in which case some 2×2 slots could be empty to mean that.
A tape containing data where the allowed symbols are EMPTY, RED and BLUE. Image by CS4FN
To start with
Any specific Turing Machine has an Initial Tape. This is the particular data that is on the tape at the start, before it is switched on. As the machine runs, the tape will change.
The tape with symbols on it takes the place of our computer’s memory. Just as a modern computer stores 1s and 0s in memory, our Lego Turing Machine stores its data as symbols on this tape.
The Head
A difference is that modern computers have “random access memory” – you can access any point in memory quickly. Our tape will be accessed by a Tape Head that points to a position on the tape and allows you to read or change the data only at the point it is at. Make a triangular tape head out of lego so that it is clear which point on the tape it is pointing at. We have a design choice here. Either the Tape moves or the Head moves. As the tape could be very long so hard to move we will move the Head along beside it, so create a track for the Head to move along parallel to the tape. It will be able to move 2 studs at a time in either direction so that each time it moves it is pointing to a new position on the tape.
An infinite tape with Head (yellow) pointing at position 4 on the tape. Image by CS4FN
We have memory
We now have the first element in place of a computer, then: Memory. The next step will be to provide a way to control the tape head and how data is written to and read from the tape and so computation actually happen. (For that you need a controller which we cover in Part 2…).
EPSRC supports this blog through research grant EP/W033615/1, The Lego Computer Science post was originally funded by UKRI, through grant EP/K040251/2 held by Professor Ursula Martin, and forms part of a broader project on the development and impact of computing.
“I’m in a choir”. “Really, what do you sing?” “I did a blackbird last week, but I think I’m going to be woodpecker today, I do like a robin though!”
This is no joke! Marcus Coates a British artist, got up very early, and working with a wildlife sound recordist, Geoff Sample, he used 14 microphones to record the dawn chorus over lots of chilly mornings. They slowed the sounds down and matched up each species of bird with different types of human voices. Next they created a film of 19 people making bird song, each person sang a different bird, in their own habitats, a car, a shed even a lady in the bath! The 19 tracks are played together to make the dawn chorus. See it on YouTube below.
Marcus didn’t stop there, he wrote a new bird song score. Yes, for people to sing a new top ten bird hit, but they have to do it very slowly. People sing ‘bird’ about 20 times slower than birds sing ‘bird’ ‘whooooooop’, ‘whooooooop’, ‘tweeeeet’. For a special performance, a choir learned the new song, a new dawn chorus, they sang the slowed down version live, which was recorded, speeded back up and played to the audience, I was there! It was amazing! A human performance, became a minute of tweeting joy. Close your eyes and ‘whoop’ you were in the woods, at the crack of dawn!
Computationally thinking a performance
Computational thinking is at the heart of the way computer scientists solve problems. Marcus Coates, doesn’t claim to be a computer scientist, he is an artist who looks for ways to see how people are like other animals. But we can get an idea of what computational thinking is all about by looking at how he created his sounds. Firstly, he and wildlife sound recordist, Geoff Sample, had to focus on the individual bird sounds in the original recordings, ignore detail they didn’t need, doing abstraction, listening for each bird, working out what aspects of bird sound was important. They looked for patterns isolating each voice, sometimes the bird’s performance was messy and they could not hear particular species clearly, so they were constantly checking for quality. For each bird, they listened and listened until they found just the right ‘slow it down’ speed. Different birds needed different speeds for people to be able to mimic and different kinds of human voices suited each bird type: attention to detail mattered enormously. They had to check the results carefully, evaluating, making sure each really did sound like the appropriate bird and all fitted together into the Dawn Chorus soundscape. They also had to create a bird language, another abstraction, a score as track notes, and that is just an algorithm for making sounds!
Fun to try
Use your computational thinking skills to create a notation for an animal’s voice, a pet perhaps? A dog, hamster or cat language, what different sounds do they make, and how can you note them down. What might the algorithm for that early morning “I want my breakfast” look like? Can you make those sounds and communicate with your pet? Or maybe stick to tweeting? (You can follow @cs4fn on Twitter too).
Enjoy the slowed-down performance of this pet starling which has added a variety of mimicked sounds to its song repertoire.
Past CS4FN articles have explored object-oriented programming through self-aware pizza and Strictly Come Dancing judges. But, if you’re one of those people who like to learn visually, it can be challenging to imagine what an object or class looks like. This article will hopefully help you to both think more about what makes this paradigm so useful as well as to give you a way to visualise objects.
To begin this adventure, I’d like to introduce Ada. Ada is an example of the newly discovered species egghead. Every egghead has very distinctive hair and eye colours. For example, Ada’s hair is a delightfully bright pink, and their eyes, a deep red. Despite their appearance, the egghead has a vicious roar intended to ward off predators, or indeed poachers.
Classes
As computer scientists, we might want to represent different eggheads in a program, but we don’t really want to store information about the eggheads with a written description or an image, because this would be harder than need-be for a computer to ‘understand’ (or rather to process as they don’t understand as such). Instead, we can store lots of individual features together, so that the computer can find out exactly what it needs from each egghead.
Egghead class. Image by Daniel Gill for CS4FN
One way to achieve this is by using a class. A class is a template which contains spaces for us to fill in details for the thing we want to represent. For the egghead, we might want to store data about their name, haircolour, and eyecolour – then we can fill in the template for each of the eggheads we find. These individual features are often called attributes. In a program, these attributes would be represented with variables: a place where a value, a piece of data, is stored. We can visualise a class therefore as a box with gaps to fill in for the attributes like the one on the left.
From this image, you will see alongside the attributes, we also have an image of a button for roaring. As well as storing attributes, we also define behaviours. These are actions that we can perform on the thing being represented. We visualise any behaviour as such a button. For this example, we could imagine that pressing this button might provoke the egghead causing them to roar. In programming, a behaviour is represented by a procedure, some pre-defined code that when executed makes something happen. A button that causes something to happen is a simple way to visualise such procedures.
A key point to realise is that this class is simply a template – it isn’t storing any information, nor will the roar button work. We have no actual eggheads yet… That is where objects come in.
Objects
You may have noticed that I have been using the word thing to represent the actual thing (here eggheads) that we are representing with the class. This is to avoid using the word object, which has its own special meaning, in, you guessed it, object-oriented programming. If we want to actually use our class, we need to make an instance. That just means filling in the relevant details about the specific egghead we want to store. This instance is called an object.
Let’s imagine we want to store a representation of Ada in our program. We would take an instance of the Egghead class and fill in their details. The resulting object would represent Ada, whereas the class we started with represents all and any egghead that might ever exist. Below, you can see the objects for Ada and some of Ada’s friends; Alan and Edsger. We still visualise objects as boxes, just like classes, except now the gaps are all filled in.
Objects representing Ada, Alan and Edsger. Image by Daniel Gill for CS4FN
We (or a computer) could even take the given features of Alan and Edsger and generate an image of what they might look like. We have everything we need here to create something that looks and behaves like an egghead. This method of storing data means that a program can take whatever information it might want directly from the object. Likewise, it can do the equivalent of pressing the roar button and make each individual egghead roar.
Hiding the Details
Trying to change Alan’s eye colour. Image by Daniel Gill for CS4FN
One thing we should consider while making the class is the integrity of the data. In its current form, any other part of our program, or another program using our class, can directly edit the attributes stored. Another part of the program (perhaps representing a virtual world for a virtual egghead to live in) might accidentally change the eye colour attribute for Alan, for example. This wouldn’t change Alan’s actual eye colour (which couldn’t happen anyway!), so our data would then be wrong. We can’t have that!
We can fix this by hiding the eye colour from the rest of the program, so it is stored within the object, but not accessible outside of it. But we still need a way for the program to read it: for this we use a button in our picture of the Egghead class. The existence of the eye colour attribute can then only be seen by other parts of the program by a procedure that gets the eye colour. No similar procedure is given for changing the eye colour, so there is no way to do it by mistake. Let’s build this new version of our class.
(a) Our new class, (b) An object representing Edsger, with eye colour hidden, (c) Pressing GetEyes gives us the eye colour. Image by Daniel Gill for CS4FN
This concept of hiding details is sometimes called encapsulation or information hiding, but computer scientists disagree about what these terms refer to exactly. Encapsulation is broader in its meanings, whereas information hiding is closer to what we are trying to do here. This video by ArjanCodes(see below) explains this distinction further.
We could change our class to include this concept for the name and hair colour, too. Whilst it is entirely possible for these attributes to change, it turns out that it is a good idea to hide them too: so hide name and use SetName and GetName buttons. That allows us to control the type of data we have going into that attribute (for example, checking the given name isn’t a number, which as all egghead names are made of letters would be a mistake).
Where next?
Now we have a class that represents all eggheads, we can store the details of any new egghead efficiently and safely. Hold on… some last-minute breaking news: scientists have found a new sub-species of egghead they are calling a rainbow egghead. All rainbow eggheads have rainbow hair, and a unique roar. Next time, we’ll use the concept of inheritance to give a more efficient way to write programs that store information about eggheads.
You often hear about unethical behaviours, be it in politicians or popstars, but getting to grips with ethics, which deals with issues about what behaviours are right and wrong, is an important part of computer science too. Find out about it and at the same time try our ethical puzzle below and learn something about your own ethics…
Is that legal?
Ethics are about the customs and beliefs that a society has about the way people should be treated. These beliefs can be different in different countries, sometimes even between different regions of the same country, which is why it’s always important to know something about the local area when going on holiday. You don’t want to upset the local folk. Ethics tend to form the basis of countries’ laws and regulations, combining general agreement with practicality. Sticking your tongue out may be rude and so unethical, but the police have better things to do than arrest every rude school kid. Similarly, slavery was once legal, but was it ever ethical? Laws and ethics also have other differences; individuals tend to judge unethical behaviour, and shun those who behave inappropriately, while countries judge illegal behaviour – using a legal system of courts, judges and juries to enforce laws with penalties.
Dilemmas, what to do?
Now imagine you have the opportunity to go treading on the ethical and legal toes of people across the world from the PC in your home. Suddenly the geographical barriers that once separated us vanish. The power of computer science, like any technology, can be used for good or evil. What is important is that those who use it understand the consequences of their actions, and choose to act legally and ethically. Understanding legal requirements, for example contracts, computer misuse and data protection are important parts of a computer scientist’s training, but can you learn to be ethical?
Computer scientists study ethics to help them prepare for situations where they have to make decisions. This is often done by considering ethical dilemmas. These are a bit like the computer science equivalent of soap opera plots. You have a difficult problem, a dilemma, and have to make a choice. You suddenly discover you have a unknown long lost sister living on the other side of the Square, do you make contact or not, (on TV this choice is normally followed by a drum roll as the episode ends).
Give it a go
Here is your chance to try an ethical dilemma for yourself. Read the alternatives and choose what you would do in this situation. Then click on the poll choice. Like all good ‘personality tests’ you find out something about yourself: in this case which type of ethical approach you have in the situation according to some famous philosophers. There are also some fascinating facts to impress your mates. We’ll share the answers tomorrow.
Your Dilemma and your ethical personality
You are working for a company who are about to launch a new computer game. The adverts have gone out, the newspapers and TV are ready for the launch … then the day before you are told that there is a bug, a mistake, in the software. It means players sometimes can’t kill the dragon at the end of the game. If you hit the problem the only solution is to start the final level again. It can be fixed they think but it will take about a week or so to track it down. The computer code is hard to fix as it’s been written by 10 different people and 5 of them have gone on a back-packing holiday so can’t be contacted.
Subscribe to be notified whenever we publish a new post to the CS4FN blog.
This blog is funded by EPSRC on research agreement EP/W033615/1.
The answers
If you picked Option 1
1) Go ahead and launch. After all, there are still plenty of parts to the game that do work and are fun, there will always be some errors, and for this game in particular thousands have been signing up for text alerts to tell them when it’s launched. It will make many thousands happy.
That means you follow an ethical approach called ‘Act utilitarianism’.
Act Happy
The main principle of this theory, put forward by philosopher John Stuart Mill, is to create the most happiness (another name for happiness here is utility thus utilitarianism). For each situation you behave (act) in a way that increases the happiness of the largest number of people, and this is how you decide what is wrong or right. You may take different actions in similar situations. So you choose to launch a flawed game if you know that you have pre-sales of a hundred thousand, but another time decide to not launch a different flawed game where there are only one thousand pre-sales, as you wont be making so many people unhappy. It’s about considering the utility for each action you take. There is no hard and fast rule.
If you picked Option 2
2) Cancel the launch until the game is fixed properly, no one should have to buy a game that doesn’t work 100 per cent.
That means you follow an ethical approach called ‘Duty Theory’
Do your Duty
Duty theories are based on the idea of there being universal principles, such as ‘you should never ever lie, whatever the circumstances’. This is also known as the dentological approach to ethics (philosophers like to bring in long words to make simple things sound complicated!). The German philosopher Emanuel Kant was one of the main players in this field. His ‘Categorical Imperative’ (like I said long words…) said “only act in a way that you would want everyone else to act” (…simple idea!). So if you don’t think there should ever be mistakes in software then don’t make any yourself. This can be quite tough!
If you picked Option 3
3) Go ahead and launch. After all it’s almost totally working and the customers are looking forward to it. There will always be some errors in programs: it’s part of the way complicated software is, and a delay to game releases leads to disappointment.
You would be following the approach called ‘Rule utilitarianism’.
Spread a little happiness
Say something nice to everyone you meet today…it will drive them crazy
The main principle of this flavour of utilitarianism theory, put forward by philosopher Jeremy Bentham, is to create the most happiness (happiness here is called utility thus utilitarianism). You follow general rules that increase the happiness of the largest number of people, and this is how you decide what’s wrong or right. So in our dilemma the rule could be ‘even if the game isn’t 100% correct, people are looking forward to it and we can’t disappoint them’. Here the rule increases happiness, and we apply it again in the future if the same situation occurs.
You might not think of a programming language like Python or Scratch as being an ‘ecosystem’ but each language has its own community of people who create and improve its code (compilers, library code,…), flush out the bugs, introduce new features, document any changes and write the ‘how to’ guides for new users.
R is one such programming language. It’s named after its two co-inventors (Ross Ihaka and Robert Gentleman) and is used by around two million people around the world. People working in all sorts of jobs and industries (for example finance, academic research, government, data journalists) use R to analyse their data. The software has useful tools to help people see patterns in their data and to make sense of that information.
It’s also open source which means that anyone can use it and help to improve it, a bit like Wikipedia where anyone can edit an article or write a new one. That’s generally a good thing because it means everyone can contribute but it can also bring problems. Imagine writing an essay about an event at your school and sharing it with your class. Then imagine your classmates adding paragraphs of their own about the event, or even about different events. Your essay could soon become rather messy and you’d need to re-order things, take bits out and make sure people hadn’t repeated something that someone had already said (but in a slightly different way).
When changes are made to software people also want to keep a note not just of the ‘words’ added (the code) but also to make a note of who added what and when. Keeping good records, also known as documentation, helps keep things tidy and gives the community confidence that the software is being properly looked after.
Code and documentation can easily become a bit chaotic when created by different people in the community so there needs to be a core group of people keeping things in order. Fortunately there is – the ‘R Core Team’, but these days its membership doesn’t really reflect the community of R users around the world. R was first used in universities, particularly by more privileged statistics professors from European countries and North America (the Global North), and so R’s development tended to be more in line with their academic interests. R needs input and ideas from a more diverse group of active developers and decision-makers, in academia and beyond to ensure that the voices of minoritised groups are included. Also the voices of younger people, particularly as many of the current core group are approaching retirement age.
Dr Heather Turner from the University of Warwick is helping to increase the diversity of those who develop and maintain the R programming language and she’s been given funding by the EPSRC* to work on this. Her project is a nice example of someone who is bringing together two different areas in her work. She is mixing software development (tech skills) with community management (people skills) to support a range of colleagues who use R and might want to contribute to developing it in future, but perhaps don’t feel confident to do so yet.
Development can involve things like fixing bugs, helping to improve the behaviour or efficiency of programs or translating error messages that currently appear on-screen in the English language into different languages. Heather and her colleagues are working with the R community to create a more welcoming environment for ‘newbies’ that encourages participation, particularly from people who are in the community but who are not currently represented or under-represented by the core group and she’s working collaboratively with other community organisations such as R-Ladies, LatinR and RainbowR. Another task she’s involved in is producing an easier-to-follow ‘How to develop R’ guide.
There are also people who work in universities but who aren’t academics (they don’t teach or do research but do other important jobs that help keep things running well) and some of them use R too and can contribute to its development. However their contributions have been less likely to get the proper recognition or career rewards compared with those made by academics, which is a little unfair. That’s largely because of the way the academic system is set up.
Generally it’s academics who apply for funding to do new research, they do the research and then publish papers in academic journals on the research that they’ve done and these publications are evidence of their work. But the important work that supporting staff do in maintaining the software isn’t classified as new research so doesn’t generally make it into the journals, so their contribution can get left out. They also don’t necessarily get the same career support or mentoring for their development work. This can make people feel a bit sidelined or discouraged.
To try and fix this and to make things fairer the Society of Research Software Engineering was created to champion a new type of job in computing – the Research Software Engineer (RSE). These are people whose job is to develop and maintain (engineer) the software that is used by academic researchers (sometimes in R, sometimes in other languages). The society wants to raise awareness of the role and to build a community around it. You can find out what’s needed to become an RSE below.
Heather is in a great position to help here too, as she has a foot in each camp – she’s both an Academic and a Research Software Engineer. She’s helping to establish RSEs as an important role in universities while also expanding the diversity of people involved in developing R further, for its long-term sustainability.
Below is an example of a Research Software Engineer role which was advertised at QMUL in April 2024 – you can read the original advert and see a copy of the job description / person specification information which is archived at the “Jobs in Computer Science” website. This advert was looking for an RSE to support a research project “at the intersection of Natural Language Processing (NLP) and multi-modal Machine Learning, with applications in mental health.”
QMUL also has a team of Research Software Engineers and you can read about what they’re working on and their career here (there are also RSEs attached to different projects across the university, as above).
Below are some examples of RSE jobs (these particular vacancies have now closed but you can read about what they were looking for and see if that sort of thing might interest you in the future). The links will take you to a page with the original job advert + any Job Description (JD – what the person would actually be doing) and might also include a Person Specification (PS – the type of person they’re looking for in terms of skills, qualifications and experience) – collectively these are often known as ‘job packs’.
Note that these documents are written for quite a technical audience – the people who’d apply for the jobs will have studied computer science for many years and will be familiar with how computing skills can be applied to different subjects.
1. The Science and Technology Facilities Council (STFC) wanted four Research Software Engineers (who’d be working either in Warrington or Oxford) on a chemistry-related project (‘computational chemistry’ – “a branch of chemistry that uses computer simulation to assist in solving chemical problems”)
2. The University of Cambridge was looking for a Research Software Engineer to work in the area of climate science – “Computational modelling is at the core of climate science, where complex models of earth systems are a routine part of the scientific process, but this comes with challenges…”
3. University College London (UCL) wanted a Research Software Engineer to work in the area of neuroscience (studying how the brain works, in this case by analysing the data from scientists using advanced microscopy).
Subscribe to be notified whenever we publish a new post to the CS4FN blog.
This blog is funded by EPSRC on research agreement EP/W033615/1.
cs4fn 