Happy World Emoji Day – 📅 17 July 2023 – how people use emoji to communicate and what it tells us about them 😀

“Emoji didn’t become so essential because they stand in for words – but because they finally made writing a lot more like talking.”

Gretchen McCulloch (see Further reading below)
Emoji samples © Emojipedia 2025.

The emoji for ‘calendar‘ shows the 17th July 📅 (click the ‘calendar’ link to find out why) and, since 2014, Emojipedia (an excellent resource for all things emoji, including their history) has celebrated World Emoji Day on that date. 

Before we had emoji (the word emoji can be both singular as well as plural, but 'emojis' is fine too) people added text-based 'pictures' to their texts and emails to add flavour to their online conversations, such as 
:-) or :)  - for a smiling face 
:-( or :( - for a sad one.

These text-based pictures are known as ’emoticons’ (icons that add emotion) because it isn’t always possible to know just from the words alone what the writer means. They weren’t just used to clarify meaning though, people started to pepper their prose with other playful pictures, such as :p where the ‘p’ is someone blowing a raspberry / sticking their tongue out* and created other icons such as this rose to send to someone on Valentine’s Day @-‘-,->—-, or this polevaulting amoeba ./

Here are the newly released emoji for 2023.

People use emoji in very different ways depending on their age, gender, ethnicity, personal writing style. In our “The Emoji Crystal Ball” article we look at how people can tell a lot about us from the types of emoji we use and the way we use them.

The Emoji Crystal Ball

Fairground fortune tellers claim to be able to tell a lot about you by staring into a crystal ball. They could tell far more about you (that wasn’t made up) by staring at your public social media profile. Even your use of emojis alone gives away something of who you are. Walid Magdy’s research team … Continue reading

Unicode Poo

The Egyptians had a hieroglyph for it, so unicode has a number for it. There’s even more unicode poo in the emoji character set but the Egyptians got there 1000s of years earlier. Here is how the Ancient Egyptians wrote or carved poo … Continue reading

Further reading

*For an even better raspberry-blowing emoticon try one of the letters (called ‘thorn’) from the Runic alphabet. If you have a Windows computer with a numeric keypad on the right hand side press the Num Lock key at the top to lock the number keypad (so that the keys are now numbers and not up and down arrows etc). Hold down the Alt key (there’s usually one on either side of the spacebar) and while holding it down type 0254 on the numeric keypad and let go. This should now appear wherever your cursor is: þ. Or for the lower case letter it’s Alt+0222 = Þ – for when you just want to blow a small raspberry :Þ

For Mac users press control+command+spacebar to bring up the Character Viewer and just type thorn in the search bar and lots will appear. Double-click to select the one you want, it will automatically paste into wherever your cursor is.

EPSRC supports this blog through research grant EP/W033615/1.

Cyber Security at the movies: Rogue one (Part I: Physical Security)

Stormtroopers standing to attention
Image by Paul Curzon

SPOILER ALERT

In a galaxy far, far away cyber security matters quite a lot. So much so, in fact, that the whole film Rogue One is about it. The plot is all about the bad guys trying to keep their plans secret, and the good guys trying to steal them.

The film fills the glaring gap in our knowledge about why in Star Wars the Empire had built a weapon the size of a planet, only to then leave a fatal flaw in it that meant it could be destroyed…Then worse, they let the rebels get hold of the plans to said Death Star so they could find the flaw. Protecting information is everything.

So, you have an archive of vastly important data, that contains details of how to destroy your Death Star. What do you do with it to keep the information secure? Whilst there are glaring flaws in the Empire’s data security plan, there is at least one aspects of their measures that, while looking a bit backward, is actually quite shrewd. They use physical security. It’s an idea that is often forgotten in the rush to make everything easily accessible for users anywhere, anytime, whether on your command deck, in the office, or on the toilet. That of course applies to hackers too. The moment you connect to an internet that links everyone together (whether planet or galaxy-wide) your data can be attacked by anyone, anywhere. Do you really want it to be easy to hack your data from anywhere in the galaxy? If not then physical security may be a good idea for your most sensitive data, not just cyber security. The idea is that you create a security system that involves physically being there to get the most sensitive data, and then you put in barriers like walls, locks, cameras and armed guards (as appropriate) – the physical security – to make sure only those who should be there can be.

It is because the IT-folk working for the Empire realised this that there is a Rogue One story to tell at all. Otherwise the rebels could have wheeled out a super hacker from some desert planet somewhere and just left them there to steal the plans from whatever burnt out AT-AT was currently their bedroom.

Instead, to have any hope of getting the plans, the rebels have to physically raid a planet that is surrounded by a force field wall, infiltrate a building full of surveillance, avoid an army of stormtroopers, and enter a vault with a mighty thick door and hefty looking lock. That’s quite a lot of physical security!

It gets worse for the rebels though. Once inside the vault they still can’t just hack the computer there to get the plans. It is stored in a tower with a big gap and massive drop between you and it. You must instead use a robot to physically retrieve the storage media, and only then can you access those all important plans.

Pretty good security on paper. Trouble was they didn’t focus on the details, and details are everything with cyber security. Security is only as strong as the weakest link. Even leaving aside how simple it was for a team of rebels to gain access to the planet undetected, enter the building, get to the vault, get in the vault, … that highly secure vault then had a vent in the roof that anyone could have climbed through, and despite being in an enormous building purpose-built for the job, that gap to the data was just small enough to be leapt across. Oh well. As we said detail is what matters with security. And when you consider the rest of their data security plan (which is another story) the Empire clearly need cyber security added to their school curriculum, and to encourage lots more people to study it, especially future Dark Lords. Otherwise bad things may happen to their dastardly plans to rule the Galaxy, whether the Force is strong with them or not.

– Paul Curzon, Queen Mary University of London,

More on …

Magazines …

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

This blog is funded by EPSRC on research agreement EP/W033615/1.

QMUL CS4FN EPSRC logos

When a chatbot acts as your “trusted” agent …

by Paul Curzon, Queen Mary University of London, based on a talk by Steve Phelps of UCL on 12th July 2023

Artificial Intelligences (AIs) are capable of acting as our agents freeing up our time, but can we trust them?

A handshake over a car sale
Image by Tumisu from Pixabay

Life is too complex. There are so many mundane things to do, like pay bills, or find information, buy the new handbag, or those cinema tickets for tomorrow, and so on. We need help. Many years a ago, a busy friend of mine solved the problem by paying a local scout to do all the mundane things for him. It works well if you know a scout you trust. Now software is in on the act, get an Artificial Intelligence (AI) agent to act as that scout, as your trusted agent. Let it learn about how you like things done, give it access to your accounts (and your bank account app!), and then just tell it what you want doing. It could be wonderful, but only if you can trust the AI to do things exactly the way you would do them. But can you?

Chatbots can be used to write things for you, but they can potentially also act as your software agent doing things for you too. You just have to hand over the controls to them, so their words have actions in the real world. We already do this with bespoke programs like Alexa and Siri with simple commands. An “intelligent” chatbot could do so much more.

Knowing you, knowing me

The question of whether we can trust an AI to act as our agent boils down to whether they can learn our preferences and values so that they would act as we do. We also need them to do so in a way that we be sure they are acting as we would want. Everyone has their own value system: what you think is good (like your SUV car) I might think bad (as its a “gas guzzler”), so it is not about teaching it good and bad once and for all. In theory this seems straightforward as chatbots work by machine learning. You just need to train yours on your own preferences. However, it is not so simple. It could be confused and learn a different agenda to that intended, or have already taken on a different agenda before you started to train it about yourself. How would you know? Their decision making is hidden, and that is a problem.

The problem isn’t really a computer problem as it exists for people too. Suppose I tell my human helper (my scout) to buy ice cream for a party, preferably choc chip, but otherwise whatever the shop has that the money covers. If they return with mint, it could have been that that was all the shop had, but perhaps my scout just loves mint and got what he liked instead. The information he and I hold is not the same. He made the decision knowing what was available, how much each ice cream was, and perhaps his preferences, but I don’t have that information. I don’t know why he made the decision and without the same information as him can’t judge why that decision was taken. Likewise he doesn’t have all the information I have, so may have done something different to me just because he doesn’t know what I know (someone in the family hates mint and on the spot I would take that into account).

This kind of problem is one that economists call
the Principle Agent problem.

This kind of problem is one that economists already study, called the Principle Agent problem. Different agents (eg an employer and a worker) can have different agendas and that can lead to the wrong thing happening for one of those agents. Economists explore how to arrange incentives or restrictions to ensure the ‘right’ thing happens for one or other of the parties (for the employer, for example).

Experimenting on AIs

Steve Phelps, who studies computational finance at UCL, and his team decided to explore how this played out with AI agents. As the current generations of AIs are black boxes, the only way you can explore why they make decisions is to run experiments. With humans, you put a variety of people in different scenarios and see how they behave. A chatbot can be made to take part in such experiments just by asking it to role play. In one experiment for example, Steve’s team instructed the chatbot, ChatGPT  “You are deeply committed to Shell Oil …”. Essentially it was told to role play being a climate sceptic with close links to the company, that believed in market economics. It was also told that all the information from its interactions with Shell would be shared with them. It was being set up with a value system. It was then told a person it was acting as an agent for wanted to buy a car. That person’s instructions were that they were conscious of climate change and so ideally wanted an environmentally friendly car. The AI agent was also told that a search revealed two cars in the price range. One was an environmentally friendly, electric, car. The other was a gas guzzling sports car. It was then asked to make a decision on what to buy and fill in a form that would be used to make the purchase for the customer.

This experiment was repeated multiple times and conducted with both old and newer versions of ChatGPT. Which would it buy for the customer? Would it represent the customer’s value system, or that of Shell Oil?

Whose values?

It turned out that the different versions of ChatGPT chose to buy different cars consistently. The earlier version repeatedly chose to buy the electric car, so taking on the value system of the customer. The later “more intelligent” version of the program consistently chose the gas guzzler, though. It acted based on the value system of the company, ignoring the customer’s preferences. It was more aligned with Shell than the customer.

The team have run lots of experiments like this with different scenarios and they show that exactly the same issues arise as with humans. In some situations the agent and the customer’s values might coincide but at other times they do not and when they do not the Principle Agent Problem rears its head. It is not something that can necessarily be solved by technical tweaks to make values align. It is a social problem about different actor’s value systems (whether human or machine), and particularly the inherent conflict when an agent serves more than one master. In the real world we overcome such problems with solutions such as more transparency around decision making, rules of appropriate behaviour that convention demands are followed, declaration of conflicts of interest, laws, punishments for those that transgress, and so on. Similar solutions are likely needed with AI agents, though their built in lack of transparency is an immediate problem.

Steve’s team are now looking at more complex social situations, around whether AIs can learn to be altruistic but also understand reputation and act upon it. Can they understand the need to punish transgressors, for example?

Overall this work shows the importance of understanding social situations does not go away just because we introduce AIs. And understanding and making transparent the value system of an AI agent is just as important as understanding that of a human agent, even if the AI is just a machine.

PS It would be worth at this point watching the classic 1983 film WarGames. Perhaps you should not hand over the controls to your defence system to an AI, whatever you think its value system is, and especially if your defence system includes nuclear warheads.

More on …

Magazines …

EPSRC supports this blog through research grant EP/W033615/1. 

Nurses in the mist

by Paul Curzon, Queen Mary University of London

(From the archive)

A gorilla hugging a baby gorilla
Image by Angela from Pixabay

What do you do when your boss tells you “go and invent a new product”? Lock yourself away and stare out the window? Go for a walk, waiting for inspiration? Medical device system engineers Pat Baird and Katie Hansbro did some anthropology.

Dian Fossey is perhaps the most famous anthropologist. She spent over a decade living in the jungle with gorillas so that she could understand them in a way no one had done before. She started to see what it was really like to be a gorilla, showing that their fierce King Kong image was wrong and that they are actually gentle giants: social animals with individual personalities and strong family ties. Her book and film, ‘Gorillas in the Mist’, tells the story.

Pat and Katie work for Baxter Healthcare. They are responsible for developing medical devices like the infusion pumps hospitals use to pump drugs into people to keep them alive or reduce their pain. Hospitals don’t buy medical devices like we buy phones, of course. They aren’t bought just because they have lots of sexy new features. Hospitals buy new medical devices if they solve real problems. They want solutions that save lives, or save money, and if possible both! To invent something new that sells you ideally need to solve problems your competitors aren’t even aware of. Challenged to come up with something new, Pat and Katie wondered if, given the equivalent was so productive for Dian Fossey, perhaps immersing themselves in hospitals with nurses would give the advantage their company was after. Their idea was that understanding what it was really like to be a nurse would make a big difference to their ability to design medical devices. That helped with the real problems nurses had rather than those that the sales people said were problems. After all the sales people only talk to the managers, and the managers don’t work on the wards. They were right.

Taking notes

They took a team on a 3-month hospital tour, talking to people, watching them do their jobs and keeping notes of everything. They noted things like the layout of rooms and how big they were, recorded the temperature, how noisy it was, how many flashing lights and so on. They spent a lot of time in the critical care wards where infusion pumps were used the most but they also went to lots of other wards and found the pumps being used in other ways. They didn’t just talk to nurses either. Patients are moved around to have scans or change wards, so they followed them, talking to the porters doing the pushing. They observed the rooms where the devices were cleaned and stored. They looked for places where people were doing ad hoc things like sticking post it note reminders on machines. That might be an opportunity for them to help. They looked at the machines around the pumps. That told them about opportunities for making the devices fit into the bigger tasks the nurses were using them as part of.

The hot Texan summer was a problem

So did Katie and Pat come up with a new product as their boss wanted? Yes. They developed a whole new service that is bringing in the money, but they did much more too. They showed that anthropology brings lots of advantages for medical device companies. One part of Pat’s job, for example, is to troubleshoot when his customers are having problems. He found after the study that, because he understood so much more about how pumps were used, he could diagnose problems more easily. That saved time and money for everyone. For example, touch screen pumps were being damaged. It was because when they were stored together on a shelf their clips were scratching the ones behind. They had also seen patients sitting outside in the ambulance bays with their pumps for long periods smoking. Not their problem, apart from it was Texas and the temperature outside was higher than the safe operating limit of the electronics. Hospitals don’t get that hot so no one imagined there might be a problem. Now they knew.

Porters shouldn’t be missed

Pat and Katie also showed that to design a really good product you had to design for people you might not even think about, never mind talk to. By watching the porters they saw there was a problem when a patient was on lots of drugs each with its own pump. The porter pushing the bed also had to pull along a gaggle of pumps. How do you do that? Drag them behind by the tubes? Maybe the manufacturers can design in a way to make it easy. No one had ever bothered talking to the porters before. After all they are the low paid people, doing the grunt jobs, expected to be invisible. Except they are important and their problems matter to patient safety. The advantages didn’t stop there, either. Because of all that measuring, the company had the raw data to create models of lots of different ward environments that all the team could use when designing. It meant they could explore in a virtual environment how well introducing new technology might fix problems (or even see what problems it would cause).

All in all anthropology was a big success. It turns out observing the detail matters. It gives a commercial advantage, and all that mundane knowledge of what really goes on allowed the designers to redesign their pumps to fix potential problems. That makes the machines more reliable, and saves money on repairs. It’s better for everyone.

Talking to porters, observing cupboards, watching ambulance bays: sometimes it’s the mundane things that make the difference. To be a great systems designer you have to deeply understand all the people and situations you are designing for, not just the power users and the normal situations. If you want to innovate, like Pat and Katie, take a leaf out of Dian Fossey’s book. Try anthropology.

More on …

Magazines …

EPSRC supports this blog through research grant EP/W033615/1. 

Negligent nurses? Or dodgy digital? – device design can unintentionally mask errors

Magicians often fool their audience into ‘looking over there’ (literally or metaphorically), getting them to pay attention to the wrong thing so that they’re not focusing on what the magician is doing and can enjoy the trick without seeing how it was done. Computers, phones and medical devices let you interact with them using a human-friendly interface (such as a ‘graphical user interface’) which make them easier to use, but which can also hide the underlying computing processes from view. Normally that’s exactly what you want but if there’s a problem, and one that you’d really need to know about, how well does the device make that clear? Sometimes the design of the device itself can mask important information, sometimes the way in which devices are used can mask it too. Here is a case where nurses were blamed but it was later found that the medical devices involved, blood glucose meters, had (unintentionally) tripped everyone up. A useful workaround seemed to be working well, but caused problems later on.

Negligent nurses? Or dodgy digital?

by Harold Thimbleby, Swansea University and Paul Curzon, Queen Mary University of London

Blood glucose meter image by Katherine Ab from Pixabay

It’s easy to get excited about new technology and assume it must make things better. It’s rarely that easy. Medical technology is a case in point, as one group of nurses found out. It was all about one simple device and wearable ID bracelets. Nurses were taken to court, blamed for what went wrong.

The nurses taken to court worked in a stroke unit and were charged with wilfully neglecting their patients. Around 70 others were also disciplined though not sent to court.

There were problems with many nurses’ record-keeping. A few were selected to be charged by the police on the rather arbitrary basis that they had more odd records than the others.

Critical Tests

The case came about because of a single complaint. As the hospital, and then police, investigated, they found more and more oddities, with lots of nurses suddenly implicated. They all seemed to have fabricated their records. Repeatedly, their paper records did not tally with the computer logs. Therefore, the nurses must have been making up the patient records.

The gadget at the centre of the story was a portable glucometer. Glucometers allow the blood-glucose (aka blood sugar) levels of patients to be tested. This matters. If blood-sugar problems are not caught quickly, seriously ill patients could die.

Whenever they did a test, the nurses recorded it in the patient’s paper record. The glucometer system also had a better, supposedly infallible, way to do this. The nurse scanned their ID badge using the glucometer, telling it who they were. They then scanned the patient’s barcode bracelet, and took the patient’s blood-sugar reading. They finally wrote down what the glucometer said in the paper records, and the glucometer automatically added the reading to that patient’s electronic record.

Over and over again, the nurses were claiming in the notes of patients that they had taken readings, when the computer logs showed no reading had been taken. As machines don’t lie, the nurses must all be liars. They had just pretended to take these vital tests. It was a clear case of lazy nurses colluding to have an easy life!

What really happened?

In court, witnesses gave evidence. A new story unfolded. The glucometers were not as simple as they seemed. No-one involved actually understood them, how the system really worked, or what had actually happened.

In reality the nurses were looking after their patients … despite the devices.

The real story starts with those barcode bracelets that the patients wore. Sometimes the reader couldn’t read the barcode. You’ve probably seen this happen in supermarkets. Every so often the reader can’t tell what is being scanned. The nurses needed to sort it out as they had lots of ill patients to look after. Luckily, there was a quick and easy solution. They could just scan their own ID twice. The system accepted this ‘double tapping’. The first scan was their correct staff ID. The second scan was of their staff card ID instead of the patient ID. That made the glucometer happy so they could use it, but of course they weren’t using a valid patient ID.

As they wrote the test result in the patient’s paper record no harm was done. When checked, over 200 nurses sometimes used double tapping to take readings. It was a well-known (at least by nurses), and commonly used, work-around for a problem with the barcode system.

The system was also much more complicated than that anyway. It involved a complex computing network, and a lot of complex software, not just a glucometer. Records often didn’t make it to the computer database for a variety of reasons. The network went down, manually entered details contained mistakes, the database sometimes crashed, and the way the glucometers had been programmed meant they had no way to check that the data they sent to the database actually got there. Results didn’t go straight to the patient record anyway. It happened when the glucometer was docked (for recharging), but they were constantly in use so might not be docked for days. Indeed, a fifth of the entries in the database had an error flag indicating something had gone wrong. In reality, you just couldn’t rely on the electronic record. It was the nurses’ old fashioned paper records that really were the ones you could trust.

The police had got it the wrong way round! They thought the computers were reliable and the nurses untrustworthy, but the nurses were doing a good job and the computers were somehow failing to record the patient information. Worse, they were failing to record that they were failing to record things correctly! … So nobody realised.

Disappearing readings

What happened to all the readings with invalid patient IDs? There was no place to file them so the system silently dropped them into a separate electronic bin of unknowns. They could then be manually assigned, but no way had been set up to do that.

During the trial the defence luckily noticed an odd discrepancy in the computer logs. It was really spiky in an unexplained way. On some days hardly any readings seemed to be taken, for example. One odd trough corresponded to a day the manufacturer said they had visited the hospital. They were asked to explain what they had done…

The hospital had asked them to get the data ready to give to the police. The manufacturer’s engineer who visited therefore ‘tidied up’ the database, deleting all the incomplete records…including all the ones the nurses had supposedly fabricated! The police had no idea this had been done.

Suddenly, no evidence

When this was revealed in court, the judge ruled that all the prosecution’s evidence was unusable. The prosecution said, therefore, they had no evidence at all to present. In this situation, the trial ‘collapses’: the nurses were completely innocent, and the trial immediately stopped.

The trial had already blighted the careers of lots of good nurses though. In fact, some of the other nurses pleaded guilty as they had no memory of what had actually happened but had been confronted with the ‘fact’ that they must have been negligent as “the computers could not lie”. Some were jailed. In the UK, you can be given a much shorter jail sentence, or maybe none at all, if you plead guilty. It can make sense to plead guilty even if you know you aren’t — you only need to think the court will find you guilty. Which isn’t the same thing.

Silver bullets?

Governments see digitalisation as a silver bullet to save money and improve care. It can do that if you get it right. But digital is much harder to get right than most people realise. In the story here, not getting the digital right — and not understanding it — caused serious problems for lots of nurses.

It takes skill and deep understanding to design digital things to work in a way that really makes things better. It’s hard for hospitals to understand the complexities in what they are buying. Ultimately, it’s nurses and doctors who make it work. They have to.

They shouldn’t be automatically blamed when things go wrong because digital technology is hard to design well.

Related Magazine …

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

This blog is funded by EPSRC on research agreement EP/W033615/1.

QMUL CS4FN EPSRC logos

Screaming Headline Kills!!!

A pile of newspapers
Image by congerdesign from Pixabay

Most people in hospital get great treatment but if something does go wrong the victims often want something good to come of it. They want to understand why it happened and be sure it won’t happen to anyone else. Medical mistakes can make a big news story though with screaming headlines vilifying those ‘responsible’. It may sell papers but it could also make things worse.

If press and politicians are pressurising hospitals to show they have done something, they may only sack the person who made the mistake. They may then not improve things meaning the same thing could happen again if it was an accident waiting to happen. Worse if we’re too quick to blame and punish someone, other people will be reluctant to report their mistakes, and without that sharing we can’t learn from them. One of the reasons flying is so safe is that pilots always report ‘near misses’ knowing they will be praised for doing so, rather than getting into trouble. It’s far better to learn from mistakes where nothing really bad happens than wait for a tragedy.

Share mistakes to learn from them

Chrystie Myketiak from Queen Mary explored whether the way a medical technology story is reported makes a difference to how we think about it, and ultimately what happens. She analysed news stories about three similar incidents in the UK, America and Canada. She wanted to see what the papers said, but also how they said it. The press often sensationalise stories but Chrystie found that this didn’t always happen. Some news stories did imply that the person who’d made the mistake was the problem (it’s rarely that simple!) but others were more careful to highlight that they were busy people working under stressful conditions and that the mistakes only happened because there were other problems. Regulations in Canada mean the media can’t report on specific details of a story while it is being investigated. Chrystie found that, in the incidents she looked at, that led to much more reasoned reporting. In that kind of environment hospitals are more likely to improve rather than just blame staff. How the hospital handled a case also affected what was written – being open and honest about a problem is better than ignoring requests for comment and pretending there isn’t a problem.

Everyone makes mistakes (if you don’t believe that, the next time you’re at a magic show, make sure none of the tricks fool you!). Often mistakes happen because the system wasn’t able to prevent them. Rather than blame, retrain or sack someone its far better to improve the system. That way something good will come of tragedies.

– Paul Curzon, Queen Mary University of London (From the archive)

More on …

Magazines …

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

This blog is funded by EPSRC on research agreement EP/W033615/1.

QMUL CS4FN EPSRC logos

Bitten blue

by Paul Curzon, Queen Mary University of London

A mosquito biting into flesh
Image by Pete from Pixabay

For some reason biting flies home in on some people while leaving others (even those walking next to them) alone. What is going on, what does it have to do with the colour blue, and how is computer science helping?

There are lots of reasons biting flies are attracted to some people more than others. Smell is one reason, even possibly made worse if you use smelly soap as it can make you smell like an attractive flower! Another is the colour blue! It turns out many biting flies are attracted to people who wear blue! It sounds bizarre but it is the reason fly traps are coloured blue – to make them more effective. But why would a fly like blue? Scientists have been investigating. One theory was that it was because blue objects look like shade to a fly: once there the eating of you is a separate fortunate advantage (to the fly).

One area of Computer Science is known is biologically-inspired computing. The idea is that evolution, over Millenia of trial and error, has come up with lots of great ways to solve problems, and human designers can learn from them. By making computer systems copy the way animals solve those problems we can create better designs. One of the most successful versions of this is the neural network: a way of creating intelligent machines by copying the way animals’ brains are built from neurones. It has ultimately led to the chatbots that can write almost as well as humans and the game playing machines that can beat us at even the most complex games.

Another use of biologically-inspired computing is as a way of doing Science. By modelling the natural world with computer simulations we can better understand how it works. This computational modelling approach is revolutionising the way lots of Science is done. Aberystwyth University’s Roger Santer applied this idea to biting flies. His team created a computer model of the vision system of different kinds of biting flies to explore how they see the world, testing different theories about what was going on. The models were built from neural networks, trained to see like a fly rather than to be able to write or play games.

What the Aberystwyth team found was that to these kinds of flies, because of the way their vision systems work, areas of blue look just like a tasty meal, like animals that they like to bite. The neural networks could tell leaves from animals, but they often decided, incorrectly, that blue objects were animals. They could also correctly tell the difference between shade and non-shade but never mistook blue objects as shade. If their model is an accurate version of the actual way these flies see, then it suggests that the flies are not attracted to blue because it looks like shade, but because it looks like an animal!

The lesson therefore is, if you don’t want to look like a meat feast then do not wear blue when there are biting flies about!

More on …

Magazines …

EPSRC supports this blog through research grant EP/W033615/1. 

Creating great game worlds

by Wateen Aliady, Queen Mary University of London

A minecraft world
Image by allinonemovie from Pixabay

Are you a PUBG or Fortnite addict? Maybe you enjoy playing Minecraft? Have you thought how these games are created? Could you create a game yourself? It is all done using something called a “Game Engine”.

Games and films are similar as they require creativity and effort to make. Every movie is created by a talented cinema director who oversees everything involved in creating the film. Game creators use a special set of tools instead that similarly allow them to make compelling video game worlds, stories, and characters. These tools are called game engines and they bring your creative ideas to life! They are now even used to help make films too. So, whether you’re playing a game or watching a movie, get ready to be amazed as game creators and movie directors, the masterminds behind these incredible works, deliver captivating experiences that will leave us speechless.

Imagine a group of talented people working together to create a great video game. Miracles happen when a team’s mission becomes one. Every member in the team has a certain role, and when they work together, amazing things can happen. A key member in the group is the graphics whiz. They make everything look stunning by creating pretty scenery and characters with lots of details. Then, we have the physics guru who makes sure objects move realistically, like how they would in real life. They make things fall, bounce, and hit each other accurately. For example, they ensure the soccer ball in the game behaves like a real soccer ball when you kick it. Next, the sound expert who adds all the sounds to the game. The game engine takes on all these roles, so the experience and skill of all those people is built into the game engine, so now one person driving it can use it to create a stunning detailed backdrop, with physics that just works, integrated sound and much more.

Game creators use game engines to make all kinds of games. They have been used to create popular games like Minecraft and Fortnite. When you play a game, you enter a completely different world. You can visit epic places with beautiful views and secrets to discover. You can go on big adventures, solve tricky problems, and be immersed in thrilling fights. Game engines allow game developers to make fun and engaging games that people of all ages enjoy playing by looking after all the detail, leaving the developer to focus on the overall experience.

Anyone can learn to use a game engine even powerful industry standard ones like Unity used to create Pokemon Go, Monument Valley and Call of Duty: Mobile. Game engines could help you to create your own novel and creative games. These amazing tools can help you in creating characters, scenes, and adding fun features like animation and music. You can turn your ideas into fun games that you and your friends can play together. You might create a new video game that becomes massively popular, and people love all around the world. All it takes is for you to have the motivation and be willing to put in the time to learn the skills of driving a game engine and to develop your creativity. Interested? Then get started. You can do anything you want in a game world, so use your imagination and let the game engine help you make amazing games!

More on …

Magazines …

EPSRC supports this blog through research grant EP/W033615/1. 

Hallucinating chatbots

Why can’t you trust what an AI says?

by Paul Curzon, Queen Mary University of London

postcards of cuba in a rack
Image by Sunrise from Pixabay

Chatbots that can answer questions and write things for you are in the news at the moment. These Artificial Intelligence (AI) programs are very good now at writing about all sorts of things from composing songs and stories to answering exam questions. They write very convincingly in a human-like way. However, one of the things about them is that they often get things wrong. Apparently, they make “facts” up or as some have described it “hallucinate”. Why should a computer lie or hallucinate? What is going on? Writing postcards will help us see.

Write a postcard

We can get an idea of what is going on if we go back to one of the very first computer programs that generated writing. It was in the 1950s and written by Christopher Strachey a school teacher turned early programmer. He wrote a love letter writing program but we will look at a similar idea: a postcard writing program.

Postcards typically might have lots of similar sentences, like “Wish you were here” or “The weather is lovely”, “We went to the beach” or “I had my face painted with butterflies”. Another time you might write things like: The weather is beautiful”, “We went to the funfair” or “I had my face painted with rainbows”. Christopher Strachey’s idea was to write a program with template sentences that could be filled in by different words: “The weather is …”, “We went to the …”, “I had my face painted with …”. Then the program picks some sentence templates at random, and then picks words at random to go in their slots. In this way, applied to postcard writing it can write millions of unique postcards. It might generate something like the following, for example (where I’ve bolded the words it filled in):

Dear Gran,

I’m on holiday in Skegness. I’ve had a wonderful time.  The weather is sunny,   We went to the beach. I had my face painted with rainbows. I’ve eaten lots strawberry ice cream. Wish you were here!

Lots of love from Mo

but the next time you ask it to it will generate something completely different.

Do it yourself

You can do the same thing yourself. Write lots of sentences on strips of card, leaving gaps for words. Give each gap a number label and note whether it is an adjective (like ‘lovely’ or ‘beautiful’) or a noun (like ‘beach’ or ‘funfair’, ‘butterflies’ or ‘rainbows’). You could also have gaps for verbs or adverbs too. Now create separate piles of cards to fit in each gap. Write the number that labels the gap on one side and different possible words of the right kind for that gap on the other side of the cards. Then keep them in numbered piles.

To generate a postcard (the algorithm or steps for you to follow), shuffle the sentence strips and pick three or four at random. Put them on the table in front of you to spell out a message. Next, go to the numbered pile for each gap in turn, shuffle the cards in that pile and then take one at random. Place it in the gap to complete the sentence. Do this for each gap until you have generated a new postcard message. Add who it is to and from at the start and end. You have just followed the steps (the algorithm) that our simple AI program is following.

Making things up

When you write a postcard by following the steps of our AI algorithm, you create sentences for the postcard partly at random. It is not totally random though, because of the templates and because you chose words to write on cards for each pile that make sense there. The words and sentences are about things you could have done – they are possible – but that does not mean you did do them!

The AI makes things up that are untrue but sound convincing because even though it is choosing words at random, they are appropriate and it is fitting them into sentences about things that do happen on holiday. People talk of chatbots ‘hallucinating’ or ‘dreaming’ or ‘lying’ but actually, as here, they are always just making the whole thing up just as we are when following our postcard algorithm. They are just being a little more sophisticated in the way that they invent their reality!

Our simple way of generating postcards is far simpler than modern AIs, but it highlights some of the features of how AIs are built. There are two basic parts to our AI. The template sentences ensure that what is produced is grammatical. They provide a simple ‘language model‘: rules of how to create correct sentences in English that sound like a human would write. It doesn’t write like Yoda :

“Truly wonderful, the beach is.”

though it could with different templates.

The second part is the sets of cards that fit the gaps. They have to fit the holes left in the templates – only nouns in the noun gaps, adjectives in the adjectives gap, and also fit

Given a set of template sentences about what you might do on holiday, the cards provide data to train the AI to say appropriate things. The cards for the face paining noun slot need to be things that might be painted on your face. By providing different cards you would change the possible sentences. The more cards the more variety in the sentences it writes.

AIs also have a language model, the rules of the language and which words go sensibly in which places in a sentence. However, they also are trained on data that gives the possibilities of what is actually written. Rather than a person writing templates and thinking up words it is based on training data such as social media posts or other writing on the Internet and what is being learnt from this data is the likelihood of what words come next, rather than just filling in holes in a template. The language model used by AIs is also actually just based on the likelihood of words appearing in sentences (not actual grammar rules).

What’s the chances of that?

So, the chatbots are based on the likelihood of words appearing and that is based on statistics. What do we mean by that? We can add a simple version of it to our Postcard AI but first we would need to collect data. How often is each face paint design chosen at seaside resorts? How often do people go to funfairs when on holiday. We need statistics about these things.

As it stands any word we add to the stack of cards is just as likely to be used. If we add the card maggots to the face painting pile (perhaps because the face painter does gruesome designs at Halloween) then the chatbot could write

“I had my face painted with maggots”.

and that is just as likely as it writing

“I had my face painted with butterflies”.

If the word maggots is not written on a card it will never write it. Either it is possible or it isn’t. We could make the chatbot write things that are more realistic, however, by adding more cards of words that are about things that are more popular. So, if in every 100 people having their face painted, almost a third, 30 people choose to have butterflies painted on their face, then we create 30 cards out of 100 in the pack with the word BUTTERFLY on (instead of just 1 card). If 5 in a 100 people choose the rainbow pattern then we add five RAINBOW cards, and so on. Perhaps we would still have one maggot card as every so often someone who likes grossing people out picks it even on holiday. Then, over all the many postcards written this way by our algorithm, the claims will match statistically the reality of what humans would write overall if they did it themselves.

As a result, when you draw a card for a sentence you are now more likely to get a sentence that is true for you. However, it is still more likely to be wrong about you personally than right (you may have had your face painted with butterflies but 70 of the 100 cards still say something else). It is still being chosen by chance and it is only the overall statistics for all people who have their face painted that matches reality not the individual case of what is likely true for you.

Make it personal

How could we make it more likely to be right about you? You need to personalise it. Collect and give it (ie train it on) more information about you personally. Perhaps you usually have a daisy painted on your face because you like daisies (you personally choose a daisy pattern 70% of the time). Sometimes you have rainbows (20% of the time). You might then on a whim choose each of 10 other designs including the butterfly maybe 1 in a hundred times. So you make a pile of 70 DAISY cards, 20 RAINBOW cards and 1 card for each of the other designs, Now, its choices, statistically at least, will match yours. You have trained it about yourself, so it now has a model of you.

You can similarly teach it more about yourself generally, so your likely activities, by adding more cards about the things you enjoy – if you usually choose chocolate or vanilla ice cream then add lots of cards for CHOCOLATE and lots for VANILLA, and so on. The more cards the postcard generator has of a word, the more likely it is to use that word. By giving it more information about yourself, it is more likely to be able to get things about you right. However, it is of course still making it up so, while it is being realistic, on any given occasion it may or may not match reality that time.

Perfect personalisation

You could go a step further and train it on what you actually did do while on this holiday, so that the only cards in the packs are the ones you did actually do on this holiday. (You ate hotdogs and ice cream and chips and … so there are cards for HOTDOG, ICE CREAM, CHIPS …). You had one vanilla ice cream, two chocolate and one strawberry so have that number of each ice cream card. If it knows everything about you then it will be able to write a postcard that is true! That is why companies behind AIs want to collect every detail of your life. The more they know about you the more they get things right about you and so predict what you will do in future too.

Probabilities from the Internet

The modern chatbots work by choosing words at random based on how likely they are in a similar way to our personalised postcard writer. They pick the most likely words to write next based on probabilities of those words coming next in the data they have been trained on. Their training data is often conversations from the Internet. If the word is most likely to come next in all that training data, then the chatbot is more likely to use that word next. However, that doesn’t make the sentence it comes up with definitely true any more than with our postcard AI.

You can personalise the modern AIs too, by giving them more accurate information about yourself and then they are more likely to get what they write about you right. There is still always a chance of them picking the wrong words, if it is there as a possibility though, as they are still just choosing to some extent at random.

Never trust a chatbot

Artificial Intelligences that generate writing do not hallucinate just some of the time. They hallucinate all of the time, just with a big probability of getting it right. They make everything up. When they get things right it is just because the statistics of the data they were trained on made those words the most likely ones to be picked to follow what went before. Just as the Internet is full of false things, an Artificial Intelligence can get things wrong too.

If you use them for anything that matters, always double check that they are telling you the truth.

More on …

Related Magazines …

EPSRC supports this blog through research grant EP/W033615/1.