What’s on your mind?

Telepathy is the supposed Extra Sensory Perception ability to read someone else’s mind at a distance. Whilst humans do not have that ability, brain-computer interaction researchers at Stanford have just made the high tech version a virtual reality.

Image by Andrei Cássia from Pixabay

It has long been know that by using brain implants or electrodes on a person’s head it is possible to tell the difference between simple thoughts. Thinking about moving parts of the body gives particularly useful brain signals. Thinking about moving your right arm, generates different signals to thinking about moving your left leg, for example, even if you are paralysed so cannot actually move at all. Telling two different things apart is enough to communicate – it is the basis of binary and so how all computer-to-computer communication is done. This led to the idea of the brain-computer interface where people communicate with and control a computer with their mind alone.

Stanford researchers made a big step forward in 2017, when they demonstrated that paralysed people could move a cursor on a screen by thinking of moving their hands in the appropriate direction. This created a point and click interface – a mind mouse – for the paralysed. Impressively, the speed and accuracy was as good as for people using keyboard applications

Stanford researchers have now gone a step even further and used the same idea to turn mental handwriting into actual typing. The person just thinks of writing letters with an imagined pen on imagined paper, the brain-computer interface then picks up the thoughts of subtle movements and the computer converts them into actual letters. Again the speed and accuracy is as good as most people can type. The paralysed participant concerned could communicate 18 words a minute and made virtually no mistakes at all: when the system was combined with auto-correction software, as we now all can use to correct our typing mistakes, it got letters right 99% of the time.

The system has been made possible by advances in both neuroscience and computer science. Recognising the letters being mind-written involves distinguishing very subtle differences in patterns of neurons firing in the brain. Recognising patterns is however, exactly what Machine Learning algorithms do. They are trained on lots of data and pick out patterns of similar data. If told what letter the person was actually trying to communicate then they can link that letter to the pattern detected. Here each letter will not lead to exactly the same pattern of brain signals firing each time, but they will largely clump together,. Other letters will also group but with slightly different patterns of firings. Once trained, the system works by taking the pattern of brain signals just seen and matching it to the nearest clumping pattern. The computer then guesses that the nearest clumping is the letter being communicated. If the system is highly accurate, as this one was at 94% (before autocorrection), then it means the patterns of most letters are very distinct. A letter being mind-written rarely fell into a brain pattern gap, which would have meant that letter could as easily have been the pattern of one letter as the other.

So a computer based “telepathy” is possible. But don’t expect us all to be able to communicate by mind alone over the internet any time soon. The approach involves having implants surgically inserted into the brain: in this case two computer chips connecting to your brain via 100 electrodes. The operation is a massive risk to take, and while perhaps justifiable for someone with a problem as severe as total paralysis, it is less obvious it is a good idea for anyone else. However, this shows at least it is possible to communicate written messages by mind alone, and once developed further could make life far better for severely disabled people in the future.

Yet again science fiction is no longer fantasy, it is possible, just not in the way the science fiction writers perhaps originally imagined by the power of a person’s mind alone.

Paul Curzon, Queen Mary University of London, Spring 2021.

Smart bags

In our stress-filled world with ever increasing levels of anxiety, it would be nice if technology could sometimes reduce stress rather than just add to it. That is the problem that QMUL’s Christine Farion set out to solve for her PhD. She wanted to do something stylish too, so she created a new kind of bag: a smart bag.

Christine realised that one thing that causes anxiety for a lot of people is forgetting everyday things. It is very common for us to forget keys, train tickets, passports and other everyday things we need for the day. Sometimes it’s just irritating. At other times it can ruin the day. Even when we don’t forget things, we waste time unpacking and repacking bags to make sure we really do have the things we need. Of course, the moment we unpack a bag to check, we increase the chance that something won’t be put back!

Electronic bags

Christine wondered if a smart bag could help. Over the space of several years, she built ten different prototypes using basic electronic kits, allowing her to explore lots of options. Her basic design has coloured lights on the outside of the bag, and a small scanner inside. To use the bag, you attach electronic tags to the things you don’t want to forget. They are like the ones shops use to keep track of stock and prevent shoplifting. Some tags are embedded into things like key fobs, while others can be stuck directly on to an object. Then when you pack your bag, you scan the objects with the reader as you put them in, and the lights show you they are definitely there. The different coloured lights allow you to create clear links – natural mappings – between the lights and the objects. For her own bag, Christine linked the blue light to a blue key fob with her keys, and the yellow light to her yellow hayfever tablet box.

In the wild

One of the strongest things about her work was she tested her bags extensively ‘in the wild’. She gave them to people who used them as part of their normal everyday life, asking them to report to her what did and didn’t work about them. This all fed in to the designs for subsequent bags and allowed her to learn what really mattered to make this kind of bag work for the people using it. One of the key things she discovered was that the technology needed to be completely simple to use. If it wasn’t both obvious how to use and quick and simple to do it wouldn’t be used.

Christine also used the bags herself, keeping a detailed diary of incidents related to the bags and their design. This is called ‘autoethnography’. She even used one bag as her own main bag for a year and a half, building it completely into her life, fixing problems as they arose. She took it to work, shopping, to coffee shops … wherever she went.

Suspicious?

When she had shown people her prototype bags, one of the common worries was that the electronics would look suspicious and be a problem when travelling. She set out to find out, taking her bag on journeys around the country, on trains and even to airports, travelling overseas on several occasions. There were no problems at all.

Fashion matters

As a bag is a personal item we carry around with us, it becomes part of our identity. She found that appropriate styling is, therefore, essential in this kind of wearable technology. There is no point making a smart bag that doesn’t fit the look that people want to carry around. This is a problem with a lot of today’s medical technology, for example. Objects that help with medical conditions: like diabetic monitors or drug pumps and even things as simple and useful as hearing aids or glasses, while ‘solving’ a problem, can lead to stigma if they look ugly. Fashion on the other hand does the opposite. It is all about being cool. Christine showed that by combining design of the technology with an understanding of fashion, her bags were seen as cool. Rather than designing just a single functional smart bag, ideally you need a range of bags, if the idea is to work for everyone.

Now, why don’t I have my glasses with me?

– Paul Curzon, Queen Mary University of London, Autumn 2018

Download Issue 25 of the cs4fn magazine “Technology Worn Out (and about) on Wearable Computing here.

The computer vs the casino: Wearable tech cheating

What happened when a legend of computer science took on the Las Vegas casinos? The answer, surprisingly, was the birth of wearable computing.

There have always been people looking to beat the system, to get that little bit extra of the odds going their way to allow them to clean up at the casino. Over the years maths and technology have been used, from a hidden mechanical arm up your sleeve allowing you to swap cards, to the more cerebral card counting. In the latter, a player remembers a running total of the cards played so they can estimate when high value cards will be dealt. One popular game to try and cheat was Roulette.

A spin of the wheel

Roulette, which comes from the French word ‘little wheel’, involves a dish containing a circular rotating part marked into red and black numbers. A simple version of the game was developed by the French mathematician, Pascal, and it evolved over the centuries to become a popular betting game. The central disc is spun and as it rotates a small ball is thrown into the dish. Players bet on the number that the ball will eventually stop at. The game is based on probability, but like most games there is a house advantage: the probabilities mean that the casino will tend to win more money than it loses.

Gamblers tried to work out betting strategies to win, but the random nature of where the ball stops thwarted them. In fact, the pattern of numbers produced from multiple roulette spins was so random that mathematicians and scientists have used these numbers as a random-number generator. Methods using them are even called Monte Carlo methods after the famous casino town. They are ways to calculate difficult mathematical functions by taking thousands of random samples of their value at different random places.

A mathematical system of betting wasn’t going to work to beat the game, but there was one possible weakness to be exploited: the person who ran the game and threw the ball into the wheel, the croupier.

No more bets please

There is a natural human instinct to spin the wheel and throw the ball in a consistent pattern. Each croupier who has played thousands of games has a slight bias in the speed and force with which they spin the wheel and throw the ball in. If you could just see where the wheel was when the spin started and the ball went in, you could use the short time before betting was suspended to make a rough guess of the area where the ball was more likely to land, giving you an edge. This is called ‘clocking the wheel’, but it requires great skill. You have to watch many games with the same croupier to gain a tiny chance of working out where their ball will go. This isn’t cheating in the same way as physically tampering with the wheel with weights and magnets (which is illegal), it is the skill of the gambler’s observation that gives the edge. Casinos became aware of it, so frequently changed the croupier on each game, so the players couldn’t watch long enough to work out the pattern. But if there was some technological way to work this out quickly perhaps the game could be beaten.

Blackjack and back room

Enter Ed Thorpe, in the 1950s, a graduate student in physics at MIT. Along with his interest in physics he had a love of gambling. Using his access to one of the world’s few room filling IBM computers at the university he was able to run the probabilities in card games and using this wrote a scientific paper on a method to win at Blackjack. This paper brought him to the attention of Claude Shannon, the famous and rather eccentric father of information theory. Shannon loved to invent things: the flame throwing trumpet, the insult machine and other weird and wonderful devices filled the basement workshop of his home. It was there that he and Ed decided to try and take on the casinos at Roulette and built arguably the first wearable computer.

Sounds like a win

The device comprised a pressure switch hidden in a shoe. When the ball was spun and passed a fixed point on the wheel, the wearer pressed the switch. A computer timer, strapped to the wrist, started and was used to track the progress of the ball as it passed around the wheel, using technology in place of human skill to clock the wheel. A series of musical tones told the person using the device where the ball would stop, each tone represented a separate part of the wheel. They tested the device in secret and found that using it gave them a 44% increased chance of correctly predicting the winning numbers. They decided to try it for real … and it worked! However, the fine wires that connected the computer to the earpiece kept breaking, so they gave up after winning only a few dollars. The device, though very simple and for a single purpose, is in the computing museum at MIT. The inventors eventually published the detail in a scientific paper called “The Invention of the First Wearable Computer,” in 1998.

The long arm of the law reaches out

Others followed with similar systems built into shoes, developing more computers and software to help cheat at Blackjack too, but by the mid 1980’s the casino authorities became wise to this way to win, so new laws were introduced to prevent the use of technology to give unfair advantages in casino games. It definitely is now cheating. If you look at the rules for casinos today they specifically exclude the use of mobile phones at the table, for example, just in case your phone is using some clever app to scam the casinos.

From its rather strange beginning, wearable computing has spun out into new areas and applications, and quite where it will go next is anybody’s bet.

– Peter W. McOwan, Queen Mary University of London, Autumn 2018

Sick tattoos

Image by Anand Kumar from Pixabay

Researchers at MIT and Harvard have new skin in the game when it comes to monitoring people’s bodily health. They have developed a new wearable technology in the form of colour- and shape-changing tattoos. These tattoos work by using bio-sensitive inks, changing colour, fading away or appearing under different coloured illumination, depending on your body chemistry. They could, for example, change their colour, or shape as their parts fade away, depending on your blood glucose levels.

This kind of constantly on, constantly working body monitoring ensures that there is nothing to fall off, get broken or run out of power. That’s important in chronic conditions like diabetes where monitoring and controlling blood glucose levels is crucial to the person’s health. The project, called Dermal Abyss, brings together scientists and artists in a new way to create a data interface on your skin.

There are still lots of questions to answer, like how long will the tattoos last and would people be happy displaying their health status to anyone who catches a glimpse of their body art? How would you feel having your body stats displayed on your tats? It’s a future question for researchers to draw out the answer to.

– Peter W. McOwan, Queen Mary University of London, Autumn 2018

One in the eye for wearable tech

Contact lenses, normally used to simply, but usefully, correct people’s vision, could in the future do far more.

Tiny microelectronic circuits, antennae and sensors can now be fabricated and set in the plastic of contact lenses. Researchers are looking at the possibility of using such sensors to sample and transmit the glucose level in the eye moisture: useful information for diabetics. Others are looking at lenses that can change your focus, or even project data onto the lens, allowing new forms of augmented and virtual reality.

Conveniently, you can turn the frequent natural motion from the blinks of your eye into enough power to run the sensors and transmitter, doing away with the need for charging. All this means that smart contact lenses could be a real eye opener for wearable tech.

– Peter W. McOwan, Queen Mary University of London, Autumn 2018

Smart tablets (to swallow)

The first ever smart pill has been approved for use. It’s like any other pill except that this one has a sensor inside it and it comes with a tracking device patch you wear to make sure you take it.

A big problem with medicine is remembering to take it. It’s common for people to be unsure whether they did take today’s tablet or not. Getting it wrong regularly can make a difference to how quickly you recover from illness. Many medicines are also very, very expensive. Mass-produced electronics, on the other hand, are cheap. So could the smart pill be a new, potentially useful, solution? The pill contains a sensor that is triggered when the pill dissolves and the sensor meets your stomach acids. When it does, the patch you wear detects its signal and sends a message to your phone to record the fact. The specially made sensor itself is harmless and safe to swallow. Your phone’s app can then, if you allow it, tell your doctor so that they know whether you are taking the pills correctly or not.

Smart pills could also be invaluable for medical researchers. In medical trials of new drugs, knowing whether patients took the pills correctly is important but difficult to know. If a large number of patients don’t, that could be a reason why the drugs appeared less effective than expected. Smart pills could allow researchers to better work out how regularly a drug needs to be taken to still work. 

More futuristically still, such pills may form part of a future health artificial intelligence system that is personalised to you. It would collect data about you and your condition from a wide range of sensors recording anything relevant: from whether you’ve taken pills to how active you’ve been, your heart rate, blood pressure and so on: in fact anything useful that can be sensed. Then, using big data techniques to crunch all that data about you, it will tailor your treatment. For example, such a system may be better able to work out how a drug affects you personally, and so be better able to match doses to your body. It may be able to give you personalised advice about what to eat and drink, even predicting when your condition could be about to get better or worse. This could make a massive difference to life for those with long term illnesses like rheumatoid arthritis or multiple sclerosis, where symptoms flare up and die away unpredictably. It could also help the doctors who currently must find the right drug and dose for each person by trial and error.

Computing in future could be looking after your health personally, as long as you are willing to wear it both inside and out.

– Paul Curzon, Queen Mary University of London, Spring 2021

Download Issue 27 of the cs4fn magazine on Smart Health here.

This post and issue 27 of the cs4fn magazine have been funded by EPSRC as part of the PAMBAYESIAN project.