The audio CAPTCHA

Audio CAPTCHAs are a type of CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart) used to determine if a user is a human or a bot. Audio CAPTCHAs are designed to be accessible to users with visual impairments, who may have difficulty reading visual CAPTCHAs. Hearing-based "I'm not a robot" quizzes play audio clips that read out the text or letters presented in the CAPTCHA. These recordings usually feature some form of background noise, which is intended to fool bots that try to interpret the audio. They used to be fairly effective since first generation bots struggled to properly interpret audio.  They even were used for shadow crowdsourcing, making people work for others without realizing it - eg, translate old newspapers and characters from street images for Google Maps. 

However, the effectiveness of audio CAPTCHAs has been challenged by the improving accuracy of automated speech recognition (ASR) systems, which are able to solve audio CAPTCHAs even in the presence of background noise. As a result, researchers have focused on developing new audio CAPTCHA designs that are more resistant to ASR attacks, as well as evaluating the performance of different audio CAPTCHA designs under various conditions.

Some of the audio CAPTCHA papers have explored the use of different types of audio signals, such as noise and music, to create audio CAPTCHAs that are more difficult for ASR systems to solve. Other papers have examined the use of multiple audio CAPTCHAs in sequence, or the use of visual and audio CAPTCHAs in combination, in an effort to increase the overall security of the CAPTCHA system.

Both text and image-based CAPTCHAs are becoming difficult for humans to resolve. Bruce Schneier, an internationally renowned security technologist, offered this humorous example on his blog: "select all images where if you were to add a decrescendo it would add to the musicality of the piece without being interpreted as an overly heavy-handed metaphor within the context of the thematic material "

ChatGPT answered to this challenge by writing lyrics to short musical pieces incorporating a decrescendo: 

 Verse 1: (Moderately fast tempo, 8th note = 120 bpm) 

The sun is shining bright, 

The birds are singing sweet, 

My heart is filled with joy, 

As I dance down the street. 

Chorus: (8th note = 120 bpm) 

Life is good, life is grand, 

I am free to roam this land. 

Verse 2: (Moderately fast tempo, 8th note = 120 bpm) 

But as the day wears on, The clouds start to roll in. My steps slow down, As the rain begins. 

Chorus: (8th note = 120 bpm) Life is good, life is grand, But sometimes it's hard to understand. 

Bridge:(Moderately slow tempo, 8th note = 60 bpm) The rain falls heavy now, My mood starts to change. I feel a sense of sadness, As I contemplate the pain. 

Outro: (Moderately slow tempo, 8th note = 60 bpm) But even in the darkest times, There is always a way. The rain will eventually stop, And a brighter day will come my way. (Decrescendo from 8th note = 60 bpm to pianissimo) 

In this musical piece, the decrescendo at the end helps to convey a sense of resolution and hopefulness, as the music gradually fades out to a pianissimo dynamic. The use of a decrescendo in this context adds to the musicality of the piece without being overly heavy-handed or interpreting the thematic material in a specific way.

It's a challenge to develop effective CAPTCHAs that are able to differentiate between humans and bots, while also being accessible to all users.

REFERENCES

Alqarni M, Yu F, Raavi R, Sukhai M. An Accessible Web CAPTCHA Design for Visually Impaired Users.

Solanki S, Krishnan G, Sampath V, Polakis J. In (cyber) space bots can hear you speak: Breaking audio captchas using bots speech recognition. InProceedings of the 10th ACM Workshop on Artificial Intelligence and Security 2017 Nov 3 (pp. 69-80).

Reinheimer BM, Islam F, Shumailov I. Audio CAPTCHA with a Few Cocktails: It’s so Noisy I Can’t Hear You. InCambridge International Workshop on Security Protocols 2019 Apr 10 (pp. 20-40). Springer, Cham.

Shirali-Shahreza S, Penn G, Balakrishnan R, Ganjali Y. Seesay and hearsay captcha for mobile interaction. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems 2013 Apr 27 (pp. 2147-2156).

Huang CH, Wu PH, Liu YW, Wu SH. Attacking and Defending Behind A Psychoacoustics-Based Captcha. InICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2021 Jun 6 (pp. 895-899). IEEE.

IG: Special thanks to OpenAI's Assistant for their help with writing this article.

The Brain Filters

Our brains have a built-in "noise cancelling" circuit known as a cortical filter that helps us ignore the sounds of our own movements, such as footsteps. This filter is essential for our daily lives, as it allows us to focus on important sounds and conversations without being distracted by the noise of our own actions (or even voice of people we got used to filter out, like mothers or wives). However, in some cases, this filter may not work properly, leading to conditions such as tinnitus, where the brain struggles to filter out certain sounds. For example, a person with tinnitus may hear a constant ringing or buzzing noise, even in a quiet room. 

Cochlear implant recipients may also struggle to adjust to the new sounds they are able to hear after receiving the implant, as their brains may have difficulty filtering out their own movement sounds at first. However, with time and practice, the brain can learn to use the cortical filter more effectively, allowing people to focus on the sounds that matter most to them.

Sounds that arise from our own movements, such as vocalizing or walking, are known as reafferent sounds. These sounds are essential for normal hearing, as they help us to anticipate and distinguish them from sounds that come from the environment. However, the neural circuits that allow us to learn to anticipate and distinguish these sounds are not well understood. 

In order to study these neural circuits, researchers developed a system called acoustic virtual reality (aVR), in which mice were trained to associate a novel sound with their own movements. By using this system, the researchers were able to identify the neural mechanisms that learn to suppress reafferent sounds and study the behavioral consequences of this predictable sensorimotor experience. 

The researchers found that aVR experience gradually and selectively suppressed auditory cortical responses to the reafferent frequency, in part by strengthening motor cortical activation of auditory cortical inhibitory neurons that respond to the reafferent tone. This plasticity, or ability to change in response to experience, was behaviorally adaptive, as mice that had experienced aVR showed an enhanced ability to detect non-reafferent tones during movement. 

Overall, these findings suggest that there is a dynamic sensory filter in the brain that involves motor cortical inputs to the auditory cortex, and that this filter can be shaped by experience to selectively suppress the predictable acoustic consequences of movement.

REFERENCES

Schneider DM, Sundararajan J, Mooney R. A cortical filter that learns to suppress the acoustic consequences of movement. Nature. 2018 Sep;561(7723):391-5.

Sounds to Cure

Sounds can have a powerful effect on living beings - humans, animals, plants. Relaxing music can have a healing effect. A new study showed this specifically for premature babies.

Live ocean disc whoosh sounds, gato box rhythms, and parent's sung
lullabies - these sounds have been found to be especially beneficial increasing infants' capacity to feed, sleep, and self-regulate.

Baby's hearing develops in the womb. The very first sound unborn baby hears at about 16 weeks of development is the mother's heartbeat. Gato - a small rectangular wooden drum - creates a rhythm in soft timbre that reminds the heartbeat. Ocean waves resemble the fluid sounds of the womb, as well as inhalation and exhalation sounds. And a sweet lullaby could definitely soothe anyone who has problems trying to get to sleep or suffer with insomnia - as this is one of the oldest, most natural forms of human interaction that brings feelings of warmth and togetherness.

REFERENCES
Loewy J, Stewart K, Dassler AM, Telsey A, Homel P.The Effects of Music Therapy on Vital Signs, Feeding, and Sleep in Premature Infants. Pediatrics. 2013 Apr 15. [Epub ahead of print] 

Big Data of Sounds

Hearing begins with the ears, but it's more than the sum of sounds. We are able to recognize piano notes and experience music, understand speech in noisy surroundings and localize voice in 3D.  Is it because the sound transmitted to the inner ear is broken down into frequencies and mapped onto the brain like musical notes are mapped on a piano keyboard? But are not we able to hear different things at once in three dimensions? Are not we translating frequencies into meanings on the fly, processing Big Data better than any famous statistician? As Jacob Oppenheim and Marcelo Magnasco showed that simple decomposition of sounds into its components by Fourier transform loses important information about the sound's duration, something that our brain is actually able to overcome. Humans can beat the limits of Fourier analysis, and the Brain processes the big data of sounds better than existing algorithms do. As a matter of fact, training our brain on music makes us better in math and problem solving skills. While inability to process signals in the brain leads to developmental disorders. More to discover, more to learn.

REFERENCES

Oppenheim, Jacob N., and Marcelo O. Magnasco. "Human Time-Frequency Acuity Beats the Fourier Uncertainty Principle." arXiv preprint arXiv:1208.4611(2012).

Díaz, Begoña, et al. "Dysfunction of the auditory thalamus in developmental dyslexia." Proceedings of the National Academy of Sciences 109.34 (2012): 13841-13846.

Can Your Baby Hear Well?

About 2 to 3 out of every 1,000 children in the United States are born deaf or hard-of-hearing. Nine out of every 10 children who are born deaf are born to parents who can hear. 


Unfortunately existing hearing tests do not always catch hearing loss in newborns. According to a new study, one-third of children later treated for deafness with cochlear implants had actually passed the newborn screening.

The graph on the right shows summary rates of the percentage of newborns screened for hearing loss prior to hospital discharge based on individual State reports posted on the Early Hearing Detection and Intervention Program (EHDI), 1999–2005 Web site. Trend data on newborn hearing screening rates are also available from the DHHS Healthy People 2010 Project (US DHHS, 2006) and from CDC Wonder: Healthy People 2010. Newborn hearing screening rates increase in a linear fashion from 46.1% in 1999 to 82.9% in 2002; thereafter, while the rates continue to increase, the trend line begins to flatten out, reaching 91.5% in 2005. 

So more babies are screened for hearing loss while they are still in the hospital before they go home. 

Yet, as researchers at Children's Memorial Hospital in Chicago found out - by  looking at the records of 127 children who had hearing problems severe enough to be treated with cochlear implants: one-third of these babies had passed the newborn hearing test.

This is not necessarily because the screening is not accurate, but rather the result of the fact that some hearing loss develops slowly. Babies at risk for progressive deafness include those who had cytomegalovirus infections, and those who were in a neonatal intensive care unit.

The earlier children with hearing problems get help, the better they will do at developing language skills. And more testing screens may be needed - Perhaps, even including genetic tests like those developed by Harvard's Center for PErsonalized Genetic Medicine. Or more at-home screening, including iPhone apps or  other inexpensive devices.


References

Nancy Melinda Young, MD; Brian Kip Reilly, MD; Larisa Burke, BA
Limitations of Universal Newborn Hearing Screening in Early Identification of Pediatric Cochlear Implant Candidates. Arch Otolaryngol Head Neck Surg. 2011;137(3):230-234. doi:10.1001/archoto.2011.4


U.S. Department of Health and Human Services. Chapter 28: Vision and Hearing. In:Healthy People 2010 Midcourse Review. Washington, DC: U.S. Government Printing Office, December 2006. (Available at www.healthypeople.gov/


White KR. Research review—Early hearing detection and intervention programs: Opportunities for genetic services. Am J Med Genet 2004; 130A: 29–36.

Temperature-induced hearing loss

There are many reasons why people loose hearing. Environmental factors  - especially prolonged loud noise and infection or injury - are among the leading causes in the US (figure on the right). Drugs, and chemical toxins could also play a role. Genes accounts for less than half of causes. Gene-environment interactions are contributing too. 

• Drug induced - some medications can result in damage to the auditory system with prolonged use. They are called ototoxic. Here are a few drugs that are known to cause hearing loss: aminoglycoside antibiotics (such as streptomycin, neomycin, kanamycin); salicylates in large quantities (aspirin), loop diuretics (Lasix, ethacrynic acid); and drugs used in chemotherapy regimens (cisplatin, carboplatin, nitrogen mustard). Genetics may predispose to vulnerability to drugs.
• Noise-induced hearing loss (NIHL) - this is hearing loss due to exposure to either a sudden, loud noise or exposure to loud noises for a period of time. A dangerous sound is anything that is 85 dB (sound pressure level - SPL) or higher. Genetics may determine vulnerability thresholds to noise.
• Temperature-induced hearing loss.  Genetics could cause temporary hearing loss caused by a high body temperature. They recover some time after their body temperature has returned to normal. Cold weather could also affect hearing (eg, by hardening earwax blocking the ear canal) leading to conditions such as tinnitus. 

It is well known that high fevers cause hearing loss. High fevers account for about 1,200 to 1,800 cases of hearing loss in the US each year. Artificially induced high body temperature - for example by a strenuous workout in a 105-degree room - may be the culprit too.

Mutations in the otoferlin (OTOF) gene - known to cause neurosensory nonsyndromic recessive deafness - have been recently linked to rare temporary hearing loss caused by a high body temperature.

In one of the cases examined, hearing of a young boy in the morning was better than in the afternoon, and temperature measurements showed that his body temperature in the afternoon was generally higher than that in the morning. Hospital examination showed that when his body temperature rose above 36.5°C, the hearing loss was severe (70-80dB HL) and this symptom could last for a whole day. The boy with the temperature-dependent hearing loss had two variant OTOF genes, each of which is not usually seen in normal-hearing people.

Another study of a family with three children whose hearing was sensitive to temperature also linked their ailments to OTOF gene.

References

Fukushima K, Ramesh A, Srisailapathy CR, et al. (1996). An autosomal recessive nonsyndromic form of sensorineural hearing loss maps to 3p-DFNB6. Genome Res. 5 (3): 305–8. PMID 8593615

Varga, R. (2005). OTOF mutations revealed by genetic analysis of hearing loss families including a potential temperature sensitive auditory neuropathy allele Journal of Medical Genetics, 43 (7), 576-581 DOI: 10.1136/jmg.2005.03861

Wang, D., Wang, Y., Weil, D., Zhao, Y., Rao, S., Zong, L., Ji, Y., Liu, Q., Li, J., Yang, H., Shen, Y., Benedict-Alderfer, C., Zheng, Q., Petit, C., & Wang, Q. (2010). Screening mutations of OTOF gene in Chinese patients with auditory neuropathy, including a familial case of temperature-sensitive auditory neuropathy BMC Medical Genetics, 11 (1) DOI: 10.1186/1471-2350-11-79

Marlin, S., Feldmann, D., Nguyen, Y., Rouillon, I., Loundon, N., Jonard, L., Bonnet, C., Couderc, R., Garabedian, E., & Petit, C. (2010). Temperature-sensitive auditory neuropathy associated with an otoferlin mutation: Deafening fever! Biochemical and Biophysical Research Communications, 394 (3), 737-742 DOI: 10.1016/j.bbrc.2010.03.06

Drug-induced hearing loss: new SNPs in old genes

Drug-induced ototoxicity is known and well documented. Merck manual  lists several antibiotics (streptomycin, neomycin, vancomycin), chemotherapeutic drugs (such as cisplatin), diuretics (like ethacrynic acid and furosemide), Quinine and Salicylates among the prime suspects.

The impact of these drugs on individuals depends on their genetic makeup.

Here is a new addition to the collection of genes associated with hearing loss caused by gene-environment-interaction:  Genetic variants in TPMT and COMT have been found to affect hearing of children receiving cisplatin chemotherapy (Ross et al., 2009)

220 drug-metabolism genes are suspected to be responsible for genetic susceptibility to cisplatin-induced hearing loss in children. Metabolism genes such as Glutathione S-transferases and megalin were previously studied. Genotyping of 1,949 SNPs in these candidate genes in an initial cohort of 54 children lead to  identification of new genetic variants in TPMT (rs12201199, P value = 0.00022, OR = 17.0, 95% CI 2.3–125.9) and COMT (rs9332377, P value = 0.00018, OR = 5.5, 95% CI 1.9–15.9.

What do we know about these genes?

TPMT is aS-adenosyl methionine-dependent methyltransferase that converts 6TI to 6MMPr. Its sequence is conserved in human, chimpanzee, dog, cow, rat, chicken, and zebrafish.  Hard to say how our ancestors were exposed to this compounds, but as thiopurines are known inhibitors of de novo purine synthesis and cellular proliferation, it makes sense to have ways to deal with this molecule.

The enzyme is mostly known because it metabolizes thiopurine drugs via S-adenosyl-L-methionine as the S-methyl donor and S-adenosyl-L-homocysteine as a byproduct. Accordingly, genetic polymorphisms that affect this enzymatic activity are correlated with variations in sensitivity and toxicity to drugs such as 6-mercaptopurine within individuals. A pseudogene for this locus is located on chromosome 18q

COMT is a Catechol-O-methyltransferase that catalyzes the transfer of a methyl group from S-adenosylmethionine to catecholamines - chemicals produced in the medulla of the adrenal gland. COMT is also known for its role in the metabolism of catechol drugs used in the treatment of hypertension, asthma, and Parkinson disease.
Location: 22q11.21-q11.23|22q11.21

Now that we know, what can we do with the new SNP information?

Perhaps, investigate cost-effectiveness of genotyping for ototoxycity implications - clinical trial  NCT00521950 is already doing it for Inflammatory Bowel Diseases.