Acoustic cues to femininity and masculinity in spontaneous speech Open Access

While central to our perception of voices, there is no comprehensive model that acoustically describes how feminine or masculine a speaking voice is likely to be perceived by a listener. This lack of a reliable description of femininity and masculinity in a voice has direct implications in all applied acoustic efforts related to how a voice is perceived, but we put forward here specifically the field of clinical gender-affirming voice training as a frame for illustrating the benefit offered by a reliable model of these properties of voices.

An increasing number of transgender and gender-diverse persons have been seeking health care services due to gender dysphoria, inferring that their gender identity is not congruent with their assigned gender at birth, associated with significant distress and impaired social functioning.¹ Pursuing a gender expression that is more congruent with the identity of the person can be a way to lessen gender dysphoria, and voice modification toward a better alignment with the gender identity has been put forward as a critical factor for being perceived in accordance with their gender identity by others and increased well-being.^2–4 The speech-language pathologist (SLP) taking on a new client cannot assume that the decision on a particular speech pattern as the target for training will lead to beneficial outcomes in the client's well-being. Some transgender and gender-diverse clients aim for an unequivocally female or male voice expression, that is, a voice that conforms with the voices of cisgender speakers whose gender identity aligns with the biologically and anatomically defined sex (man or woman) they were assigned at birth. For other clients, the gender identity does not align with a binary gender presentation (non-binary), and their preferred gender expression may relate more freely to the perceptual scales of femininity and masculinity.^5,6 SLPs providing gender-affirming voice training, therefore, need solid knowledge about which aspects of voice and speech are most salient for clients' desired vocal expression to successfully deliver the best match between the desired training target and the outcome. Thus, while we envision that a thorough understanding of the acoustic cues listeners may use to determine the femininity or masculinity of voices could be applied in many domains, gender-affirming voice training is put forward here as one domain where a reliable model of perception could be directly applied to result in a more reliable practice.

To provide the foundation for answering the question of what aspects of voice (and speech) are most salient in listener perceptions of speaker gender, Leung et al.⁷ provided a systematic review of 38 peer-reviewed published articles and theses concerned with acoustic cues to speaker gender, or level of femininity or masculinity perceived through the voice. The review identified the most salient acoustic and auditory-perceptual vocal parameters in guiding listeners' perceptions, grouped into three domains: voice (pitch, resonance, loudness, and vocal quality), speech (articulation), and prosody (intonation, tempo, and stress). The notation used for acoustic quantities follows the suggestions by Titze et al.⁸ The fundamental frequency of vocal fold oscillations ( f_o), center frequencies of the first four vowel formants (F₁–F₄), and the intonation patterns used were identified as key features in the perception of a speaker as male/masculine or female/feminine. In addition, aspects of voice quality (e.g., breathiness), prosodic variation (of speech rate and vowel duration), and variation in loudness were auditory-perceptual features that contributed to the perception of a female or feminine voice. Examples of acoustic measures influencing the perception of femininity in voice were seen related to articulation (spectral properties of /s/, and of /ʃ/, which have been recently highlighted as gender markers⁹), voice quality (differences between the level of the second and fourth harmonics, L₂–L₄, and the level of the first harmonic and the third formant frequency, L₁–L_F3), semitone, and dynamic range. The perception of a voice as male or masculine was related to acoustic measures of formant spacing.⁷ 

The acoustic predictors have primarily been studied in terms of their ability to, in isolation, separate speakers with different sexes or genders (main effects) and have only more recently been evaluated in terms of the relative strength of the acoustic cues. Leung et al.¹⁰ recently evaluated the relative contribution and cross-correlations of vowel formant frequencies (F₁–F₃) and the fundamental frequency (f_o) to the perception of a speaker's gender and vocal expression on a femininity–masculinity scale using principal component analysis. They found significant contributions of primarily F₁ and f_o to the model of femaleness and all general levels of f_o and F₁–F₃ to the perception of a voice on a femininity–masculinity scale. This finding was recently replicated in parts by Merritt et al.¹¹ who found that the rated similarity of voices using tasks related to the perception of level of femininity and masculinity was closely connected with f_o, speaker indexed F₁, and vowel space area, and by Södersten et al.¹² who showed that f_o, average formant frequency, and L_eq, most strongly predicted listener ratings of a female sounding voice in trans women before and after gender-affirming voice training and cisgender voices. Leung et al.¹⁰ further found that the listener's initial determination that a voice belonged to a female or male speaker influenced how the voice was perceived in terms of femininity–masculinity. The combined effect of modifying both f_o and F₁–F₄ has been shown to be more likely to change listeners' perceptions compared to shifting one measure alone.^10,13 The analysis of Leung et al.¹⁰ did, however, show a substantial unexplained variation, further emphasizing that the listeners' classification of a speaker in terms of gender, or in terms of level of femininity and masculinity is multidimensional in nature.^7,13 Previous studies have, with a few exceptions,^11,12,14,15 used synthesized speech, or recordings of the speaker reading a standard text¹⁶ or other forms of controlled speech tasks in the data collection.⁷ When spontaneous speech has been analyzed, it has been confirmed that f_o is not a sufficient predictor of the perceived level of femininity and masculinity in voices, and that shifted formant frequencies provide important additional cues.¹⁵ A more extensive evaluation of which acoustic properties serve as cues to the perceived level of femininity and masculinity in spontaneous speech have not been attempted.

The current investigation aimed to determine which of 312 acoustic properties identified in the literature⁷ most accurately predict the level of femininity and masculinity in spontaneous speech. While speaker sex, gender, and the level of perceived femininity and masculinity have not always been treated separately in the previous research on which this study was developed, the dimensions of concern here were the continua of perceived level of femininity and masculinity perceived in the voice, independent of sex assigned at birth or the gender identity of the speaker. A secondary aim was to provide information on how the acoustic predictors co-occur and provide supplementary or supporting cues to the perception of a voice. A penalized regression procedure, the Least absolute shrinkage and selection operator (LASSO), was used to provide linear models with a high likelihood of being generalizable outside the training data. We also wanted to take advantage of the feature selection effect of L1 penalization to build the models from minimally correlated acoustic predictors. The perceived level of femininity and the level of masculinity in voices were assessed separately, with stimuli presented in randomized order, to minimize the potential order effect of stimuli¹⁷ and provide the most reliable and unbiased model.

The study aimed to develop and perceptually evaluate a bank of voices with a full range of levels of femininity and masculinity as perceived by cisgender men and women, transgender and gender-diverse listeners, and professional speech-language pathologists.

Voice samples from 132 adult speakers were recorded in the speaker's homes or another suitable recording environment. The sound level of the surrounding was ensured to be less than 38 dB(A)¹⁸ before recording started. The speech recordings were collected using the application AVR X (Newkline Co., Ltd.) on Android mobile phones. A 44.1 kHz sampling rate and 128 kbps recording rate were used, with automatic gain control to be disabled.¹⁹ To capture the recordings, an omnidirectional RØDE SmartLav+ microphone with a frequency range of 20 Hz to 20 kHz and a signal-to-noise ratio of 67 was used. The microphone was head-mounted 5 cm from the speaker's mouth.²⁰ Calibration was performed at a 5 cm distance using SPL level meter and a reference tone using the Sopran software.²¹ No reference tone was recorded with the speech of the participant. The participants were recruited to cover a wide age range of speakers and collect speech recordings likely to cover the full range of perceived femininity and masculinity levels in their voices. The recruitment effort was therefore directed toward both cisgender speakers using a combination of convenience and snowball sampling by the authors, associated students, and research assistants, and toward transgender and gender-diverse clients starting or undergoing gender-affirming voice training. Speakers were asked to self-report their gender identity by choosing from given options (“man,” “woman,” “non-binary,” “unsure/do not know”), or “other” in which case the listeners were asked to define their gender identity in their own words.

As a result of the recruitment, voice samples were collected from 54 cisgender women (36.1 ± 12.3 years), 41 cisgender men (35.8 ± 12.3 years), and 37 transgender and gender-diverse persons with varying gender identities (28.3 ± 9.9 years). Among the transgender and gender-diverse group, 11 persons were undergoing voice training in a voice masculinizing direction, 11 persons were undergoing voice training toward a more feminine sounding voice, and 15 persons had not taken part in gender-affirming voice training. All speakers were native speakers of Swedish. Information about speakers' hearing, smoking history, gender-affirming hormone therapy, and preexisting disorders that could potentially affect voice or speech, including medically treated asthma, neurological, and neuropsychiatric disorders, was collected. Mild dysphonia was not judged as a reason for exclusion. A few speakers reported a previous or present voice disorder; however, no speaker was judged by the authors to be influenced by any of these factors to a degree that their voice did not reflect typical voices perceived in everyday life.

The speakers were asked to describe some activity or hobby they enjoyed performing. The sample assessed in the perceptual rating included spontaneous speech that was considered free of distracting noise and with content judged by the authors not to contain gendered information. For each speaker, extraction of a sample of approximately 10 s in length and containing at least three phrases was aimed for. The resulting spontaneous speech samples used in the listening experiment contained 3–5 phrases and had an average duration, with standard deviation, of 11.4 ± 3.8 s.

Listeners for the perceptual evaluation of masculinity and femininity in the collected bank of voices were recruited using a combination of broad and targeted strategies. The primary recruitment approach was based on the networks of the researchers and student research assistants associated with the study. In addition, interest groups for transgender and gender-diverse people were approached in the recruitment effort to support a broad representation of gender identities in the sample of listeners. Finally, SLPs providing gender-affirming voice training nationally were approached and asked to participate in the evaluation. All listeners were asked to self-report their gender identity by choosing from given options (“man,” “woman,” “transmasculine,” “transfeminine,” “non-binary,” “unsure/do not know”), or “other” in which case the listeners were asked to define their gender identity in their own words. The listeners were native speakers of Swedish and stated their hearing to be sufficient for verbal communication. Information about preexisting neurologic and neuropsychiatric disorders that could affect voice perception was collected.²² An overview of biographic information for the 121 listeners who took part in the perceptual evaluation and the listening context of their participation is presented in Table I.

All 132 extracted voice samples were rated by listeners two times, in two blocks, with stimuli presentation orders randomized separately for each block and each listener. The block orders were counterbalanced so that every other participant was asked to rate masculinity first, either by the researcher (in person) or by the experiment link they were provided (online). In each trial, the listener was asked to rate either the level of perceived masculinity (Block 1) or femininity (Block 2) in the speaker's voice on a visual analog scale presented in the center of the screen once the stimulus presentation had been completed. The total listening time for each block was 25 min, and the listeners were encouraged to take a short break between the two blocks. Perceptual ratings of femininity and masculinity in the voice samples were managed using the PsychoPy²³ in in-person sessions or, due to the onset of the Covid-19 pandemic, online on the Pavlovia^24,25 experiment presentation platform. The distribution of listeners across online and in-person listening situations is presented in Table I.

In addition to the auditory-perceptual evaluations of the voices, the listeners were asked to report their own age, sex assigned at birth, and gender identity in an online form. Listeners who were speech and language pathologists providing gender-affirming voice training in their profession were asked to indicate this in the form.

The agreement of raters' perception of voices regarding the level of femininity and masculinity was evaluated using the intraclass correlation coefficient (two-way mixed effects model, absolute agreement).²⁶ The strength of the association between listeners' ratings of femininity and masculinity for a particular voice was assessed using Pearson's correlation coefficient.

The study aimed to establish the relative strengths of acoustic measures contributing to the perception of femininity and masculinity in speakers' speech.

Spontaneous speech samples produced by the 132 speakers were submitted to a manual markup procedure to identify all intervals of the speech signal required to compute 312 acoustic measures inferred from previous reports of acoustic properties influencing the perception of femininity or masculinity in the voice.⁷ The manual markup procedure identified the start and end of all individual speech utterances, fricatives, plosives, and vowels in stressed syllables (Table II). Further, the start and end points of plosive bursts and aspiration and the release burst were marked manually from a waveform display of the speech signal. The appropriateness of f_o analysis floor and ceiling settings suggested by the speaker's reported sex assigned at birth were further evaluated visually against the harmonic structure in a narrow-band spectrogram and adjusted to provide the best agreement between the computed f_o and the narrow-band spectrogram display. Similarly, formant frequency ceiling and analysis window size from the settings suggested by the speaker's sex assigned at birth were selected to fit the individual speaker. Individualized f_o and formant frequency extraction settings were noted for subsequent acoustic analyses and listed in supplementary material A for each participant. All analyses were performed using the Praat software package (version 6.3.10); settings not specified in supplementary material A were left at default values.

In addition, each utterance was provided with a description of whether the utterance was perceived as prosodically rising, falling, or produced with a sustained tone by the phrase-final f_o contour. A rising or falling phrase-final intonation was defined as an increase or decrease in at least 2 St as the final tonal movement of a phrase; an even phrase-final intonation was defined as a change of no more than 2 St.^33,34 The final intonation shift³⁵ was calculated as the difference between f_o levels in the vowel in the last stressed syllable and the subsequent unstressed syllable. If the last syllable in the utterance was stressed, the f_o change within this syllable was calculated instead. In rare cases, a change from unstressed words to the last stressed vowel was used as reference points for the f_o measurement to align better with the perceptual impression of the tonal shift at the end of the phrase. Perceptual considerations were determined by consensus by two listeners who were advanced-level SLP students and native speakers of Swedish.

The acoustic description of each speaker's spontaneous speech (Table III) was combined with the femininity and masculinity ratings of the speaker provided by each of the 121 raters and normalized to a unit scale. The 15 972 perceptual evaluations of speakers' level of femininity or masculinity by listeners were split into one dataset used for training and tuning of models (75% training data) and a second set used only for the final evaluation of models (25% evaluation data). Two L1 penalized (LASSO) regression models were developed in the training dataset using separate cross-validation procedures, one for the model of the perceived level of femininity and one for the perceived level of masculinity. The penalization of linear models was employed to scale unimportant predictors and predictors low levels of unique contributions to zero (feature selection), and to guard against overfitting against the training data to ensure generalizability of the models. The model penalization parameters (ℓ_f and ℓ_m for femininity and masculinity models, respectively) were tuned in a tenfold cross-validation procedure. The minimized root mean squared (RMS) error of predictions of masculinity or femininity when compared with the true perceptual ratings of speakers in the holdout 1/10th of the data were used as a criterion for the selection of ℓ_f and ℓ_m. The ten computed models of femininity and masculinity perception were then averaged to derive the final models and evaluated in the 25% evaluation data not used in model training. The variable importance of acoustic predictors for the final models was computed by the Feature Importance Ranking Measure approach.⁴⁰ 

The average rating of femininity and masculinity for the 132 voice samples as rated by the 121 listeners is presented in Fig. 1. Perceptual ratings of voices showed an ICC(A,1) = 0.88 (CI_95% = [0.86, 0.91]) for level of femininity and ICC(A,1) = 0.87 (CI_95% = [0.84, 0.90]) for level of masculinity. A Pearson correlation analysis showed that the independently rated levels of femininity and masculinity in the voice samples were significantly correlated (r = –0.93 CI_95% =  [–0.933, –0.924], p < 0.001). The R² of a linear model of femininity rating as a function of the masculinity rating was 0.863.

The best models were constructed using 223 acoustic predictors. The acoustic models of femininity and masculinity using the acoustic predictors in Table III and perceptual predictors of utterance final rising, falling, and level tones showed a R²_f = 0.89 and R²_m = 0.87 when evaluated in the 25% validation dataset. The models' prediction errors showed no clear pattern of dependence on the part of the masculinity and femininity scales asked to predict (Fig. 2). Supplementary material B presents the variable importance of acoustic properties that contributed to models' ability to predict the perceived level of femininity and masculinity in voice samples not used in model training. The top six predictors “mean f_o of utterances,” “variability in smoothed CPP of utterances,” “median RMS amplitude of plosive release bursts,” “variability in L_H(F2) in vowels in stressed syllables,” “variability in F₄ and F₁ differences in vowels in stressed syllables,” and “variability of F₁ of vowels in stressed syllables,” were consistent across the femininity and masculinity models, with minor differences in relative importance. When evaluated separately in the validation set, these predictors together provided linear models of femininity and masculinity with R²_f = 0.77 and R²_m = 0.76, respectively (87% of the total explained variance of the models). Figure 3 indicates a strong association between the signed variable importance of predictors with non-zero coefficients obtained for the independently trained models of masculinity and femininity. One acoustic property (“variability of smoothed cepstral peak prominence in vowels in stressed syllables”) was indicated to increase both the perceived level of femininity and the perceived level of masculinity in the independently trained models. Two predictors (“median difference between L_F5 and L_F3 in vowels in stressed syllables” and “variability in median RMS amplitude of utterances”) provided small negative contributions to both independently trained acoustic models of femininity and masculinity. The level of correlation between each pair of the predictors is provided in supplementary material C. A total of 0.7% of measure comparisons in supplementary material C indicated an absolute correlation above 0.90 and 3% of comparisons showed an absolute correlation above 0.50.

The aim of this investigation was to determine which acoustic properties discussed in the literature make the most substantial individual contribution to a prediction of the perceived level of femininity and masculinity in voices. A voicebank expected to cover the full range of levels of femininity and masculinity in voices was developed and assessed in a blinded and randomized perceptual evaluation procedure. A comprehensive set of acoustic properties was extracted with speaker-specific settings for identifying f_o and formants before applying them in derived measures. Two models of femininity and masculinity perception were developed from separate perceptual evaluations and in separate training procedures to ensure that findings can be generalized to a broader set of voices to a high degree.

While acoustic properties observed in the literature to contribute to the perception of increased levels of femininity or masculinity in voices have received attention in previous reports,⁷ this study is the first to evaluate which predictors make the most substantial independent contribution to statistical models. The overall explained variances of femininity and masculinity in voices were 89% and 87%, respectively. Thus, the results support the conclusion that the feature elimination that is a result of LASSO regularization managed to retain acoustic features of importance to the modeling of femininity and masculinity in voices not included in the training set. The separate modeling efforts of femininity and masculinity predominately identified the same predictors, but with opposite signs in variable importance. The most prominent acoustic features identified are well attested in the literature, and f_o on a semitone scale was confirmed to be the main acoustic property by which listeners determined the perceived level of femininity and masculinity in the voice. Complementary acoustic predictors were found in other properties of the voice source, the interaction between frequencies and amplitudes of the harmonics and formants in vowels, and in the strength and dynamics of plosive release and aperiodic portions. Some additional perceptual cues to the perception of a level of femininity and masculinity were found in the spectral distribution of fricatives. Unfortunately, all fricative types were not well represented in the spontaneous speech samples that the participants chose to produce. As a result, the importance of spectral properties of fricatives may have been undervalued here. The high levels of explained variances in both models do, however, indicate that, while an increased understanding of the importance of spectral distributions of fricatives may have to be revisited in later research with more control over the spoken content, and ensured sufficient presence of fricatives, the models can still be viewed as acceptable representations of what primarily affects the perception of femininity and masculinity in voices. Finally, it is important to note that what was developed here was an optimal acoustic model of the acoustic predictors of the level of femininity and masculinity in the voice, with competing or duplicating information substituting for the most informative predictor. Clear examples of this are the “proportion of utterances with a falling final tonal movement” and “proportion of utterances with a rising final tonal movement.” The proportion of utterances with falling final tonal movement will be dependent on the proportion of rising, and flat, tonal movement since the sum of them is 1. Therefore, “proportion of utterances with a falling final tonal movement” and “proportion of utterances with a rising final tonal movement” are highly interchangeable once the small perceptual contribution of “proportion of utterances with a flat final tonal movement” has been established. There were further high levels of correlation between a small proportion of measures. Measures that are highly linked in these ways may therefore likely be interchanged if required by the context in which it is to be applied.

In this study, the listeners were asked to rate either the level of femininity or masculinity of a voice in independent, individually randomized procedures. Despite being designed to afford independent judgment of femininity and masculinity level in a voice, the perceptual assessments of voices show strong interdependence and an inverse relationship, and a simple linear model of the masculinity score as a function of the femininity score showed an 86.3% explained variance across all perceptual ratings. The ratings did not, however, show a strong pattern of being dichotomous overall. The data show, instead, that the listeners can place voices on a continuous femininity and masculinity scale with reasonable reliability and use the entire scale if provided with the appropriate voice samples.

The results presented here provide the best available acoustic model of what constitutes an auditory-perceptual cue to perceived femininity and masculinity in voices and suggest that the information can be used in gender-affirming voice training to guide acoustic properties to select as the basis for training evaluation based on a client-selected target voice. What is not, however, offered is guidance on how to achieve the specific property of a voice through voice and speech modification within the boundaries set up by the physical properties of the speech apparatus that a voice client is born with. The continued work in the evaluation of goal achievement in gender-affirming voice training would be greatly benefited by and provide additional external validation to the results and analyses presented here. Further, it should be kept in mind that, while acoustic data are important to researchers and clinicians, patient-reported outcomes are, as has recently been highlighted,^12,17 the most important when evaluating gender affirming voice training. Merrit et al.¹¹ have further recently suggested that gender prototypicality may be a dimension to consider in relation to perceptual data such as ours. We did not ask the listeners how prototypical they perceived voices and collected no data, such as reaction times, of sufficient reliability to serve as a proxy for this determination and cannot elucidate on this aspect of femininity and masculinity perception. We further note that our survey of acoustic properties guiding femininity and masculinity perceptions includes many properties not investigated by Merrit et al.¹¹ It is important to recognize that acoustic parameters may probably never fully explain listener ratings related to gender because of its complex nature and speakers' association with different genders is also socially constructed and therefore subject to change.⁴¹ 

While the goal of this study was to establish which acoustic features of the speech signal contribute the most to our perception of femininity and masculinity for use in guiding gender-affirming voice training, we suggest that the results can also be used in other domains. Speech and voice synthesis is a continuing evolving field, but when a particular level of femininity and masculinity is desired, past and recent efforts focus predominately on adjustment of the fundamental frequency of the voice source,^{13,15,42–45} of speaking amplitude,⁴² or of the simulated vocal tract length^13,15,45,46 to achieve the desired result. How we perceive a computer-generated voice affects how it may interact with us;^46–49 and we propose, in line with previous work,⁵⁰ that the incorporation of more of the acoustic markers of femininity and masculinity in the voice into speech synthesis systems can lead to more acceptable, desirable, or trustworthy communication partners. Using a broader spectrum of acoustic properties may alleviate the need to exaggerate the salience of a few acoustic markers, such as an exaggerated f_o or formant frequency spacing to clearly signal femininity. It may further be highlighted that more fine-tuned synthesis may reduce the risk of inadvertent encoding of unbalanced gender stereotypes in human–computer interactions.^51,52 Methods for incorporating acoustic properties identified here into speech synthesis is, however, beyond the scope of the present work and should be the focus of further research.

While the acoustic analysis presented here was based on the collected literature on the perception of femininity and masculinity internationally, we acknowledge that some aspects of the acoustic models may be language-specific or only generalizable to some grouping of languages close to spoken Swedish. More research is needed to elucidate this issue.

This study identified 223 acoustic properties that described 89% and 87% of variances in perceived levels of femininity and masculinity in voices, respectively. While the mean fundamental frequency of oscillation (f_o) is the most prominent cue to the perceived level of femininity and masculinity in voices, and the top six properties accounting for 87% of the total explained variance, this study confirmed that listeners' perceptions are described more accurately when complementary information from other acoustic cues is added to the model. The developed models are proposed to afford communication and evaluation of gender-affirming voice training goals and improve voice synthesis efforts.

See the supplementary material A for a description of parameters used for each speaker when extracting acoustic properties using Praat, supplementary material B for variable importance of the acoustic properties identified as important predictors of perceived level of femininity and masculinity, and supplementary material C for the level of pairwise (Pearson) correlations between acoustic predictors.

The authors received funding from internal grants given by the Department of Clinical Sciences, Umeå University, and partial funding for a Ph.D. education (to J.H.) from Umeå Center for Gender Studies for this study. The authors thank the participants for donating their time for the listening experiment and Speech-language pathology students and research assistants Emelie Granlund, Amanda Nyberg, Elenor Anderson Stål, Julia Häger, Felicia Gregory, Cecilia Mellberg, and Anna-Karin Sparre for their assistance in making speech recordings and performing the manual markup. We thank Svante Granqvist for suggesting a notation for harmonics near a formant of interest. The technical support of the Visible Speech platform developed as part of the Swedish national research infrastructure Språkbanken and Swe-Clarin, funded jointly by the Swedish Research Council (2018-2024, Contract 2017-00626) and the 10 participating partner institutions, is gratefully acknowledged.

The authors declare no conflict of interest.

This study was approved by the National Ethical Review Authority (Case No. 2019-05374). Informed consent was obtained from all subjects involved in the study before being audio recorded.

Under national law, speech recordings are considered personally identifiable information and cannot be distributed to other research groups or the public. Derived signals are made available upon request.