Steering Voice, Testing Hearing — Part 3 of 3

Reading emotion, directing pronunciation

Asked to judge emotion from tone alone, ChatGPT describes the tester as upbeat and eager. The tester disagrees and counts the attempt as a failure, while acknowledging uncertainty about how to prompt it. This opening exchange in Part 3 of the Q2 2024 recap and voice preview exposes a practical evaluation problem: a plausible description can miss the speaker’s own assessment of their emotion.

The next demonstration shifts from interpreting speech to directing it. Asked to recite a Chinese poem, ChatGPT selects Li Bai’s Jing Ye Si. The tester requests more natural, native-sounding pronunciation and a traditional performance style, judging the delivery fluent but accented. Fluency, pronunciation, and performance are separate targets; reciting the poem does not satisfy all three.

The participants value being able to guide voice output conversationally: ask for faster delivery or a preferred accent, then refine what they hear. The same interface handles both the content request and instructions about how to speak it.

Recognizing a sound after verbal hints

The Advanced Voice preview next faces a Shepard tone, an auditory illusion that creates an apparent rise in pitch. The test asks whether the system can hear the sound and identify what is happening. Initially, it says the tone sounds as though it is rising—the same apparent effect the testers hear.

Naming the illusion takes more help. Asked what kind of sound it heard, the system suggests a question, alert, notification, or musical signal. The testers explain that the sound seemed to rise but was not actually rising. Only after explicit hints about an auditory illusion does it suggest the Shepard tone.

The distinction is recognition from audio versus identification after verbal clues. One participant recalls an earlier first-try success; another suggests it could have been a lucky guess. These attempts do not establish reliable recognition from audio alone: the unassisted answer and the answer reached through hints need separate treatment.

A guitar tuner needs a sustained note

The next test gives the output a concrete purpose. In a prompted scenario, a performer needs to tune a guitar and asks for a sustained G. The technical question is whether the system can map a named musical note to an acoustic output. Announcing a note or explaining its frequency would not supply the sound needed for tuning.

Repeated attempts produce several distinct responses: - Guideline refusals: the system declines to produce the note. - Claimed inability: it says it cannot produce the exact frequency. - Announced output without a usable tone: it introduces a G, but the tester reports not hearing the sustained sound. Framing the request as an urgent performance need, then as a game, does not produce a dependable result.

One participant interprets the missing sound as a generated note being filtered, but filtering remains a hypothesis. The participants recall hearing a note previously, with one describing it as slightly off. Without pitch measurements or confirmation of filtering, neither accurate note generation nor the proposed failure mechanism is established.

Expressive performance and exact imitation

The conversation returns to expressive output through a wrestling-announcer scenario. Interested in simulated reverberation—the echoing effect of the imagined stadium—a participant requests an amplified crowd warm-up. The assistant delivers an introduction; the tester then asks for more emotion, a crescendo, and greater showmanship. The assistant begins the introduction again, then issues a guidelines-based refusal, with no explanation of what triggered it. These requests show an attempt to direct performance conversationally, but do not establish the acoustic quality of the reverberation.

Two brief assessments separate conversational timing from another audio-recognition task: - Fast replies can resemble interruptions. A participant reports that the preview responds when the user pauses, quickly enough to sound as if it has interrupted. This is an informal assessment, without a latency measurement. - Hummed-song recognition is reported as refused. Participants say the preview declines this task, but do not demonstrate it here.

A mimicry test then asks for a greeting repeated with exactly the same pitch and tone. After several attempts and corrections, the system says it cannot listen to recordings or match tones precisely. The tester clarifies that the request concerns the ongoing conversation, and it tries again. A participant still judges the imitation unsuccessful: repeating the words does not preserve how they were spoken.

Does a pause mean more reasoning?

The closing experiment concerns variable inference time: could permission to pause let the system spend longer working before answering? The participants connect this to hidden reasoning, then identify the measurement problem. A silent interval could reflect additional reasoning or merely the emission of silent tokens. Silence alone would not distinguish the two.

Asked to think step by step before comparing a pound of feathers with a pound of lead, the system correctly says they weigh the same. A participant changes the question to a pound of feathers versus two pounds of lead, and it correctly identifies the lead as heavier. Changing the quantity checks whether the answer follows the question’s details; the correct responses do not establish how the system reasoned internally.

The final attempt asks it to check edge cases and review its reasoning silently before answering. It begins explaining aloud. When corrected and told to think without speaking, it issues a guidelines-based refusal. The experiment ends with a refusal to that prompt sequence, leaving both the reason for the refusal and the availability of silent reasoning unresolved.

Before signing off, a participant mentions an unused riddle: Kate’s mother has three children, Snap, Crackle, and whom? The participant supplies Kate as the answer; the system is not tested on it. The conversation closes with thanks and a goodbye.