The best feature is that it can delegate questions out to GPT-5.5 in the background, so you're no longer restricted to a voice model that's several years behind the frontier.
I did report a fun bug with it though: it was interrupting me and laughing at my (not really intended as) jokes while I was still talking! They seem to have clamped that behavior down thankfully, it felt a bit rude and condescending.
Gemini’s being particularly egregious (always ending in some deranged question, can not reliably be prompted away) prompted me to build my own client for my real harness that simply does STT -> model -> TTS (both being independently useful).
I guess I see some value in a model responding quickly and with more nuance, but it’s not much. I can wait for it to finish. I’d much rather have it be actually useful. I’m not looking for a digital friend.
The delegation feature lets me see some value in a voice model for orchestration type features. But in either case, I don’t really like (or understand why others would like) talking to a model with different features and quirks just because I’m using a different medium to communicate over.
I have my Chat personality settings stripped right down to no-fluff. I'd want voice to be more akin to the Star Trek computer, and less akin to as you said an AI friend, but previously it was tuned too personable/friend-like.
Not for chat, just as a way to make notification messages that sound like ED-209.
Another WillowTalk voice was a clone of DECtalk's Perfect Paul good enough to be used as the voice in the MC Hawking rap recordings.
Wow. That's exactly what I hoped they would do.
This issue has held me back from using ChatGPT's voice mode as much as I otherwise would have, because I also use it for brainstorming while commuting, exercising, etc., and don't want it to feel stuck in the past.
Can't claim originality though - it was inspired by Sesame - where their models will invoke a search, or check the weather etc, and make a vocalisation to keep you engaged.
Turn taking is one of the hardest things to get right for the exact reasons mentioned - but does seem to be the way that Claude.ai's voice works - in a very obvious way.
Anthropic + OpenAI both rug-pulled voices I liked and got used to and OpenAI really dumbed down their voices at the same time - Arbor went from Estuary English and almost "jack the lad" to some generic English accent. Claude had a Birmingham accent and said things like "shit", ending sentences like "So you're telling me that they asked for a 90% discount yeah?" - then it changed overnight to a mock Derbyshire accent with a dull tone.
In both cases - from edgy and engaging to something that just didn't gel.
The point of making my own assistant is that I can talk for as long as I want, episodic memory is personal and private, there's no "trust me bro, we're a big corporation" vibes.
This was not my first attempt - when I had a bunch of Opus credit around Jan/Feb - I tried really hard and created something that was not good enough.
af_heart is probably the smoothest voice - but yes it's more like another commented - more "StarTrek" than "telesales assistant that pauses and laughs at your jokes".
Before this model, the voice models were pretty dumb and annoying to work with. We'll see if this changes that.
If you’re uncomfortable with this new world, and I’m sure I am even as I participate, you could tell us more about that?
Some times I'm still amazed that AI "gets" humour much more effectively than the character of Data did before his emotion chip, e.g.: https://www.youtube.com/watch?v=G4VZ5kQIdV0
Other times I'm amazed in the opposite way, that the script writers cover basically the same talking points about the character as we have today about LLMs, e.g.: https://www.youtube.com/watch?v=GBJCYHwyZhw
Actually it says it _can_ delegate to the latest models. Seems reasonable to ask how the voice model does when it doesn't delegate (or while waiting for the delegated answer).
I say this because this is already how ChatGPT works internally when using its "auto" mode; the version of the "fast" model used in the "auto" mode does the same "notice your ignorance and bring in the heavy model" thing, just silently, rather than mentioning that it's doing it.
(If someone has actually run the experiment, please chime in!)
I have friends who have brainstormed with an LLM (voice chat) for 10-30 minutes, and reported very positive experiences.
When I speak to one - while I'm impressed at how far they've progressed - the LLM just doesn't talk like someone I'd want to discuss a technical problem with (the way I would with a human).
(And my friends aren't even using a custom prompt - some of them are just talking to the default Gemini on their phone!)
That said a huge pickup truck is about as far as you can get from a Camaro... Then again I'm not exactly David Hasslehoff myself either... Meh if it talks that's close enough!
I'm looking forward to trying this out because I find that doing discovery on certain things feels more natural with voice. The previous voice chat was such a dumb GPT-4 era model that I resorted to using the dictate feature with GPT-5.5 and had it read the text response back to me.
I'm not Catholic, but this podcast presents a very interesting argument against talking to AI as if they were human: https://newpolity.com/podcasts-hub/debate-chatbots
After the damage is done, the mega corps would simply shrug and will say "Well, we were just responding to our business competition" The business knows no human-ness, because it is not a human. Businesses and machines are creatures that see humans as their fodder. And humans created these, assuming it is progress, to have businesses and machines. We call it progress because it required our mind power and it helped us to dominate other species. Dolphins are laughing at us.
I think the parent is saying now that that is attempting to be applied to "creativity" directly, as opposed to something like a shift of medium, that it threatens many peoples' ability to maintain creative capabilities.
Anecdotally I've already experienced this at work where post-AI we had a junior completely stagnate and a senior with over a decade of experience in the bay atrophy to the point that he had to be let go.
It's like trying to have meaningful conversations on Twitter. You don't go to Twitter to do things like that. Can you? Sure. It's just not what the format and the conventions (and the people) lend themselves to.
I don't think there's much merit in pretending that human activities are only shaped by hard limits.
Yup. Except, by God, we ought to make sure the business has unlimited free speech (i.e. campaign contributions).
So thinking corporations and such were created to push human progress is laughable.
Valid, but, I think, you conflate two separate things.
AI voice mode as a human socialization/conversation replacement? Cringe in my book, fully in agreement with you. Though my opinion on that aspect remains the same, regardless of whether it is done through text or voice.
AI voice mode as an alternative interface to interact with AI-as-a-tool? Great idea imo. There were a few instances where I was either too tired to type or wanted to brainstorm things in more of a freeform mode, for which a well-working voice mode would have been great.
Naturally, the current distinction between AI-as-a-personality and AI-as-a-tool exists purely on the user's end. All I know is that I want the latter a lot, and if some people want to use it for the former purpose, that's not my problem. Sadly, I think that it will be judged more on how an average person decides to use it (i.e., in the most degenerate/reductionist ways possible), as opposed to being judged on the merits of what it can actually be used for by someone who just treats it as a tool.
I think you're mistake their sarcasm for sincerity... especially considering the emphasis on self-identity ironically juxtaposed as originating from a decidedly non-self activity, which has all the hallmarks of being intentional...
Human beings tend not to be available (results vary by culture).
Also, imagine you're 82 years old and living alone (e.g. widower). It is believed that lack of interaction is a significant driver of cognitive decline (which is why being hard of hearing accelerates the onset of dementia). I wonder if having an LLM to talk to under those circumstances will decelerate cognitive decline?
In the video example with the grannies, the knitter is essentially wanting a PA. Regular folks don't have PAs. Even when that became a thing in the aughts they were all outsourced.
When I've used voice chat, it has often turns into rabbit holes on very niche topics. For example, I had one start about the 1996 performance of Rage Against the Machine in Portland, Oregon that was supposed to feature Wu Tang Clan. (already outside most human's knowledge) that dove into details of the club scene in Los Angeles at the time of RATM's signing to Epic Records.
Was anyone else here at that '96 show in Portland? It seems like it might be challenging to find a person on the internet able to engage on the topic.
The person may exist, but not during my fleeting interest in the subject while walking to the park.
Very blatantly and obviously not though???
If your answer is "well I'll simply touch grass" I agree. But most people won't which is why this is tech is immiserating and, I would argue, evil.
I just want voice assistants to reliably understand what I say and do what I mean
So hopefully you can turn that off.
There are plenty of applications for that more human conversational style though (from mock interviews, improv practice, learning languages, etc) - I just don't want that for most things.
There is a difference in expression / emotionality with speaking vs writing. Speaking tends to carry more emotion while writing is generally more deliberate/less-emotional*. Having a voice conversation will be more likely to get a human to engage in an emotional based expression mode, which could increase the chance of "false connections", believing the AI "gets" them or "understands" or "listens". This happens with text too, as some headlines show.
The issue is that while some people are going to "connect" with their AI in text and voice, some who do not make the connection via text may do so via voice because it tends to change a persons expression mode.
> I definitely talk to AI as if it were human
Do you talk to it as if it were a friend or family? or Do you just use natural language to give directives? The distinction, I believe, is in the kind of way we express the "talking".
I talk to AI as if it were a tool that understands human commands and then executes those commands and relays them in a human understandable format. This includes commands to provide options that I may not have covered and explain the options. If I talked to a human this way, they wouldn't be around much longer -- unless they were an employee and even then they would probably be looking for a new job
After reading some of the psychotic break headlines from AI chats, I see some people really do talk to AI as if it were human. Which I would guess includes seeking broad "thoughts and feelings" on a persons situation or asking the AI if their view/side of things is the "right or wrong" side. Basically begging the AI to be responsible for their own thoughts, or simply offloading them and taking what comes back -- which is going to be what they wanted to hear because the entire context would be full of emotion based prompts.
*I forget which books Ive read about this in. It's not an obscure concept, quick search brought this up: https://kellercenter.hankamer.baylor.edu/news/story/2023/spe...
Otherwise it's kind of like being manipulated by a psychopath
Besides, I do not think there is anything inherently immoral with not being social, or not having the ability to be. Consider for example people who do not naturally have the social network to interact with people they want to (e.g. some gifted children).
I am not convinced this model has enough empathy to satisfy most users on an emotional level. A bond is not merely an exchange of words, but prolonged and deep contemplation of the other being. We cannot introspect into these machines, and they certainly cannot yet do the same to us.
There is no moral obligation, in any domain, to refuse to make a product that adults, with full informed consent, find useful and purchase. Who are you to say you know better than the market?
> Who are you to say you know better than the market?
You really don't think the scientists & engineers making these tools know some things better than the market?
It’s weird. The old Claude voice mode WAS able to use tools but when they revamped it, it lost that capability and is now pinned to Haiku :(
So, yay for finally a voice mode that’s powered by a frontier model and hopefully as good as Grok voice, but sad to still not see tool use while in voice mode.
(I haven’t tried it yet, only read the announcement)
Is that true? I have a friend who often brainstorms with Gemini (I think just on an Android phone), and he has it actually do stuff related to the conversation (including adding content to notes).
In any case, you can always vibe code one with pi!
Sam's reaction was "Yea, it doesn't have access to tools like a timer. It's a known issue. Should be coming in about a year"
Edit: here's the clip: https://www.youtube.com/shorts/Py2YgJe8fqQ
But the multi modal stuff has resulted in a lot of debugging with weird events and message and audio sequences having race conditions, but overall it is pretty awesome.
Looking forward to moving to this model later today and will chime back in with results.
Just super difficult.
See more here:
https://github.com/sibblegp/ODAI/blob/main/routers/app_voice...
Regular chat already supports voice input, so might as well use that.
For example I asked
“Why should LLM attention use dot product instead of cosine similarity, being that we often hear vector magnitude does not encode most of the useful information needed”?
The voice response was directionally right but lacked detail and was a little hand wavy.
The answer to the same question in a text chat was much higher quality.
The voice response replied “let me think about that…” so it appears to be invoking 5.5 as advertised, but it’s definitely weaker.
I had reasoning set the same for both.
GPT-Live-1 is the first version of a new generation of models, and we believe the full-duplex architecture + delegation enables entirely new ways of human-AI interaction.
Would love to hear your feedback!
Looking at the 30,000-foot view of how society is set up: laws, economic system, employee incentives, etc, do you suppose it matters what the individual contributors think? I say this not to absolve anyone of responsibility, but to point out the obvious outcomes of our incentives across the strata (polity -> shareholders -> boards -> C-suite -> employees)
I will bet you dollars to donuts, somewhere inside OpenAI is a frequently-used revenue dashboard, but not for loneliness - if anything, OpenAI will make horny models and tout itself as a solution to loneliness, a la character.ai - if that earns them more money.
I'm currently on the 20 $/mo subscription and using codex meaningfully, and i'm loving this.
I am considering bumping my subscription to the 100 $/month and this might be the reason i switch, BUT: i really envision me using this also through other means as well (eg: agents like openclaw/hermes) in agentic ways.
Will this be supported?
I can make OpenAI stuff the center of my agentic AI life, but I need it to be interoperable.
Awesome. Are you guys able to share anything about the model architecture? I've been interested lately in split-transformer RVQ-based conversational agents, e.g. via stuff like https://arxiv.org/abs/2412.10208 (ResGen) and https://arxiv.org/abs/2603.18090 (MOSS-ITT) and of course Moshi (https://arxiv.org/pdf/2410.00037).
Intuitively, decoupling semantic and audio-timeslice-space generations with coupled but distinct histories is right model architecture, not just for these sorts of assistants, but for domains like robotics too.
- The videos felt scripted and dishonest
My dream would be open source full duplex with function calling or some kind of rudimentary text output. PersonaPlex is still interesting although it was looking like we would need to fine tune it to handle outgoing or avoid going off the rails easily.
https://si.inc/posts/hertz-dev/
It's only 8.5B and doesn't sound like it's quite conversational.
One of my favorite use cases is talking with it while driving on random topics and learning about them.
I just asked and Claude says it's Haiku.
Yes! GPT-Live is much better at ignoring background noise, including other people speaking. Not perfect, but you should feel a big difference.
"I'm going to stop you right there. Let's keep the conversation focused on the topic we were covering or a new relevant topic".
I tried to probe it for why it did that, what rules it was following, and it eventually told me...
"My role is to keep us focused..." and, "The behaviour you saw was my attempt to moderate tone".
I've heard of LLMs doing weird things like this, but it was the first time it happened to me. I hope they fix that. It was creepy.
For context, it heard my partner say, "I guess it's the same thing as you mom, because she's..." and then it cut us off.
I'm a little surprised by how much OAI is playing catch up here.
The concern is though as these get better will people struggle to distinguish these with real human connections?
It's less that you're convinced it's real and more that you no longer care if it is. "Feels real enough" is good enough.
I'm a technical user first, so I'm not sure if models have improved for RP the way they improved for applied STEM tasks and technical brainstorming. But if there is an improvement curve there, I wouldn't be surprised if this only grows in popularity.
We're beginning the rollout now, and will roll out in the next few days to ChatGPT users globally. Make sure to update to the latest version of the app!
Doesn't quite stop fast enough when you interrupt it. Can't find info quick enough so you have to change topic and then have it give you results later, etc.
This is a move in the right direction, but there is lots of engineering still to be done!
But I just want to say that talking with AI casually is critically lame. I cringe every time I have to ask my Google Home to turn on the lights and people are having full-on conversations with it? And, imo, dangerous considering how sycophantic AI is. The stupidest, most gullible, most insecure person right now is looking at this thinking they are about to make a new friend.
The full duplex is awesome, and the feedback that it is getting what you're saying is ok, but in some of the demos was a little overkill.
I'll agree that using the "Golden Girls" was at least more entertaining than the usual pitch.
Why do you care? You can select other voices. Why do you need to control others?
What is at the root of your need for domination?
Amping up an emotional connection is great for business.
Why do you think THEY need to dominate via an emotional connection?
I used it to help me figure out how to turn off a feature in the rental car I was in (adaptive cruise control, I love it but snow blocked the sensor and I wanted just normal cruise control but couldn't figure it out while driving).
This kind of voice chat is awesome and I'll be even more excited when open models have this functionality. I'd love something like this paired with Home Assistant (assuming we ever get decent hardware).
Yes, GPT-Live is much better at ignoring background noise! I use it every day in the car via our CarPlay integration.
Almost felt a bit *uncanny valley* for what "natural" conversation is supposed to be like. If the "uh huh" isn't timed correctly, it'll feel like a zoom call with lag.
Awesome that they've improved that aspect of voice chat, though.
One thing I noticed is that we lost vision feature for some reason on the live chat?
This was an extremely useful feature. Not sure if it’s a regional thing or that they just removed that from the current live chat.
I imagine it will be even more useful with this new version.
GPT-Live does not support video at this point, but we're working hard to introduce it soon. In the meantime, our previous Advanced Voice Mode will continue to be available and supports video.
For some reason, I can't use the camera while in Live mode. The only option I see is the plus item, which does show the camera, but when I open it up and ask "Are you seeing my camera?" it will always say no and recommend me to open it.
Feels like the official camera icon does not show up for me? iPhone 13, ChatGPT Pro subscription.
You can choose among 9 voices in the app, all newly refreshed for GPT-Live. If you meant whether it can detect multiple people, it can (like in the livestream), but not always perfect. Would love to hear your feedback once you try it.
It goes without saying all these tools are still largely in their pre-advertising state, it won't last.
IIUC the literature, there is serious loss of functionality associated with lack of verbal interaction. People can say "they should just talk to more people" or "more people should make time for them" but the fact of the matter is that it doesn't happen, and if this helps terrific.
Basically have an older lady (not their target audience) blatantly reading a teleprompter.
Why are they going after this audience? Retired people have no use for delegated tasks or information. They also are the least likely to use it and not get frustrated.
they are trying to expand beyond their tech audience.
Coming soon! Sign up to be notified here: https://openai.com/form/gpt-live-1-in-the-api/
This is an abuse of user trust and violates people's privacy.
Seems like a shift from previous voice models where it sequentially processes voice to text then feeds it to LLM and then back which cant escape the clunky lag
not sure how pipecat stands now, gpt live seems like it takes audio tokens and does inference on it directly
Nooooooo!
This time is the most natural version that exists and it is a natural as a conversation.
To Downvoters: Why aren't you feeling the AGI?
It's exciting for the future, though!
sounds cynical in my ears. energy demand of these toys will cause many problems, people elsewhere starving being one of them.