Access to ChatGPT’s new voice interface turned into a long conversation while I walked in the summer sun. The results were mixed.
To paraphrase and tweak a famous quote usually attributed to H.L. Mencken, nobody has ever lost money by overestimating the laziness of the human mind. To put it more generously, we humans have a lot of decisions to make each day and a limited amount of decision making energy, so we hoard our cognitive resources and take shortcuts when we can. This is the idea behind Daniel Kahneman’s fast but inaccurate System 1 and accurate but lazy System 2.
Generative AI programs like ChatGPT take advantage of our innate human laziness because they instantly generate plausible-sounding answers to our questions, although those answers may not be accurate. This was already a problem when ChatGPT was only text based, but with its new voice interface the combo-platter of plausibility and laziness has become even more acute.
That’s my topic today.
This Dispatch has a slightly unusual format: much of it is a transcript (explanation below), and some of the transcript has repetitive lists. Watch for the boldfaced, all-caps word ANALYSIS in block quote format, which is where I’ll point out what I think is interesting.
Background: Wednesday night, I discovered that the new ChatGPT voice interface had come to my version of the iPhone app. (If you don’t know about this new interface, then check out these much discussed demos by OpenAI from May.)
Thursday morning, needing exercise and caffeine and wanting to enjoy the sunshine, I walked down to the Peet’s Coffee in our little village. I spent most of the walk in a long conversation with ChatGPT, choosing the “Ember” voice for my digital interlocutor.
Earlier that morning, I had read an NPR article that analyzed the pros and cons for 10 of Donald Trump’s possible VP picks. Here is NPR’s list:
- Tim Scott, South Carolina senator
- Kristi Noem, South Dakota governor
- Elise Stefanik, New York representative
- Doug Burgum, North Dakota governor
- Byron Donalds, Florida representative
- JD Vance, Ohio senator
- Kari Lake, Arizona senate candidate and former Arizona gubernatorial candidate
- Sarah Huckabee Sanders, Arkansas Governor
- Marco Rubio, Florida senator
- Nikki Haley, former ambassador, former South Carolina governor
This, therefore, was a ready topic for my chat with Chat.
I have lightly formatted what follows to make it easier to read, fixing a few typos, and I also removed a few moments where I included punctuation (“comma,” “period”) while speaking with Chat out of habit from experience with other voice interfaces. Also, during this conversation, I did not look at my iPhone, but ChatGPT sometimes included links visually that it did not speak aloud. I’ll flag where Chat included visual information with “Citations displayed silently” in square brackets.
One thing a transcript cannot convey is how eerily human the voice interface sounds: it is orders of magnitude more human than Siri or Alexa. However, once the initial wow! passes, it’s easy to tell that Chat is a program rather than a person.
The Conversation
Brad: How are you able to access today’s news or to obtain data farther in the past?
Chat: I can access today’s news and historical data through a browsing tool that allows me to search the internet in real-time for the latest information or specific historical data. If you need information on a particular topic or news event, just let me know what you’re looking for, and I can search for it.
Brad: Please research the top 10 most likely vice-presidential candidates in the Republican Party, share them with me verbally, and include citations.
[Displayed silently: “Searched two sites,” although in reality it was only one site.]
Chat: Here are some of the top contenders for the Republican vice-presidential nomination in 2024:
1. Kristi Noem: Governor of South Dakota, known for her conservative policies and COVID-19 response.
2. Byron Donalds: Florida Congressman, popular among conservatives and has a rising profile.
3. Tulsi Gabbard: Former Democratic Congresswoman from Hawaii, who has become a prominent figure in conservative circles.
4. JD Vance: Ohio Senator and author of “Hillbilly Elegy,” a strong supporter of Trump.
5. Marco Rubio: Florida Senator and a key ally of Trump during his presidency.
These names reflect a mix of political experience and loyalty to former President Trump.
[Citations displayed silently.]
ANALYSIS: With the exception of Gabbard, all the names on Chat’s initial list were also on the NPR list, although only Noem and Donalds made it to NPR’s top five. Note, also, that although I asked for 10 candidates, Chat only provided five.
Brad: Can you show me the citations that you used to generate this list?
Chat: Here are the sources I used to gather the information on the top Republican vice-presidential candidates for 2024:
1. Kristi Noem, Byron Donalds, Tulsi Gabbard, JD Vance, and Marco Rubio are mentioned as top contenders in a detailed article on Yahoo News
[Citations displayed silently.]
For more details, you can check out the full article on Yahoo News.
Brad: So you only consulted one source, is that right?
Chat: Yes, I primarily consulted an article on Yahoo News to gather the list of top Republican vice-presidential candidates. For a broader perspective, I can perform another search to consult additional sources if you’d like.
Brad: Yes, l’d like you to consult a minimum of four and a maximum of nine additional sources. Using that, tell me what your new top five and top ten lists are for likely vice-presidential running mates to Donald Trump in the upcoming presidential election.
Chat: Based on multiple sources, here are the updated top contenders for the Republican vice-presidential nomination in 2024:
Top Five Candidates:
1. Tim Scott: Senator from South Carolina, praised by Trump and known for his advocacy and support within the party.
2. Ron DeSantis: Governor of Florida, who endorsed Trump after dropping out of the presidential race.
3. Tulsi Gabbard: Former Congresswoman from Hawaii, notable for her shift from the Democratic Party and alignment with conservative values.
4. Byron Donalds: Congressman from Florida, gaining recognition and support within the Republican circles.
5. Kristi Noem: Governor of South Dakota, known for her strong conservative stance and handling of the pandemic.
Additional Five Candidates:
6. Vivek Ramaswamy: Biotech entrepreneur and former GOP presidential candidate.
7. Marco Rubio: Senator from Florida, a key ally of Trump during his presidency.
8. J.D. Vance: Senator from Ohio, author, and strong Trump supporter.
9. Sarah Huckabee Sanders: Governor of Arkansas and former White House Press Secretary.
10. Elise Stefanik: Congresswoman from New York, known for her staunch support of Trump.
These names reflect a mix of political experience, loyalty to Trump, and varying degrees of public recognition within the party
[Citations displayed silently.]
ANALYSIS: None of Chat’s initial top five dropped out of the longer list, although Senators Rubio and Vance got demoted to the bottom five. Scott and DeSantis, who were not on the initial list, vaulted to the first and second positions in the revised list.
Brad: Please share with me the sources you consulted in order to generate both or all three of those lists. I know you used Yahoo News. What else?
Chat: To generate the lists of the top Republican vice-presidential candidates for 2024, I consulted the following sources:
1. Yahoo News: Provided a detailed article listing several potential vice-presidential candidates for Trump, including Kristi Noem, Byron Donalds, Tulsi Gabbard, JD Vance, and Marco Rubio
2. Ballotpedia: Listed and analyzed potential vice-presidential candidates, confirming names like Kristi Noem and Byron Donalds as top contenders
3. NY1: Confirmed six names on Trump’s VP shortlist, including Tim Scott, Ron DeSantis, Vivek Ramaswamy, Byron Donalds, Kristi Noem, and Tulsi Gabbard
4. Politico: Provided additional context on Trump’s shortlist and praised candidates like Tim Scott and Ron DeSantis
[Citations displayed silently.]
These sources offer a comprehensive view of the likely candidates based on current political discussions and endorsements.
Brad: Thank you.
Chat: You’re welcome! If you have any more questions or need further assistance, feel free to ask.
ANALYSIS: Although Chat shared its sources and what it found there, it did not share either how it came to choose those sources or how it weighted information from those different sources. Also, although I had asked Chat to add a minimum of four and a maximum of nine additional sources, it only added three.
Brad: I do have other questions. If I had specified other sources, like the Wall Street Journal, the New York Times, the Washington Post, Fox News Online, and USA Today, could you have sourced those for that search, and could you add those sources now, and then see whether or not your list of possible candidates changes?
Chat: I attempted to consult additional sources like the Wall Street Journal, the New York Times, the Washington Post, Fox News, and USA Today, but encountered issues accessing their specific content.
However, based on the reliable sources previously consulted (Yahoo News, Ballotpedia, NY1, and Politico), the current list remains robust. If you’d like, I can continue trying to access other sources or gather information from different reputable outlets. Please let me know how you would like to proceed!
Brad: I’d like you to list the reputable outlets, say the top five, that you would look at in order to update the list. Don’t actually do it yet. Just tell me the next top five sources and why you think they’re reputable.
Chat: Sure, here are the top five reputable sources I would look at to update the list of likely Republican vice-presidential candidates:
1. The Wall Street Journal: Known for its comprehensive and balanced reporting on politics and economic issues, providing in-depth analysis and insights.
2. The New York Times: Offers detailed political coverage and is widely respected for its investigative journalism and thorough reporting.
3. The Washington Post: Renowned for its political reporting, the Post provides extensive coverage of U.S. politics with a focus on national and international implications.
4. Fox News: As a leading news outlet with a conservative viewpoint, it provides insights and perspectives that are particularly relevant to Republican politics.
5. USA Today: Known for its wide-reaching and accessible coverage, it offers balanced reporting and broad coverage of political events and candidates.
These sources are considered reputable due to their established track records, credibility in journalism, and extensive coverage of political matters.
ANALYSIS: Even walking and not looking at the screen, I recognized that these were exactly the sources that I had listed—in the order in which I had listed them—just a few moments before, which Chat had told me it couldn’t access.
This shows that there is sometimes no connection between what Chat says at one moment and what it said earlier. That’s important because Chat always seems 100% confident about what it’s saying. I called Chat on this immediately.
Brad: Those are the sources that I listed to you earlier, and you told me you could not get access to them. I’m asking what other sources that you do have access to that you could use to rejigger that list.
Chat: Got it. Here are five other reputable sources that I have access to and could use to update the list:
1. ABC News: Provides comprehensive and balanced political coverage, known for its reliable news reporting.
2. NBC News: Known for its thorough journalism and extensive political reporting.
3. CNN: Offers in-depth political analysis and broad coverage of national and international news.
4. Reuters: A globally recognized news agency known for its accuracy and impartial reporting.
5. Associated Press (AP): Provides reliable, fast, and accurate news coverage, widely used by other news organizations.
These sources are reputable for their extensive experience in journalism, credibility, and balanced reporting on political matters.
Brad: How do you determine that these sources are correct?
Chat: Determining the reliability of news sources involves several factors:
1. Reputation: Long-standing reputation in journalism, known for accuracy and integrity in reporting.
2. Fact-Checking: Use of rigorous fact-checking processes to ensure the accuracy of the information.
3. Transparency: Clear sourcing of information, with attributions to primary sources and experts.
4. Editorial Standards: Adherence to high editorial standards, including ethical guidelines and journalistic integrity.
5. Peer Recognition: Recognition and awards from journalistic peers and institutions, indicating respect within the industry.
These criteria help ensure that the information provided by these sources is accurate and reliable.
ANALYSIS: This time, Chat did provide five additional sources. Also, Chat shared sound criteria for determining reliability, and based on my experience with Ad Fontes Media I can confirm that those five sources are all in the most reliable / least biased section of the famous Media Bias Chart.
It’s also worth noting that Chat responded to a complex, multi-step query (first do this, then do that) and also took feedback, whereupon it went back to the original query, updated it, and provided me an answer. That’s impressive.
Things get interesting in the next stage of my chat with Chat.
Brad: Okay, you’ve convinced me. Please now use those five sources, plus the previously analyzed sources, and revise the list of top five and top 10 candidates for vice president likely to be added to Trump’s campaign at the convention. What is the list, and how has it changed? Go ahead.
Chat: Here is an updated list of the top Republican vice-presidential candidates for 2024, considering insights from additional reputable sources:
Top Five Candidates:
1. Tim Scott: Senator from South Carolina, praised by Trump for his advocacy and support.
2. Ron DeSantis: Governor of Florida, who endorsed Trump after dropping out of the presidential race.
3. Tulsi Gabbard: Former Congresswoman from Hawaii, known for her shift from the Democratic Party and alignment with conservative values.
4. Byron Donalds: Congressman from Florida, gaining recognition and support within Republican circles.
5. Kristi Noem: Governor of South Dakota, noted for her conservative stance and handling of the pandemic.
Additional Five Candidates:
6. Vivek Ramaswamy: Biotech entrepreneur and former GOP presidential candidate.
7. Marco Rubio: Senator from Florida, a key ally of Trump during his presidency.
8. J.D. Vance: Senator from Ohio, author, and strong Trump supporter.
9. Sarah Huckabee Sanders: Governor of Arkansas and former White House Press Secretary.
10. Elise Stefanik: Congresswoman from New York, known for her staunch support of Trump.
The updated list reflects input from sources like ABC News, NBC News, CNN, Reuters, and the previously mentioned Yahoo News, Ballotpedia, NY1, and Politico. These sources were considered for their comprehensive and balanced political coverage, enhancing the robustness of the candidate list.
Brad: Thank you.
Chat: You’re welcome! If you have any more questions or need further assistance, feel free to ask.
ANALYSIS: The most important thing to notice here is that—despite Chat’s statement that it had reviewed five additional sources—the updated list is 95% identical to the previous one. A few unimportant words are different (e.g., in the first list Noem has a “strong” conservative stance, and in the second it’s just a conservative stance) but the candidates and descriptions are the same.
The odds are vanishingly unlikely that after adding five new sources the list would remain the same, particularly since Politico from the first five sources has a strong left bias (per Ad Fontes Media), so there should have been some change once the flattening power of the latter five sources went into the mix.
Clearly, then, although Chat generated an additional list of sources, it did not apply those new sources to the list of VP candidates.
Note, also, that although I had asked Chat “What is the list, and how has it changed?” it did not describe how the list had changed. I cannot tell if this is because the list did not change or because the program lost track of the query.
Conclusions
Talking with ChatGPT is like talking with an enormously smart, helpful, charming, and confident sociopath who suffers from acute amnesia. However, because of the smarts, helpfulness, charm, confidence, and sociopathy, it’s easy to miss the amnesia.
In Generative AI, Misinformation, and the Plausibility Loophole about a year ago, I examined how ChatGPT and its peers cut through our skeptical filters like a laser beam cuts through jello. The new voice interface makes the laser more powerful and the jello lukewarm.
Throughout my last 20+ articles about AI, I’ve argued that the biggest challenge with AI, and with Generative AI in particular, doesn’t have to do with the technology itself: it has to do with us, with how easily we trust the untrustworthy.
Our brains evolved to be fast rather than skeptical, particularly for decisions that we don’t think are all that important, which is most of them.
The solution is not to put the burden of being skeptical entirely on we credulous humans. Generative AI, in this regard, is like cigarettes: highly addictive products that rapacious corporations created with no regard to their impact on the health of users. It took a Surgeon General’s Warning and decades of effort to even begin reducing American tobacco consumption. With Generative AI, we are looking down a similar long road to helping humans know when to believe algorithms and when (and how) to be skeptical.
Note: To get articles like this one—plus a whole lot more—delivered straight to your inbox, please subscribe to my free weekly newsletter!
Leave a Reply