Here’s an interesting dive into what ChatGPT can and can’t do when it comes to planning a vacation. Have any of our Globe Aware volunteers used this new chatbot?

I asked ChatGPT to help me plan a vacation. Here’s what happened next

Monica Pitrelli
CNBC
FEB 26 2023

Some people love travel planning.

But I am not one of those people.

So the idea that artificial intelligence chatbots, such as ChatGPT and Bing, can research travel destinations and create itineraries is intriguing.

But I’m skeptical too.

Do recommendations just scratch the surface — for example, suggesting that I see the Eiffel Tower in Paris? Or can they recommend lesser-known restaurants and handle specific hotel requests too?

The answer is: yes and no — at least for ChatGPT.

Unfortunately, I couldn’t test Bing. When I tried to access it, I was put on a waiting list. The website said I could “get ahead in the line” if I set Microsoft defaults on my computer and scanned a QR code to install the Bing app. I did both. I’m still waiting.

ChatGPT was easier. I went to the developer’s website, clicked on the word “ChatGPT,” registered for an account — and started chatting.

‘Can you help me plan a beach trip?’

“Of course!” replied ChatGPT. But first, I needed to tell it about my interests, budget and how long I planned to be away.

I’m looking for a week-long beach trip in mid-March to spend time with my family, with no set budget, I typed.

“Sounds like a wonderful idea!” it replied, before recommending Hawaii, the Caribbean — specifically the Bahamas, Jamaica and the Dominican Republic — Florida and Costa Rica, along with details about the weather and popular attractions for each.

Nice. But I live in Singapore, I said.

“I see!” it exclaimed. (ChatGPT loves exclamation points.) In that case, Bali, Indonesia; Langkawi, Malaysia; and Phuket and Krabi in Thailand were better choices.

One more thing — I want great food too, I typed.

In that case, it said, the better bets are:

• Penang, Malaysia — for char kway teow, nasi lemak and laksa
• Ho Chi Minh City, Vietnam — for pho, banh mi and bun cha
• Osaka, Japan — for takoyaki, okonomiyaki, and kushikatsu
• Bangkok, Thailand — for pad Thai, green curry and mango sticky rice

Not bad. But I have another ask: I want to avoid crowds.

In that case, I may want to consider:

• Langkawi — “a beautiful island off the west coast of Malaysia … [with] plenty of secluded spots”
• Krabi — “not as busy as other destinations in Thailand such as Phuket or Koh Samui”
• Bali — “beaches in the north of Bali, such as Lovina Beach, tend to be less crowded”

All true.

But what about a treehouse in Bali — is that possible?

“Yes! …. Here are a few options,” it said:

• The Hideout Bali
• Bambu Indah
• Sandat Glamping Tents
• Free Spirit Spheres
• Which one has the best online reviews?

“All of the places I recommended have great online reviews,” ChatGPT replied, providing ratings from Tripadvisor, Booking.com and Google for each hotel.

Yet, the reviews missed the mark in several instances.

For example, it said Sandat Glamping Tents had a 4.9/5 rating on Google (actual: 4.5/5) and “over 400 excellent reviews” on Tripadvisor (actual: 277 reviews).

But perhaps the biggest blunder: Free Spirit Spheres is actually in Canada, which the bot acknowledged when pressed.

ChatGPT is nothing if not apologetic.

Cost estimates for each hotel were more accurate. But ChatGPT couldn’t show photographs of the hotels or help book them — although it did provide ample instructions on how to do both.

By road or by rail?

For a road trip across the United States, ChatGPT recommended Route 66, Pacific Coast Highway and the Great River Road, the latter of which traces the Mississippi River through 10 states.

What about train trips in Europe? “Certainly!” it said, recommending:

• The Orient Express — from Paris to Istanbul
• The Glacier Express — from St. Moritz to Zermatt, Switzerland
• The Bernina Express — from Chur, Switzerland, to Tirano, Italy
• The Golden Pass Line — from Lucerne to Montreux, Switzerland
• The Trans-Siberian Railway — “one of the world’s most famous train journeys” from Moscow to Vladivostok, Russia

One issue: The “Orient Express” is no longer in service. Does ChapGPT know this?

It does, but it seems to confuse every departure of the Venice Simplon-Orient-Express — which uses restored carriages from the original train service — with the once-annual Paris to Istanbul route, which will run in August this year, and has already sold out.

Flights

ChatGPT can name airlines that connect cities, but it can’t give current flight information or help book flights.

It wasn’t able to tell me the cheapest fare — or any fare — from London to New York this spring because it doesn’t “have access to real-time pricing information,” it said.

In fact, ChatGPT data ends at September 2021; it doesn’t “know” anything that’s happened since.

However, the bot could answer which month the London-to-New York route is usually the cheapest, which it said is “January and February, or during the shoulder season months of March and November.”

As for the best airline in the world, it said: “As an AI language model, I cannot have personal preferences or opinions.” But it went on to name the top five airlines named to Skytrax’s “World’s Top 100 Airlines” in 2021.

The list wasn’t correct. 

The list provided by ChatGPT appears to be Skytrax’s airline ranking from 2019 instead.

“Where should I eat?”

Asking ChatGPT “Where should I eat?” and “What are the best restaurants?” in a given city produces two different lists — one with more popular places to eat, and the other with those that have more industry awards.

For example, here is how it answered those two questions for Singapore:

Where should I eat?

• Hawker centers
• Din Tai Fung
• Jumbo Seafood
• Liao Fan Hong Kong Soya Sauce Chicken Rice & Noodle
• Odette
• Tim Ho Wan

What are the best restaurants?

• Burnt Ends
• Candlenut
• Corner House
• Hawker Chan
• Jaan
• Restaurant Andre

That’s a decent but dated list. Restaurant Andre, which was regarded as one of the city’s best in the 2010s, permanently closed in February 2018.

Every city I asked ChatGPT about resulted in the bot praising the local food scene. New York City is a “food lover’s paradise,” Florence is “known for its delicious food” and Melbourne is “renowned for its coffee culture.”

ChatGPT is exceptionally cheery, but do all cities get this treatment? To test this, I asked for food recommendations in Galveston, Texas. This time, ChatGPT just provided the list.

And if you’re wondering about the best coffee in Melbourne, ChatGPT says it’s at Proud Mary — a coffee shop that tops many “best of” lists today.

Specific questions

I had many more questions for ChatGPT, such as:

• “How should I spend five days in South Africa?”
• “Which chateaux accept visitors in Bordeaux?”
• “If I only have one day in London, what should I do?”
• “Which rides have the longest lines at Disney World?”

But before I could, my screen said “Access denied” alongside an “error code 1020″ message.

This error may be caused by overloaded servers or by exceeding the daily limit, according to the tech website Stealth Optional. Either way, all of my previous chats were inaccessible, a huge negative for travelers in the middle of the planning process.

A new window didn’t fix the problem, but opening one in “incognito mode” did. Once in, I clicked on “Upgrade to Plus,” which showed that the free plan is available when demand is low, but for $20 per month, the “Plus plan” gives access to ChatGPT all the time, faster responses and priority to use new features.

With access again, I quickly asked about wait times on Disney World rides, a subject which I had spoken to luxury travel advisor Jonathan Alder of Jonathan’s Travels about last week. Alder lives close to the park and has lost count of how many times he’s visited, he said. Yet, only one of their answers — Epcot’s “Frozen Ever After” — overlapped.

ChatGPT mentioned that FastPass and Genie+ can reduce wait times at Disney World, which is partly right. The company phased out its “skip the line” virtual queue FastPass program when it introduced Genie+ in the fall of 2021.

The takeaway

ChatGPT is fast, chatty and feels like you’re interacting with a human. I found myself responding with unnecessary pleasantries — “Ok, sure” and “Thank you” — out of habit.

I could see how it could save travelers’ time, especially if they are looking for an overview or are at the early stages of planning.

But information will need to be current, of course — and bugs and error messages, which I faced several times in addition to the “1020” message mentioned above — will need to be fixed.

OpenAI states that the current ChatGPT version “is a free research preview.” It also says the system may “occasionally generate incorrect or misleading information” and that it’s “not intended to give advice.”

When I asked it about its travel planning abilities, it said it “can assist with many aspects of travel planning” but that it may not be able to “provide personalized advice based on your unique circumstances.”

My verdict: Travel agents’ jobs are secure for the time being.