ChatGPt versus Google

[ChrisGreaves]

I specified ChatGPT because that's what I was playing with.
My understanding of AI, such as ChatGPT, is that the AI is based on scoured data, and in particular data scoured from Google's vast reservoirs of [mis-]information.
That being so, I would expect answers from a question posed to ChatGPT to be close to answers resulting from a Google Search, especially in areas which have a sound factual instance in Google.

I tried "what town is 62 kilometres from Toronto Ontario" and received "At a distance of approximately 62 kilometers from Toronto, Ontario, there are several towns and cities. One such town is Guelph. Guelph is located to the west of Toronto and is roughly around 62 kilometers away, depending on the specific starting and ending points within each city. Other towns and cities that are approximately 62 kilometers from Toronto include Hamilton to the southwest and Barrie to the north."

I had previously asked "what is the radius of a semi-circle 12,000 kilometres in area?" and received " ... So, the radius of the semi-circle with an area of 12,000 square kilometers is approximately 30.935 kilometers."

Question: Have (any of ) you tested any AI against Google? If so, were there any glaring errors? Any discrepancy from what you would expect from a Google search?
Thanks, Chris

[HansV]

Like all AI services, ChatGPT sometimes provides surprisingly relevant answers, sometimes it "hallucinates" and comes up with complete nonsense (unlike a Google search, where results are usually as good as the search terms you enter).
And it uses bizarre amounts of energy, so after a few experiments I stopped using it. I can use my time in better ways.

[Graeme]

ChatGPT is an impressive chatbot, but don’t rely on it as your astrophysics tutor.

https://www.astronomy.com/science/we-as ... o-so-well/

[stuck]

There was a news story recently about Google's AI:
https://www.theguardian.com/technology/ ... nded-users

Ken

[ChrisGreaves]

ChatGPT is an impressive chatbot, but don’t rely on it as your astrophysics tutor.

"To be fair to ChatGPT, it handles many basic questions impressively. Ask it something like “What is an astronomical unit?” or “What is the life cycle of a star?” — something you might easily find in a textbook or a Wikipedia article — and it can generate a perfectly reasonable response."

Hi Graeme. I think that this quote from the article sums up what i was trying to say, that anything (humans included) who draw their sole knowledge from a known source can be bypassed if we go directly to the source. (This is not to denigrate book-learning, but rather to suggest that a true education based on experience which was based on book-learning is a recommendation for book-learning. Or in ChatGPT's case, based on Google's memory)

"But once you start asking questions whose answers are not so obvious and dig into the subtleties of astrophysics — the types of informed, discerning questions that our readers regularly send us — ChatGPT becomes prone to giving answers that seem plausible, but are very wrong"

In short, it's still tough to beat answers that come from reasoning and experienced humans. Human sources for human questions might be our slogan.

Cheers, Chris

[ChrisGreaves]

And it uses bizarre amounts of energy, so after a few experiments I stopped using it. I can use my time in better ways.

Bizarre? You got that response from ChatGPT, right?

Our web is indeed a drain of our energy (and stifles exercising of our creative minds).
An example: YouTube and the web in general is full of articles on Vermicomposting, which usually include the warning "You can/must not put meat or dairy products in your vermicomposter?" (nor plastics, cat and dog feces and urine and so on), whereas all these CAN be put in a vermicomposter.
These cautions come from municipal web sites, because the lawyers have insisted on those clauses to shield the councils from law-suits from neighbours who claim that the rat infestation is due to vermicomposting.
Similar examples abound in most articles I have examined; another example is recipes for cakes that include half-teaspoons of a dozen different spices (a.k.a."look how many little jars I have on my shelf!")

My latest request to ChatGPT (please show on a map of southern ontario, an area the size of the ile de france) fails miserably, and ends with "If you have access to mapping tools, you could measure out an area of approximately 12,011 square kilometers in southern Ontario to get a visual representation of the size comparable to Île-de-France.". Of course, if I did have access to such tools I wouldn't be asking ChatGPT.

I note that while the response is a failure, it does reproduce information that I can use going forward (approximately 12,011), which is what triggered my initial thoughts, that ChatGPT is more of a filter on the database owned by Google.

Cheers, Chris

[ChrisGreaves]

There was a news story recently about Google's AI:

Thanks Ken. Yes, news stories (and research papers) abound.
I think that we are at the same point in history (of AI) as we were with automobiles almost 200 years ago "In 1835, Professor Sibrandus Stratingh of Groningen, the Netherlands and his assistant Christopher Becker created a small-scale electrical car, powered by non-rechargeable primary cells."

I anticipate progress in AI will be fast, if only because we have computing power to speed up R&D today.
Cheers, Chris

[ChrisGreaves]

I specified ChatGPT because that's what I was playing with.

How ChatGPT shoots itself in one of its feet; or inches. Gerry Hussey has lent me a whelk barrel. What is its volume?

I asked ChatGPT "a barrel is 31 inches high and 20 inches diameter. What is its volume in gallons?"
ChatGPT came back with "So, the volume of the barrel is approximately 132.47 gallons. ChatGPT can make mistakes. Consider checking important information."

I thought that since this is the sort of answer I might obtain from Google I (see attached TXT file). While Google merely refers me to an online calculator ChatGPT was smart enough to have its own calculator.

I had two tricks up my sleeve:-
(1) I have a copy of Excel2003 (attached workbook) and
(2) When I was eleven years old I had the job of moving empty 44-gallon oil drums around the yard of a local garage. (I suspect that the owner just gave me something to do that wouldn't harm the cars in the shop). So I have two ideas of what the barrel should hold: 44 gallons, rather than 4.4 gallons or 440 gallons.

So why does ChatGPT come up with 132.47 gallons? Out by a factor of about three.

Look closely at the text file; ChatGPT has used the symbol ╥ to represent Pi, about 22/7. In the text file this shows up looking like a lower-case "n".
It seems that somewhere between using the PI symbol and effecting the calculation, Chat GPT ignores the PI symbol. Hence (last row of the spreadsheet), ChatGPT's answer is out by a factor of 3.14159

Cheers, Chris

[HansV]

Does that mean that ChatGPT has proved that π = 1? Now that would be a revolution in mathematics.

(When I was a PhD student in mathematics, an elderly man came to the Mathematics Institute with a 50 page manuscript aiming to prove that π = 3.
My thesis adviser asked me to review the manuscript. Most of it was more or less OK, except for one crucial error on page 2: he claimed that it was obvious that the circumference of a circle was equal to the circumference of its inscribed regular hexagon. Everything derived from that...)

S2529.png

[ChrisGreaves]

(When I was a PhD student in mathematics, an elderly man came to the Mathematics Institute with a 50 page manuscript aiming to prove that π = 3.

https://www.reddit.com/r/MathJokes/comments/16rfzp/a_limerick_of_pi/
Cheers, Chris

[ChrisGreaves]

... It seems that somewhere between using the PI symbol and effecting the calculation, Chat GPT ignores the PI symbol. ...

So I am somewhat delighted to believe that ChatGPT is an assembly of modules. In this example I suspect that one (set of) modules does the new-fangled text analysis stuff and requests Google to return text clues, another module sieves that text and discovers it needs a formula to calculate, another module sends off an expression to be calculated, but the final module is looking only for (decimal?) numeric strings, and ignores any symbols - such as PI - that are not decimal digits.

ChatGPT might be better if it sought out a numeric answer by two separate routes. In maths class we had to work out an approximate answer in our head and then use paper and pencil (and trig/log tables) to calculate the three significant digits. If the two results differed wildly, then at least one of them was wrong, but if they agreed, there was a good chance that we were on target, and could move on to the next problem.
Cheers, Chris