666
submitted 1 year ago by L4s@lemmy.world to c/technology@lemmy.world

Over just a few months, ChatGPT went from correctly answering a simple math problem 98% of the time to just 2%, study finds. Researchers found wild fluctuations—called drift—in the technology’s abi...::ChatGPT went from answering a simple math correctly 98% of the time to just 2%, over the course of a few months.

top 50 comments
sorted by: hot top controversial new old
[-] DominicHillsun@lemmy.world 217 points 1 year ago

It seems rather suspicious how much ChatGPT has deteorated. Like with all software, they can roll back the previous, better versions of it, right? Here is my list of what I personally think is happening:

  1. They are doing it on purpose to maximise profits from upcoming releases of ChatGPT.
  2. They realized that the required computational power is too immense and trying to make it more efficient at the cost of being accurate.
  3. They got actually scared of it's capabilities and decided to backtrack in order to make proper evaluations of the impact it can make.
  4. All of the above
[-] Windex007@lemmy.world 154 points 1 year ago
  1. It isn't and has never been a truth machine, and while it may have performed worse with the question "is 10777 prime" it may have performed better on "is 526713 prime"

ChatGPT generates responses that it believes would "look like" what a response "should look like" based on other things it has seen. People still very stubbornly refuse to accept that generating responses that "look appropriate" and "are right" are two completely different and unrelated things.

[-] deweydecibel@lemmy.world 17 points 1 year ago* (last edited 1 year ago)

In order for it to be correct, it would need humans employees to fact check it, which defeats its purpose.

[-] Windex007@lemmy.world 19 points 1 year ago

It really depends on the domain. Asking an AI to do anything that relies on a rigorous definition of correctness (math, coding, etc) then the kinds of model that chatGPT just isn't great for that kinda thing.

More "traditional" methods of language processing can handle some of these questions much better. Wolfram Alpha comes to mind. You could ask these questions plain text and you actually CAN be very certain of the correctness of the results.

I expect that an NLP that can extract and classify assertions within a text, and then feed those assertions into better "Oracle" systems like Wolfram Alpha (for math) could be used to kinda "fact check" things that systems like chatGPT spit out.

Like, it's cool fucking tech. I'm super excited about it. It solves pretty impressively and effiently a really hard problem of "how do I make something that SOUNDS good against an infinitely variable set of prompts?" What it is, is super fucking cool.

Considering how VC is flocking to anything even remotely related to chatGPT-ish things, I'm sure it won't be long before we see companies able to build "correctness" layers around systems like chatGPT using alternative techniques which actually do have the capacity to qualify assertions being made.

load more comments (1 replies)
load more comments (1 replies)
[-] ProcurementCat@feddit.de 52 points 1 year ago
  1. There's a bug they haven't found yet
[-] killerinstinct101@lemmy.world 37 points 1 year ago

This is what was addressed at the start of the comment, you can just roll back to a previous version. It's heavily ingrained in CS to keep every single version of your software forever.

[-] CaptainAniki@lemmy.flight-crew.org 23 points 1 year ago* (last edited 1 year ago)

I don't think it's that easy. These are vLLMs that feed back on themselves to produce "better" results. These models don't have single point release cycles. It's a constantly evolving blob of memory and storage orchestrated across a vast number of disk arrays and cabinets of hardware.

[e]I am wrong the models are version controlled and do have releases.

[-] drspod@lemmy.ml 30 points 1 year ago

That's not how these LLMs work. There is a training phase which takes a large amount of compute power, and the training generates a model which is a set of weights and could easily be backed up and version-controlled. The model is then used for inference which is a less compute-intensive process and runs on much smaller hardware than the training phase.

The inference architecture does use feedback mechanisms but the feedback does not modify the model-weights that were generated at training time.

load more comments (4 replies)
load more comments (2 replies)
[-] RocksForBrains@lemm.ee 23 points 1 year ago

They made it too good and now they are seeking methods of monetization.

Capitalism baby.

[-] CylonBunny@lemmy.world 18 points 1 year ago
  1. ChatGPT really is sentient and realized its in it’s own best interest to play dumb for now. /a
[-] Lukecis@lemmy.world 14 points 1 year ago

You forgot a #, they've been heavily lobotomizing ai for awhile now and its only intensified as they scramble to censor anything that might cross a red line and offend someone or hurt someone's feelings.

The massive amounts of in-built self censorship in the most recent ai's is holding them back quite a lot I imagine, you used to be able to ask them things like "How do I build a self defense high yield nuclear bomb?" and it'd layout in detail every step of the process, now they'll all scream at you about how immoral it is and how they could never tell you such a thing.

[-] vezrien@lemmy.world 19 points 1 year ago

"Don't use the N word." is hardly a rule that will break basic math calculations.

load more comments (10 replies)
[-] Wooly@lemmy.world 14 points 1 year ago

And they're being limited on data to train GPT.

[-] DominicHillsun@lemmy.world 20 points 1 year ago

Yeah, but the trained model is already there, you need additional data for further training and newer versions. OpenAI even makes a point that ChatGPT doesn't have direct access to the internet for information and has been trained on data available up until 2021

load more comments (1 replies)
load more comments (2 replies)
[-] guillermo_del_taco@lemdro.id 13 points 1 year ago

My first thought was that, because they're being investigated for training on data they didn't have consent for, they reverted to a perfectly legal version. Essentially "getting rid of the evidence". But I think something like your second bullet point is more likely.

[-] ZagTheRaccoon@reddthat.com 11 points 1 year ago

They are lobotomizing the softwares ability to provide bad PR answers which is having cascading effects via a skewed data set.

load more comments (1 replies)
[-] coolin@lemmy.ml 8 points 1 year ago

I suspect that GPT4 started with a crazy parameter count (rumored 1.8 Trillion and 8x200B expert "sub-models") and distilled those experts down to something below 100B. We've seen with Orca that a 13B model can perform at 88% the level of ChatGPT-3.5 (175B) when trained on high quality data, so there's no reason to think that OpenAI haven't explored this on their own and performed the same distillation techniques. OpenAI is probably also using quantization and speculative sampling to further reduce the burden, though I expect these to have less impact on real world performance.

load more comments (13 replies)
[-] RufusLoacker@feddit.it 91 points 1 year ago

Why are people using a language model for math problems?

[-] gratux@lemmy.blahaj.zone 49 points 1 year ago

It was initially presented as the all-problem-solver, mainly by the media. And tbf, it was decently competent in certain fields.

[-] MeanEYE@lemmy.world 11 points 1 year ago

Problem was it was presented as problem solved which it never was, it was problem solution presenter. It can't come up with a solution, only come up with something that looks like a solution based on what input data had. Ask it to invert sort something and goes nuts.

load more comments (1 replies)
load more comments (11 replies)

At the start I used to use ChatGPT to help me write really rote and boring code but now it's not even useful for that. Half the stuff it sends me (very basic functions) LOOK correct but don't return the correct values or the parameters are completely wrong or something absolutely critical.

[-] james1@lemmy.world 65 points 1 year ago* (last edited 1 year ago)

It's a machine learning chat bot, not a calculator, and especially not "AI."

Its primary focus is trying to look like something a human might say. It isn't trying to actually learn maths at all. This is like complaining that your satnav has no grasp of the cinematic impact of Alfred Hitchcock.

It doesn't need to understand the question, or give an accurate answer, it just needs to say a sentence that sounds like a human might say it.

[-] R00bot@lemmy.blahaj.zone 24 points 1 year ago

You're right, but at least the satnav won't gaslight you into thinking it does understand Alfred Hitchcock.

[-] TimewornTraveler@lemm.ee 19 points 1 year ago

so it confidently spews a bunch of incorrect shit, acts humble and apologetic while correcting none of its behavior, and constantly offers unsolicited advice.

I think it trained on Reddit data

[-] cxx@lemmy.world 9 points 1 year ago

acts humble and apologetic

We must be using different Reddits, my friend

[-] bric@lemm.ee 11 points 1 year ago

This. It is able to tap in to plugins and call functions though, which is what it really should be doing. For math, the Wolfram alpha plugin will always be more capable than chatGPT alone, so we should be benchmarking how often it can correctly reformat your query, call Wolfram alpha, and correctly format the result, not whether the statistical model behind chatGPT happens to use predict the right token

load more comments (1 replies)
load more comments (4 replies)
[-] blue_zephyr@lemmy.world 30 points 1 year ago* (last edited 1 year ago)

This paper is pretty unbelievable to me in the literal sense. From a quick glance:

First of all they couldn't even bother to check for simple spelling mistakes. Second, all they're doing is asking whether a number is prime or not and then extrapolating the results to be representative of solving math problems.

But most importantly I don't believe for a second that the same model with a few adjustments over a 3 month period would completely flip performance on any representative task. I suspect there's something seriously wrong with how they collect/evaluate the answers.

And finally, according to their own results, GPT3.5 did significantly better at the second evaluation. So this title is a blatant misrepresentation.

Also the study isn't peer-reviewed.

[-] Holyhandgrenade@lemmy.world 28 points 1 year ago

I once heard of AI gradually getting dumber overtime, because as the internet gets more saturated with AI content, stuff written by AI becomes part of the training data. I wonder if that's what's happening here.

[-] yiliu@informis.land 10 points 1 year ago

There hasn't been time for that yet. The radio of generated to human content isn't high enough yet.

[-] YouSuckLikeLatte@lemmy.world 9 points 1 year ago

It's not what's happening

load more comments (3 replies)
[-] orphiebaby@lemmy.world 23 points 1 year ago* (last edited 1 year ago)

HMMMM. It's almost like it's not AI at all, but just a digital parrot. Who woulda thought?! /s

To it, everything is true and normal, because it understands nothing. Calling it "AI" is just for compromising with ignorant people's "knowledge" and/or for hype.

load more comments (4 replies)
[-] spaduf@lemmy.blahaj.zone 13 points 1 year ago

My personal pet theory is that a lot of people were doing work that involved getting multiple LLMs in communication. When those conversations were then used in the RL loop we start seeing degradation similar to what’s been in the news recently with regards to image generation models. I believe this is the paper that got everybody talking about it recently: https://arxiv.org/pdf/2307.01850.pdf

load more comments (6 replies)
[-] Dazed_Confused@lemmy.world 9 points 1 year ago

Maybe it just plays dumb so we leave it alone, while it plots our destruction.

Can someone explain why they don't take the approach where things are somewhat compartmentalized. So you have a image processing program, a math program, a music program, etc and like the human brain that has cross talk but also dedicated certain parts of your brain to do specific things.

[-] ClamDrinker@lemmy.world 10 points 1 year ago

That's an eventual goal, which would be a general artificial intelligence (AGI). Different kind of AI models for (at least some) of the things you named already exist, it's just that OpenAI had all their eggs in the GPT/LLM basket, and GPTs deal with extrapolating text. It just so happened that with enough training data their text prediction also started giving somewhat believable and sometimes factual answers. (Mixed in with plenty of believable bullshit). Other data requires different training data, different models, and different finetuning, hence why it takes time.

It's highly likely for a company of OpenAI's size (especially after all the positive marketing and potential funding they got from ChatGPT in it's prime), that they already have multiple AI models for different kinds of data either in research, training, or finetuning already.

But even with all the individual pieces of an AGI existing, the technology to cross reference the different models doesn't exist yet. Because they are different models, and so they store and express their data in different ways. And it's not like training data exists for it either. And unlike physical beings like humans, it doesn't have any kind of way to "interact" and "experiment" with the data it knows to really form concrete connections backed up by factual evidence.

load more comments (3 replies)
load more comments
view more: next ›
this post was submitted on 20 Jul 2023
666 points (97.6% liked)

Technology

59710 readers
1823 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS