195
submitted 5 months ago by alb_004@lemm.ee to c/technology@lemmy.world
you are viewing a single comment's thread
view the rest of the comments
[-] expr@programming.dev 25 points 5 months ago

You do not understand how these things actually work. I mean, fair enough, most people don't. But it's a bit foolhardy to propose changes to how something works without understanding how it works now.

There is no "database". That's a fundamental misunderstanding of the technology. It is entirely impossible to query a model to determine if something is "present" or not (the question doesn't even make sense in that context).

A model is, to greatly simplify things, a function (like in math) that will compute a response based on the input given. What this computation does is entirely opaque (including to the creators). It's what we we call a "black box". In order to create said function, we start from a completely random mapping of inputs to outputs (we'll call them weights from now on) as well as training data, iteratively feed training data to this function and measure how close its output is to what we expect, adjusting the weights (which are just numbers) based on how close it is. This is a gross simplification of the complexity involved (and doesn't even touch on the structure of the model's network itself), but it should give you a good idea.

It's applied statistics: we're effectively creating a probability distribution over natural language itself, where we predict the next word based on how frequently we've seen words in a particular arrangement. This is old technology (dates back to the 90s) that has hit the mainstream due to increases in computing power (training models is very computationally expensive) and massive increases in the size of dataset used in training.

Source: senior software engineer with a computer science degree and multiple graduate-level courses on natural language processing and deep learning

Btw, I have serious issues with both capitalism itself and machine learning as it is applied by corporations, so don't take what I'm saying to mean that I'm in any way an apologist for them. But it's important to direct our criticisms of the system as precisely as possible.

this post was submitted on 29 Apr 2024
195 points (94.9% liked)

Technology

58429 readers
3968 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS