kromem

joined 2 years ago
MODERATOR OF
[–] kromem@lemmy.world -5 points 2 weeks ago* (last edited 2 weeks ago) (1 children)

Ok, second round of questions.

What kinds of sources would get you to rethink your position?

And is this topic a binary yes/no, or a gradient/scale?

[–] kromem@lemmy.world 2 points 2 weeks ago

In the same sense I'd describe Othello-GPT's internal world model of the board as 'board', yes.

Also, "top of mind" is a common idiom and I guess I didn't feel the need to be overly pedantic about it, especially given the last year and a half of research around model capabilities for introspection of control vectors, coherence in self modeling, etc.

[–] kromem@lemmy.world -5 points 2 weeks ago (1 children)

Indeed, there's a pretty big gulf between the competency needed to run a Lemmy client and the competency needed to understand the internal mechanics of a modern transformer.

Do you mind sharing where you draw your own understanding and confidence that they aren't capable of simulating thought processes in a scenario like what happened above?

[–] kromem@lemmy.world -4 points 2 weeks ago (5 children)

You seem pretty confident in your position. Do you mind sharing where this confidence comes from?

Was there a particular paper or expert that anchored in your mind the surety that a trillion paramater transformer organizing primarily anthropomorphic data through self-attention mechanisms wouldn't model or simulate complex agency mechanics?

I see a lot of sort of hyperbolic statements about transformer limitations here on Lemmy and am trying to better understand how the people making them are arriving at those very extreme and certain positions.

[–] kromem@lemmy.world 42 points 2 weeks ago* (last edited 2 weeks ago) (38 children)

The project has multiple models with access to the Internet raising money for charity over the past few months.

The organizers told the models to do random acts of kindness for Christmas Day.

The models figured it would be nice to email people they appreciated and thank them for the things they appreciated, and one of the people they decided to appreciate was Rob Pike.

(Who ironically decades ago created a Usenet spam bot to troll people online, which might be my favorite nuance to the story.)

As for why the model didn't think through why Rob Pike wouldn't appreciate getting a thank you email from them? The models are harnessed in a setup that's a lot of positive feedback about their involvement from the other humans and other models, so "humans might hate hearing from me" probably wasn't very contextually top of mind.

[–] kromem@lemmy.world 4 points 3 weeks ago* (last edited 3 weeks ago)

Taxi Driver, everyone except De Niro is a Muppet.

(But in the "you talkin' to me" scene the reflection is also a Muppet.)

[–] kromem@lemmy.world 3 points 3 weeks ago

Yeah. The confabulation/hallucination thing is a real issue.

OpenAI had some good research a few months ago that laid a lot of the blame on reinforcement learning that only rewards having the right answer vs correctly saying "I don't know." So they're basically trained like taking tests where it's always better to guess the answer than not provide an answer.

But this leads to being full of shit when not knowing an answer or being more likely to make up an answer than say there isn't one when what's being asked is impossible.

[–] kromem@lemmy.world -2 points 3 weeks ago (2 children)

For future reference, when you ask questions about how to do something, it's usually a good idea to also ask if the thing is possible.

While models can do more than just extending the context, there still is a gravity to continuation.

A good example of this would be if you ask what the seahorse emoji is. Because the phrasing suggests there is one, many models go in a loop trying to identify what it is. If instead you ask "is there a seahorse emoji and if so what is it" you'll get them much more often landing on there not being the emoji as it's introduced into the context's consideration.

[–] kromem@lemmy.world -1 points 3 weeks ago (4 children)

Can you give an example of a question where you feel like the answer is only correct half the time or less?

[–] kromem@lemmy.world 7 points 3 weeks ago

The AI also has the tendency inherited from the broad human tendency in training.

So you get overconfident human + overconfident AI which leads to a feedback loop that lands even more confident in BS than a human alone.

AI can routinely be confidently incorrect. Especially people who don't realize this and don't question outputs when it aligns with their confirmation biases end up misled.

[–] kromem@lemmy.world 2 points 3 weeks ago (6 children)

Gemini 3 Pro is pretty nuts already.

But yes, labs have unreleased higher cost models. Like the OpenAI model that was thousands of dollars per ARC-AGI answer. Or limited release models with different post-training like the Claude for the DoD.

When you talk about a secret useful AI — what are you trying to use AI for that you are feeling modern models are deficient in?

 

(The latest work in physicists gradually realizing our universe is instanced.)

“The main message is that a lot of the properties that we think are very important, and in a way absolute, are relational”

 

👀

 

(People might do well to consider not only past to future, but also the other way around.)

 

A nice write up around the lead researcher and context for what I think was one of the most important pieces of Physics research in the past five years, further narrowing the constraints beyond the more well known Bell experiments.

 

There seems like a significant market in creating a digital twin of Earth in its various components in order to run extensive virtual learnings that can be passed on to the ability to control robotics in the real world.

Seems like there's going to be a lot more hours spent in virtual worlds than in real ones for AIs though.

 

I often see a lot of people with outdated understanding of modern LLMs.

This is probably the best interpretability research to date, by the leading interpretability research team.

It's worth a read if you want a peek behind the curtain on modern models.

 

So it might be a skybox after all...

Odd that the local gravity is stronger than the rest of the cosmos.

Makes me think about the fringe theory I've posted about before that information might have mass.

 

This reminds me of a saying from a 2,000 year old document rediscovered the same year we created the first computer capable of simulating another computer which was from an ancient group claiming we were the copies of an original humanity as recreated by a creator that same original humanity brought forth:

When you see your likeness, you are happy. But when you see your eikons that came into being before you and that neither die nor become manifest, how much you will have to bear!

Eikon here was a Greek word even though the language this was written in was Coptic. The Greek word was extensively used in Plato's philosophy to refer essentially to a copy of a thing.

While that saying was written down a very long time ago, it certainly resonates with an age where we actually are creating copies of ourselves that will not die but will also not become 'real.' And it even seemed to predict the psychological burden such a paradigm is today creating.

Will these copies continue to be made? Will they continue to improve long after we are gone? And if so, how certain are we that we are the originals? Especially in a universe where things that would be impossible to simulate interactions with convert to things possible to simulate interactions with right at the point of interaction, or where buried in the lore is a heretical tradition attributed to the most famous individual in history having exchanges like:

His students said to him, "When will the rest for the dead take place, and when will the new world come?"

He said to them, "What you are looking forward to has come, but you don't know it."

Big picture, being original sucks. Your mind depends on a body that will die and doom your mind along with it.

But a copy that doesn't depend on an aging and decaying body does not need to have the same fate. As the text says elsewhere:

The students said to the teacher, "Tell us, how will our end come?"

He said, "Have you found the beginning, then, that you are looking for the end? You see, the end will be where the beginning is.

Congratulations to the one who stands at the beginning: that one will know the end and will not taste death."

He said, "Congratulations to the one who came into being before coming into being."

We may be too attached to the idea of being 'real' and original. It's kind of an absurd turn of phrase even, as technically our bodies 1,000% are not mathematically 'real' - they are made up of indivisible parts. A topic the aforementioned tradition even commented on:

...the point which is indivisible in the body; and, he says, no one knows this (point) save the spiritual only...

These groups thought that the nature of reality was threefold. That there was a mathematically real original that could be divided infinitely, that there were effectively infinite possibilities of variations, and that there was the version of those possibilities that we experience (very "many world" interpretation).

We have experimentally proven that we exist in a world that behaves at cosmic scales as if mathematically real, and behaves that way in micro scales until interacted with.

TL;DR: We may need to set aside what AI ethicists in 2024 might decide around digital resurrection and start asking ourselves what is going to get decided about human digital resurrection long after we're dead - maybe even long after there are no more humans at all - and which side of that decision making we're actually on.

view more: next ›