this post was submitted on 04 May 2025
99 points (79.6% liked)
Technology
69845 readers
3748 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Well, that's the same situation I was in and just what I did. For that matter, Peano was also in that situation.
Not quite. It's a fundamental part of tokenization. The LLM does not "see" the individual letters. By, for example, adding spaces between the letters one could force a different tokenization and a correct count (I tried back then). It's interesting that the LLM counted 2 "r"s, as that is phonetically correct. One wonders how it picks up on these things. It's not really clear why it should be able to count at all.
It's possible to make an LLM work on individual letters, but that is computationally inefficient. A few months ago, researchers at Meta proposed a possible solution called the Byte Latent Transformer (BLT). We'll see if anything comes of it.
In any case, I do not see the relation to consciousness. Certainly there are enough people who are not able to spell or count and one would not say that they lack consciousness, I assume.
That's true. We need to observe the LLM in its natural habit. What an LLM typically does, is continue a text. (It could also be used to work backwards or fill in the middle, but never mind.) A base model is no good as a chatbot. It has to be instruct-tuned. In operation, the tuned model is given a chat log containing a system prompt, text from the user, and text that it has previously generated. It will then add a reply and terminate the output. This text, the chat log, could be said to be the sum of its "sensory perceptions" as well as its "short-term memory". Within this, it is able to distinguish its own replies, that of the user, and possibly other texts.
Can you lay out what abilities are connected to consciousness? What tasks are diagnostic of consciousness? Could we use an IQ test and diagnose people as having or not consciousness?
The brain is a physical object. Consciousness is both an emergent property and a construct; like, say, temperature or IQ.
You are saying that there are different levels of consciousness. So, it must be something that is measurable and quantifiable. I assume a consciousness test would be similar to IQ test in that it would contain selected "puzzles".
We have to figure out how consciousness is different from IQ. What puzzles are diagnostic of consciousness and not of academic ability?
I probably can't say much new, but it's a combination of memory, learning, abstract thinking, and self awareness.
I can also say that the consciousness resides in a form of virtual reality in the brain, allowing us to manipulate reality in our minds to predict outcomes of our actions.
At a more basic level it is memory, pattern recognition, prediction and manipulation.
The fact that our consciousness is a virtual construct, also acts as a shim, distancing the mind from direct dependency of the underlying physical layer. Although it still depend on it to work of course.
So to make an artificial consciousness, you don't need to create a brain, you can do it by recreating the functionality of the abstraction layer on other forms of hardware too. Which means a conscious AI is indeed possible.
It is also this feature that allows us to have free will, although that depends on definition, I believe we do have free will in an absolutely meaningful sense. Something that took me decades to realize was actually possible.
I don't know if this makes any sense to you? But maybe you find it interesting?
Yes there are different levels, actually in 2 ways. there are different levels between the consciousness of a dolphin and a human. A dolphin is also self aware and conscious. But it does not have the same level of consciousness we do. Simply because it doesn't posses the same level of intelligence.
But even within the human brain there are different levels of consciousness. It's a common term to use "sub conscious" and that is with good reason. Because there are things that are hard to learn, and we need to concentrate hard and practice hard to learn them. But with enough practice we build routine, and at some point they become so much routine we can do them without thinking about it, but instead think about something else.
At that point you have trained a subconscious routine, that is able to work independently almost without guidance of you main consciousness. There are also functions that are "automatic", like when you listen to sounds, you can distinguish many separate sounds without problem. We can somewhat mimic that in software today, separating different sounds. It's extremely complex to do, and the mathematics involved is more than most can handle. Except in our hearing we do it effortlessly. But there is obviously an intelligence at work in the brain that isn't directly tied to our consciousness.
IDK if I'm explaining myself well here, but the subconscious is a very significant part of our consciousness.
That is absolutely not a certainty. At least I don't think we can at this point in time, but in the future there may exist better knowledge and better tools. But as it is, we have been hampered by wrongful thinking in these areas for centuries, quite opposite to physics and mathematics that has helped computing every step of the way.
The study of the mind has been hampered by prejudice, thinking that humans are not animals, thinking free will is from god, with nonsense terms like id. And thinking we have a soul that is something separate from the body. Psychology basically started out as pseudo science, and despite that it was a huge step forward!
I'll stop here, these issues are very complex, and some of the above issues have taken me decades to figure out. There is much dogma and even superstition surrounding these issues, so it used to be rare to finally find someone to read or listen to that made some sense based on reality. It's seems to me basically only for the past 15 years, that it seems that science of the mind is beginning to catch up to reality.